Overview

E2E tests are missing in XB AI and it should be implemented to reduces chances of regression after updates or refactoring.

Proposed resolution

User interface changes

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

narendrar created an issue. See original summary.

narendrar’s picture

Status: Active » Needs review

I have added tests and it is ready for review. However it is blocked on #3538439: Running tests locally is difficult to match with the CI

narendrar’s picture

Assigned: narendrar » Unassigned

The blocker has been resolved. I have rebased with the latest changes, and this issue is now ready for review.

yautja_cetanu’s picture

yautja_cetanu’s picture

Possibly a follow up issue.

kunal.sachdev’s picture

This provides a good foundation!! We can include additional tests for page builder tasks and other cases in a follow-up.

kunal.sachdev’s picture

Status: Needs review » Reviewed & tested by the community

tim.plunkett made their first commit to this issue’s fork.

tim.plunkett’s picture

Status: Reviewed & tested by the community » Fixed

Merged into 1.x! Not backporting because #3538439: Running tests locally is difficult to match with the CI wasn't backported.

wim leers’s picture

Status: Fixed » Needs work

This caused a regression on 1.x: it is a new flaky test.

I've now seen these new tests fail in multiple MRs, always like this:

  3) [webkit] › tests/src/Playwright/xbAi.spec.ts:127:7 › XB AI Features › Should verify AI panel API request and response payload 
    Error: expect(received).toBe(expected) // Object.is equality
    Expected: "1"
    Received: "2"
      153 |     expect(requestBody.messages[0].text).toBe(query);
      154 |     expect(requestBody.entity_type).toBe('xb_page');
    > 155 |     expect(requestBody.entity_id).toBe('1');
          |                                   ^
      156 |     expect(requestBody.selected_component).toBe('');
      157 |     expect(requestBody.layout).toBe('{}');
      158 |     expect(requestBody.active_component_uuid).toBe('');

https://git.drupalcode.org/project/experience_builder/-/jobs/6274833#L1337

While it's still fresh in your head, please harden this 🙏

wim leers’s picture

Also, many more failures from this new test coverage in https://git.drupalcode.org/project/experience_builder/-/jobs/6275720 (4 of the 6 failures!) — although maybe those failures are caused by the changes in that MR, although I really don't see how.

I'll re-test it (queued: https://git.drupalcode.org/project/experience_builder/-/jobs/6275778), but would very much appreciate you manually testing the AI functionality with that MR checked out. 🙏

justafish made their first commit to this issue’s fork.

jessebaker made their first commit to this issue’s fork.

jessebaker’s picture

Status: Needs work » Fixed
wim leers’s picture

Much appreciated, @NarendraR, @justafish and @jessebaker! 😊🙏

jvbrian’s picture

jvbrian I deleted my push because I accidentally created my test file in the wrong issue. Since I’m creating new Playwright tests, they should be in issue #3537161.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.