--- AI TRACKER METADATA ---
Update Summary: Add AI context item usage data and page
Additional Collaborators: @kristen_pol
AI Tracker found here: https://www.drupalstarforge.ai/
--- END METADATA ---
Problem/Motivation
In order to update entities later, we need to know what context was used when touching those entities.
Steps to reproduce
Proposed resolution
Create usage data: id, date, context item id, agent machine name, entity id (if relevant)
- If the context is used for the agent and entity id pair, update the date
- Add a Usage page that shows up to the left of the Settings page and has its own permission
This mapping will be used when updating entities later when context changes
Remaining tasks
- Add schema and code
- Test and review
- Commit :)
User interface changes
See example images added in comments
API changes
Data model changes
Has own database schema
AI usage
The code associated with this issue was developed using AI tools.
A human worked with the AI tool to:
- Clarify the requirements
- Spec out the plan and architecture
- Create the code
The human:
- Reviewed the code
- Manually tested the functionality
- Requested adjustments from AI
- Adjust the code themselves when necessary
Issue credit is still relevant for humans working with AI as they are using AI as a tool and not to do everything for them. It requires human expertise and oversight to utilize AI effectively and efficiently in addressing these issues.
| Comment | File | Size | Author |
|---|---|---|---|
| #51 | ccc-usage-record.png | 232.7 KB | kristen pol |
| #51 | ccc-usage-list.png | 160.5 KB | kristen pol |
| #46 | usage_record.png | 58.15 KB | scott falconer |
| #46 | usage_page.png | 63.05 KB | scott falconer |
| #34 | ccc-usage-settings.png | 258.73 KB | kristen pol |
Issue fork ai_context-3550034
Show commands
Start within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
Comments
Comment #2
kristen polComment #3
kristen polComment #4
kristen polPostponing until we've prioritized in the roadmap
Comment #5
kristen polComment #6
kristen polI'll take a look at this one now.
Comment #7
kristen poldeleted... wrong issue
Comment #8
kristen polHere's an example of a usage table:
Comment #9
kristen polAnother example:
Comment #10
kristen polComment #11
kristen polComment #12
kristen polComment #13
kristen polComment #14
marcus_johansson commentedComment #15
kristen polComment #16
kristen polComment #17
kristen polBeen focused on other things, so unassigning this one for now
Comment #18
marcus_johansson commentedComment #19
kristen polI'm going to take a look at this now for the sprint and decide if I want to hand it off.
Comment #20
kristen polComment #21
kristen polBeen planning with Cursor.
The tricky bit is how do we know what entities have been touched?
Here's an implementation plan from Cursor (lightly updated by me):
https://docs.google.com/document/d/19GUjbEmIgxy-dGlKgeozTiRqZWU30NjfnVOU...
Are we going to be able to get the entity ids from
AgentToolFinishedExecutionEvent?Comment #22
kristen polPushed up some starting code, which I've reviewed, but I need to test
But see the Google Doc above first for some questions
New files:
Modified files:
Data tracked per record:
Comment #24
kristen polI'm still testing and fixing.
Comment #25
kristen polFixed some bugs and updated the functionality. Here's what it looks like so far.
Comment #26
kristen polSwitched to using a view
Comment #27
kristen polDone for now.
Main question: How to get the entity ID if it's not part of the route?
Comment #28
rakhimandhania commentedComment #29
kristen polComment #30
kristen polFrom meeting with Aidan+Emma:
At minimum, limit the the number of records so database isn't flooded
Need to discuss the difference between deep logging for debugging vs usage for entity association.
Follow up issue on discussion issue for using the observability API.
Comment #31
kristen polNeeds work for: At minimum, limit the the number of records so database isn't flooded
Here's the observability issue: #3567803: [Discuss] Observability support for CCC
I posted that in a few channels to see if people will chime in.
Comment #32
kristen polComment #33
kristen polComment #34
kristen polAdded a settings page and code to purge records.
Usage page
Settings page
Comment #35
bbruno commentedComment #36
bbruno commentedI'll leave the code review for someone more on the BE-side, but on the UX/frontend side everything is looking good to me. Seems like a real nice addition to the module and everything is clear on how to use it.
Comment #37
kristen polThanks, Bruno!
Comment #38
kristen polComment #39
kristen polThis can be used as a starting point if we don't do anything with the observability API before 1.0.
Comment #40
emma horrell commentedComment #41
afoster commentedEmma and reviewed and #34 looks good. We're assuming the main use case we're using these logs for is:
"why does my page suck"... Hmm is it the context or the agent prompt or what...?
Table Update suggestions:
1. Agent should link to the Agent config page
2. Entity should be a link to the rendered page / entity.
3. Add a view link to view single log record.
Create a "entity view" for each log item record. (Similar to looking at a single Drupal watchdog log)
1. Move Route info from the table into single log item view
2. Print the actual Prompt / Context info here (so I can see if something unexpected was attached)
3. Maybe a history of calls, not the full logs but if it ran 50 times on a single call I would like know if it's running
4. Token usage
Comment #42
kristen polThanks!
Comment #43
kristen polUnassigning for now in case someone else wants to pick this up
Comment #44
scott falconer commentedTaking a look at this one.
Comment #45
kristen polGreat! Merge in the latest from 1.0.x and note the new installation instructions on the project page don’t have the steps for setting up the agent side of things. I was testing with the v2025 demo when creating this
Comment #46
scott falconer commentedUpdated MR !35:
https://git.drupalcode.org/project/ai_context/-/merge_requests/35
This adds the remaining UX items discussed in the thread (agent/entity links + single record view).
Changes
- Usage view: Agent column links to the agent config form.
- Usage view: Entity column links to the target entity (when available).
- Usage view: Added a "View" link column to open a single usage record page (watchdog-style).
- Single record page (/admin/ai/context/usage/): shows Tools used, Routes, and renders the current Context item content.
- Moved Routes out of the table into the single record page (kept the Route filter exposed).
- Merged latest from 1.0.x.
- Update hook: updates the ai_context_usage View display config on existing sites.
- Update hook: installs the new ai_context_item created field storage on existing sites (prevents missing column errors).
- Composer: ensure dynamic_entity_reference is required (module still uses this field type).
- Config schema: added scheduler third_party_settings schema for ai_context_item_type.
- Added a Kernel test for the usage tracker service.
- Docs: clarified AI Agent setup steps in README (create agents at /admin/config/ai/agents).
Quick verification
- Visit /admin/ai/context/usage and confirm Agent/Entity link behavior and the new "View" link column.
- Click "View" for a record and confirm the single record page renders Tools used, Routes, and Context item content.
Testing (local DDEV)
- PHPCS (Drupal, DrupalPractice): PASS
- PHP lint (changed files): PASS
- PHPUnit (ai_context module tests): PASS (49 deprecations on Drupal 11.3.x)
- drush updb / cr: PASS
- drupal-intent-testing compare run: IDENTICAL (0 JS errors, 0 Drupal alerts); screenshots attached
Related
- Installation error "The state 'draft' already exists in workflow" is tracked in https://www.drupal.org/project/ai_context/issues/3571188 (MR !51). Tested: enabling ai_context fails without MR !51 and succeeds with it on a fresh Drupal 11 install. This MR intentionally does not include that fix.
AI disclosure
This work was AI-assisted (analysis/drafting), with human review + local testing.
Comment #47
robloachI can do a review of this one.
Comment #48
robloachWhen visiting admin/ai/context/usage, I'm getting a database error of the missing ai_context_usage table...
This is after I ran the database updates...
ddev drush updatedbddev drush crddev launch admin/ai/context/usageNeed the schema definition?
Comment #49
scott falconer commentedUpdated MR !35:
Addressed the upgrade regression where visiting /admin/ai/context/usage failed with a missing ai_context_usage table after running updates.
Changes in this update
Reproduction and verification
AI disclosure
This work was AI-assisted (analysis/drafting), with human review + local testing.
Comment #50
kristen polThanks! I'll take a look.
Please note that this module is NOT used other than for testing, so I have intentionally NOT been including update hooks. We are on the "innovation" side right now and not doing all the standard things we'd do if this were already a used module. The "alpha release" was simply a tag to let people try the Vienna DrupalCon version with the v2025 repo. The code has been pretty much rebuilt since that time.
Comment #51
kristen polMerged! Works great, and love the new usage record page. Thanks!
Comment #53
kristen polComment #54
kristen pol