[Meta] Fine-tune small Open Source Drupal specific LLMs

This will eventually need its own module but it comes out of the work here.

[Tracker]
Update Summary: [One-line status update for stakeholders]
Short Title: Fine-tune small Open Source Drupal specific LLMs
Short Description: A suite of tools that can harness submitted Drupal CMS tests to fine-tune opensource modules that will allow self-hosted AI that is more secure, private, cheaper and more effective for Drupal CMS AI Agents.
Check-in Date: MM/DD/YYYY
Due Date: MM/DD/YYYY
Blocked by: [#XXXXXX] (New issues on new lines)
Additional Collaborators: @username1, @username2
Metadata is used by the AI Tracker. Docs and additional fields here.
[/Tracker]

Problem/Motivation

Large Language models are expensive, smaller models can be run locally, are cheaper, arn't centrally controlled and allow for sovreignty and keep the data inhouse. However they are significantly less effective.

The data used to build a list of tests for the Drupal CMS AI Agents could also be used to train a small opensource model that could be locally run. Here are the steps of how we can get there.

Requires: #3562638: [Meta] Explosion of AI Agent Tests. (Easier to build, export, import and run tests]

Proposed resolution

• Create a tool that takes a series of tests that we know work and convert it into a format for training LLMs
• Create an abstraction layer to allow Drupal to connect to many different fine-tuning providers.
• Use the data to select a model and train it.
• Figure out how we store and manage these models the community produces.
• Figure out how we manage the community what models that can download and use for what purpose and how (when smaller models will only work for a very specific set of use cases.
• Create a tool that speeds up the workflow of training models and then immediately testing them with a different dataset.
• Also explore bringing RLHF training into the workflow, allowing people to click thumbs up and thumbs down and train a smaller model.
• Explore real-time RLHF training. Individuals on the live site can click thumbs up and thumbs down and their own personal model adapts to their needs.

Remaining tasks

User interface changes

API changes

Data model changes