[Spike] Research PDF upload support for CCC

Please be aware of #3528673: Create Document Loader Normalization Layer. When that is solved, this issue should be a lot simpler.

Quality of PDF to MD varies greatly depending on the tools used. If you use a frontier multimodal model (supporting PDF) they can do it, but simpler PDF to MD tools often do stuff like write the header and footer "copyright someone someone" to the MD output for every page.

(which would pollute the context)

I've tested tools like https://github.com/datalab-to/marker which need a LLM to run but I think we'll need {#3528673} to really be done before this can be used.

Can do some research around this, while paired with #3528673: Create Document Loader Normalization Layer

ETD 1w

Hi @hristem,

AI File to Text module currently supports PDF to MD conversion. While the Smalot\PdfParser (a pure PHP parser) may not provide perfect quality, it performs adequately. Table conversion presents challenges, though, because tables are just positional segments of (Y/X), making consistent results impossible.

Poppler support has been added as an option. However, a lot of hosting environments may not support the requiring system dependencies. So this is as far as the maximum capability achievable without relying on APIs - LLMs paid, or heavy system dependencies.

The issue mentioned in #15 may occur, as it's a non-AI API.

We are planning to add a document loader (#3573054) to the AI file-to-text module, which I think is worth checking out.

Most excellent, thanks. In the mean time, I've pushed the minimal one that I had put together over at:
https://www.drupal.org/project/document_loader_pdfparser

It aims to only introduce the PDF Plugin, with minimal dependencies. Can add all as maintainers, just let me know.

Other Thoughts:

• Many instances of Drupal I've found used Media to upload and select PDFs
• Within AI Context, any context should be able to have multiple files
• CKEditor does have Markdown Output support, with the CKEditor Markdown Editor module
• CKEditor also has a few plugins to reference uploads/Media too
• I understand we'd need MDXEditor for a more React-like interface elsewhere in AI, but AI Context is just a form, not an interactive chat window. Something to consider.

Spoke through this today at the stand up. A few possible approaches, each using similar modules...

1. AI File To Text

The AI File To Text module leverages AI tooling to port a file to text.

1. Add AI File To Text module
2. Add a File Field to the Context Item Entity
3. Introduce a Field Widget Action and Automator that populates the Context Entity's Content upon Save, using the AI File To Text's Automators

2. AI File To Text with Document Loader

The Document Loader module introduces a plugin architecture to instruct transformations between any kind of document type, even web scraping. This is a similar approach taken with other AI solutions, like LangChain's Document Loaders.

1. Handle AI File To Text issue: #3573054: Create Document Loader Plugin
2. Add AI File To Text module, along with Document Loader
3. Add a File Field to the Context Item Entity
4. Add a Field Widget Action that leverages File To Text's Document Loader architecture to populate the Context Entity's Content upon Save

3. Document Loader, with AI Automators

While we have the Document Loader module, there currently isn't AI Tooling directly introduced for it. This would enable transforming pretty much any document format directly through AI Automators.

1. Handle Document Loader issue: #3573331: Add standalone Automators for file to formatted text and JSON outputs
2. Add Document Loader module
3. Add a File Field to the Context Item Entity
4. Add a Field Widget Action that uses Document Loader Automators to populate the Context Item's Content field

Summary

All three approaches are similar, leveraging automators and field widget actions to fill in the content of the context entities. The benefits of leveraging Document Loader for this is that it's extendable through Plugins, even allowing scraping webpages. Given that though, we are currently not blocked on moving forwards with having a File Field attached to the Context Entity, even with AI File To Text, since the Document Loader architecture will eventually be introduced to it.

Just pushed up a hardcoded demonstration of using File To Text in Merge Request !66:

Again, this is just a quick demonstration, optimally we would take on one of the above approaches in #29...

• Use Automators/Field Widget Actions
• Adopt Document Loaders to allow a pluggable architecture around loading the different documents
• Have it append the content to the end so that it doesn't eat all the Context's content

My preference is that I think you should do Document Loader, so its future proof to use enterprise document loaders like Unstructured.io or Docling.

I think you should keep dependencies at a minimum and solve it without Automators/FWA - one thing to note and to think about as well is loading animations. Unstructured's best models can take up to a minute to get everything correctly handled.

I'm wondering on a later stage if we want to actually upcast the MDXEditor so it can loads file in general when the document loader is installed. It would be cool if you could drag a file to the textarea and its filled in :)

I had one project a while ago where the site had 100MB PDFs. We needed to use a background task on cron to process them. While I'm sure that kind of thing is out-of-scope for ai_context, I'm sure there are users who will run into it.