Add a static generation tool to core

Whilst I agree with the idea, I'm not sure wiring up all the moving pieces makes this any more approachable to hobbyists than running a lamp server

+1 for this great idea @nod_, thanks for posting it: It would re-open a more or less lost segment of Drupal web sites on limited budgets.

I really fear the upcoming end of Drupal 7, if from then on it requires a security team on stand by to roll out announced, or even unexpected security updates. This could lead to a big reduction of web sites running Drupal, and thereby community activity, starting a downward spiral.

A static generation tool in core would be a move in the right direction, to allow building and running a Drupal web site on a limited budget.

The core idea here is to make use of our incredible cache-tags implementation to leapfrog the other CMSs in the static generation space.

Love it.

I'm not sure wiring up all the moving pieces makes this any more approachable to hobbyists than running a lamp server

1. But, at least in this scenario the user won't have to bear the responsibility of having to keep Drupal core+modules+dependencies up-to-date! That's still a HUGE reduction in complexity :)
2. This is something that in theory even a non-profit like the DA could offer the hosting for … because the hosting cost is minimal. The only responses served are those for the admin UI. The export can be downloaded. If you combine that with a limit such as max Node entities, it may keep cost sufficiently low? (I know, I'm getting ahead of myself…)
   EDIT: hah, this is basically what @nod_ already refers to in #9!

I'm very excited by this idea! 🤓🤩

I love the idea of solving the static generation use case really well, whether that's fully in core, or whether it's a contrib module or distro, but where core implements whatever additional capabilities are needed to enable it to "just work".

The API part is a nice to have, so if that sidetrack too much the issue I'll open a different issue.

+1 to moving this part to a separate issue, but since that hasn't been done yet, I'll comment on it here :)

The last point is not the core of this proposal, only a nice side-effect, and could be a starting point to re implement some of Drupal features in a different stack (go? rust? js? java :p?)

Personally, I would be pretty excited about a lightweight comment submission and/or webform submission handler in JS that I could host on something like Cloudflare Workers, but where I can still use Drupal to configure the comment fields and/or build the webform how I like. This could even be something that isn't part of Drupal core, just available on npm.

I'm curious how we could make Views available with this. At least for the Views arguments whose values are bounded (e.g., anything from a drop-down), could we statically generate those pages? And perhaps if we know that you're building for the static generation use case (because you configured that somewhere), should we remove the ability to add Views arguments that are unbounded (like freeform text)?

#13: Good points! That sounds like a job for validation constraints that are added conditionally (i.e. a "validation" step prior to generating a static site: "can we generate a fully functional static site for the current configuration?") 🤓

I agree this could be an awesome addition to Drupal. Something like views might be challenging if the dataset is too large. Static filter should be reasonable to implement though, making the "backend" swappable.

I can see quite a few sites we've build work perfectly well as a static site. Especially if you mix in a few components for anything relatively dynaemic you might need.

Which would be really awesome to have. Nice idea this ^^

Smartsheet would love this dearly. I will help wherever I can. Let me know.

However, the views case is problematic. "choices are bound" is not enough, alas. Math is not your friend. Multiplication especially.

https://www.smartsheet.com/marketplace/apps this is just a few hundred nodes presented with a few facets and two sorts but if you want to turn this into a static page, under Type you have 16 valid combinations, under Category 8 and under Level another 4 and multiply by another two because of sorts. This single page explodes into 1024 pages. If you are unlucky enough to have something like this as a sidebar then every page might generate a thousand more.

I really really would love to see static pages but I would strongly recommend against having exposed filters in the MVP. Ie the answer to "can we generate a fully functional static site for the current configuration?" would be "core can't if exposed filters are present" -- at least at the beginning.

Sure you could move it back, but i was not talking about generating every possibility, but filtering a set of data represented in the source. So the html would be the unfiltered list and filtering done in the client.

Anyways, that part of the discussion is quite far into possible implementation details. :)

+1 if we could get generic enough to generate static pages for non-interactive components and at minimum standards/documentation around interactive ones (autocomplete, contact form, AJAX callbacks, etc) that would be a huge win. I have had an experience where D7 projects (and some of the early D8 projects) moved to JS frameworks just because we miss this feature.

1) Local dev or GitPod -> Static site
This is one of the coolest parts of Tome, IMHO... the ability to spin up an "ephemeral" Drupal site for content creation only, and then the static results are what is deployed. Tome already has a very lightweight Electron app to "start/stop/publish", and that model could easily be extended. In a world where ANYONE can download a Drupal content "app", manage things, create/export/publish content, we could bypass many of the issues with learning curves and get people started today.

2) Static content tied to cache
Drupal's caching system is one of the best in the world, as far as I know. Tying static creation into the cache system seems like a very natural extension (imagine Boost in D9). This means that with some basic configuration, we can leverage the existing tools.

3) Views
Without a dynamic server, any exposed filters (ajax calls) will not work. Tome gets around this by using JS libraries and then outputting the data as JSON so it can be filtered in the client. This is limited, but works for smaller use cases. Larger systems can look at API only access for live sites, SOLR or other search integration, etc. Potentially I can see how we could get really crazy with outputting large JSON groups and making a "fake" API on the static site, but not a problem to worry about today

4) Easy configuration
This is critical. The ability to not only enable static generation but also to set up the destinations (like GH pages, S3, etc.) will be critical. We don't need to support everything, but a handful of resources for free/inexpensive static hosting is crucial.

5) Time machine
Some organizations have legal and regulatory requirements to archive all of their content and be able to go back and look at it across different times/days. So, for example, a financial services company may need to prove that on October 11, 2021 their nine month CD had an interest rate of 1.35% (not 1.5%. By having a static site generation tool, they could push all updates and changes to a Git repo, and have a full archive with built in auditing.

6) Security
One of the biggest reasons I have seen for wanting a static site is security. A static site can't be hacked... especially if the generation tool (Drupal) *doesn't exist* except temporarily on someone's local computer.

Overall, I think this could be a massive feature for us. It could make it easy for anyone to spin up a Drupal site and instantly create/publish content without needing to write/deploy/composer any code. It is also a great feature for developers who want security, performance, or even a JAMStack application. This has something for everyone.

Yes. Yes. Yes!!

So many opportunities here to help onboard new community members by making it possible to “build for free”.

Traditional Drupal is expensive to host. All for good reasons. Being able to have an “out of the box” static generator, plus a widely understood and agreed best practice for actually generating and hosting, means we make Drupal approachable by people who simply cannot consider trying it out right now.

Love it. How do I help?

I have two sites that I'm still responsible for that started on Drupal 4.7 and are currently on Drupal 7, both of them could use this and it would reduce their hosting costs (and my headaches) considerably.

Make aggregated assets URLs more predictable (to avoid having to crawl the generated HTML)

#1014086: Stampedes and cold cache performance issues with css/js aggregation would get us part of the way there. Although we might want to look at the aggregate bundling logic on top of that to reduce the potential for large numbers of variations, there's some very old issues about it, nothing up-to-date that I can remember.

Edit: the old issues:

#721400: Order JS files according to weight, don't change filenames for aggregated JS/CSS
#734080: Set preprocess: false for jquery.min.js to reduce duplication between asset aggregates
#769226: Optimize JS/CSS aggregation for front-end performance and DX

Image style generation (on upload, instead of on demand for example)

Haven't used it, but there's https://www.drupal.org/project/image_style_warmer. Generation directly on upload I'd be sceptical about adding to core, but a queue would be fine (then presumably whatever export step could run down the queues)? This seems worth opening an issue for now to discuss in its own right.

@catch: This is exactly what my Image Style Warmer module provides. You could generate configured image styles on image upload or via a queue. It also provide a drush command for this, which currently need some more flexibility and tests.

I like the idea of providing this as a hosting solution for customers a little bit like wordpress.com, but with static generated sites.

Smartsheet uses image style warmer, it works.

I love all the enthusiasm for this issue, I've used Tome on a few personal projects and to 'archive' older versions of websites. It would really be amazing to have something like this in core. Game changer.

Same here, my experience with Tome for a "modest" website was really good, and having something like that in core would indeed be a game changer. What an awesome idea!

🔥 It would be a really useful tool for most of small Drupal websites. Just think of all the money saved for not having to use servers - instead one could use free github pages, netlify etc

The SSG segment is currently full of already evolved participants, let NextJS Gatsby etc. already be mature solutions and they can be easily adopted using data plugins (e.g. for Drupal). Those tools involve state-of-the art technologies like React with already well working optimisations and goodies like content preloading, and this is what makes it so attractive for many people.

Tome is great and I love it too, but it has an accessibility problem. You need to understand Drupal well in order to get Tome work right for you. And we all know that Drupal's learning curve is steep. If that segment is about to be addressed by Drupal, this should also be tried to be solved in a perspective of what the Project Browser inititative is currently trying to accomplish - lowering accessibility hurdles. If Drupal can provide a nice and easy way to setup a scheduled incremental SSG that is also usable for editors (I'm currently thinking about Workspaces and the Preview mode here), then it may be a good chance to let this ecosystem be more attractive again.

For static search functionality there's https://lunrjs.com/.
Their tagline is: "Lunr: A bit like Solr, but much smaller and not as bright." :)

I haven't used it myself with Drupal yet, it is used by our project's documentation system.

I just learnt that there is a Drupal module for it, with tome support: https://www.drupal.org/project/lunr

+1 on this. Drupal hosting is really expensive, and that keeps smaller sites away. Static site generation would go a long way to make this better.

It would be a good to have option.
Been trying to have a static site with various JS Frameworks but they are too bolted for simple static sites.
And it also helps to onboard small sites to Drupal as JS Framework echo system changes very rapidly which makes hard to keep track of JS frameworks But Drupal can provide stable environment for them.

I recently tried Bookish, and was extremely impressed by its many features:

Bookish is an install profile for Drupal 9+ that tries to make the out of the box experience for Tome users as nice as possible.

https://github.com/drupal-tome/bookish

For more details, see the blog post Meet Bookish, an install profile for static Drupal blogs by Samuel Mortenson.

I would be interested in contributing to this idea.

Especially in projects of the public hand, this would be a game changer. Often, there is a security requirement that the content editors only work in a secured environment, e.g., a VPN, and the outside world only sees a static version.

So, how can I help?

Yes - this would be a good thing to make progress on. As stated, there is a definite need and core contains much of what is needed.

I recently came across "FrankenPHP" which has a method for compiling a PHP app to a stand alone binary. So - consider that the Drupal instance is only needed for creation and editing, with the static output going to S3, GH pages, GL pages, etc.

If set up properly, then each author could simply add their content as a MR to the repo, and manage any edits or new content with Git. The DA could offer a free website to all users with GL Pages, and could offer a low cost solution for non-profits or smaller orgs. That would not only spread Drupal, but also help the DA generate a bit more funding.

Yeah, one could work locally with DDEV or with a binary (both desktop or in the browser, see https://mglaman.dev/blog/running-drupal-edge-webassembly) and commit the changes to some repository or data availability protocol in general (Github, traditional or decentralized hosting).

The Ideas project is being deprecated. This issue is moved to the Drupal project. Check that the selected component is correct. Also, add the relevant tags, especially any 'needs manager review' tags.