The 2026 Drupal Business Survey is open. Your response will help shape the future of the Drupal business ecosystem.On #2539918: Automate creation of screenshots we are working on automating screen shot creation for the User Guide. The process there uses an automated test that, kind of as a by-product, produces screen shots.
It might be useful to expand on the test so that it tests more of the User Guide itself. For instance, the first step of most Task topics is a navigation sequence that takes you to a particular admin page, by clicking on, for example, "Structure" and then "Content types". The screen shot test generally would skip the intermediate clicks and just go directly to path admin/structure/types. Which means that it doesn't verify that this part of the UI hasn't changed. (On the other hand, many of the other steps in the guide are verified, which is good.)
So, we could actually put in these navigation steps, and other test assertions. What this would buy us is that if the UI did change, for instance in 8.3.x, the test would fail and we would know that we needed to update the User Guide. Otherwise, we would be relying on someone noticing that the guide is out of synch with Drupal, and filing an issue. Automated tests are more reliable than that.
So... this issue is about putting more into the screenshot automation test, so that more of what the guide tests is verified, and we will have a better chance of noticing that we need to update the Guide (due to a failing test).
Comments
Comment #2
jhodgdonNew branch
Comment #3
jhodgdonComment #4
jhodgdonI've started working on this... Unfortunately, I am finding that either:
(a) There were changes to UI text between Drupal 8.0.x (when we wrote the English original text) and 8.2.x (now)
or
(b) There were some mistakes in the UI text mentioned in the User Guide, which were either introduced by or not caught in the editing phase.
or
(c) Some of both.
In any case... I am going to have to fix some of the User Guide text, so that the tests will:
1. Verify all of the the user interface text that is mentioned in the User Guide.
2. Match what is in the User Guide.
3. Pass for 8.2.x.
Translations... going to be annoying...
Comment #5
jhodgdonRegarding the translations, what I decided to do was to make one central issue for the UI text updates, and add a note for each translation team on another issue we have open, suggesting they look at it, because it's going to touch a lot of files and I don't want to update them in all the languages - it will cause too much disruption for such small changes to each file I think.
So, I created:
#2860295: Some mentioned UI text in User Guide is inconsistent with Drupal
and will add notes to those other issues shortly to notify the translation teams.
Comment #6
eojthebraveDo you think we should try and target 8.3.x instead of 8.2.x given that it's going to be out before to long and we're going to need to make sure the guide is up-to-date for that anyway?
Comment #7
jhodgdonPerhaps. But I am pretty sure that most of the changes that are needed fall into the "sloppiness" category, not the "updated Drupal" category. Also, 8.3.0 is not out yet, so there may still be changes -- hard to aim at a moving target. The changes needed so far (in the Basic Config chapter) are very small, and there are only 6 of these small changes to make.
And I was also hoping to try out the "one motivation for writing these tests is to be able to see what changed when a new version comes out" theory. I.e., make the tests now, and then when 8.3.0 comes out, run the tests and see what failed, thereby knowing which pages in the User Guide need to be updated for 8.3.x.
So... I would prefer to do 8.2.x for now, and make a new issue for 8.3.x updates.
Comment #8
eojthebraveWorks for me. :)
Comment #10
jhodgdonFirst batch of added test lines added! I'll continue on this over the next days... I doubt it will be perfect, but it should be better than having fewer tests.
Comment #20
jhodgdonThere, that's the last batch of tests for the UI text. I can't say that every bit of the UI mentioned in the User Guide is tested, but it's *nearly* all tested. Drupal 8.3.0 is due out on Wednesday, so I plan to update my development site soon to 8.3.0 and run the tests to see what has changed that we need to update the Guide for.
I know of at least one thing that was announced -- new format for the Status Report -- which will not get caught by these tests because it was the organization and look of the report that changed, not the navigation or UI text... meaning we just need to make a new screenshot, but the test won't fail. That kind of change won't ever be caught by the tests... Probably when each new version of Drupal comes out, we should just regenerate all the screenshots and update them in the Git repository, or check to see if they look different, or something. There may be a way to automate the "do they look different" screening from the command line -- I think ImageMagick may have some comparison functions. Anyway... not going to worry about that right now, but I will add a note to the 8.3.x issue about the Status Report.
I'm rambling. Anyway, this issue is FIXED.
Comment #21
eojthebraveAnyway, this issue is AWESOME.
:)
Thanks Jennifer.