Re-thinking admin/help and module handbook synchronization

Some background on this issue is probably in order for the uninitiated.

The idea: Create a set of unified end user documentation for modules

I can't seem to find the original issue(s) where this was talked about, but the jist is that the administer >> help section in Drupal used to be a big mess; most modules just plain had no documentation. The ones that did were all over the map in how they were presented -- some in first person, some in third person, some were several screens long, others were just a couple sentences.

So a bunch of people got together and created a set of guidelines for writing module documentation pages, and wrote a consistent set of documentation for each module that followed a common format of two paragraphs describing the module, followed by a list of links that lead to "tasks" you can do with them. The result is now found in the Drupal modules and features and Contributed modules sections.

The challenge: Synchronizing the handbook pages with module help

The next step was to take all this nice documentation that people had written, and get it into the core and contrib modules' help hooks. To solve this, I originally wrote the docbook2helphook script, which took a docbook export of the main modules book page, ran through and parsed out the page titles, help text and threw it into proper t() calls and all that stuff. After many (many, many, MANY ;)) revisions, this effort eventually culminated in a huge, 133K+ core patch that synchronized the two. And all was good. For awhile. ;)

And now, onto the problems... ;)

There are numerous issues we now face:

1. The docbook exporting functionality was removed from core during the 4.7 development cycle. So there are no longer handy "export" links on any of the pages.
2. The export functionality has now been moved to contrib modules such as Export Docbook. None of these are installed on Drupal.org, so there is no longer a way to create an export of all module handbook pages.
3. The Export Docbook module has system requirements which Drupal.org does not have. This means the current script will not work anymore to generate additional patches to keep the two synched.
4. In various "clean up" tasks that have gone on in 4.7 since the initial patch, a lot of re-wording has happened both at the core module help level and at the handbook level. As a result, the two are now out of sync in certain places.
5. This method of synchronization also creates problems in that we either need to have horrifically complicated markup syntax to delineate links (like >strong class="drupal-admin-help">administer >> help>/strong>), _or_ we have broken links/access denied all over the place on the "you can" links which is pretty sad for "official" documentation. :\
6. The link "for more information, see the XXXX handbook page" at the bottom leads to a page on Drupal.org that says _exactly_ the same thing as they were just reading (with the exception that sometimes more sub-pages are found).
7. Because not everyone has edit permissions to the handbook, doing it this way creates a barrier to people who want to make changes to the text that's currently there. They need to make a documentation issue about it and wait for one of the site admins to get a chance to change it. This is particularly irksome if I'm a contrib module author and I want to update my module's handbook page... I _could_ wait around for some person to change it, or I could just change my hook_help directly.. I know which one people will choose in most cases. :P

Some partial solutions

There are a couple possible partial solutions:

1. Use the Export DXML module instead, and rewrite the docbook2helphook script to parse dxml2helphook. This has a number of advantages... DXML outputs additional semantic information, which means far, far less parsing will probably be required on the part of the script. This also opens up the possibility of other sites importing Drupal handbook pages using the book import module. #5 is still an issue.

2. Do the opposite; update help text _only_ in core/contrib modules and have the handbook just execute the help hooks to display them as a user would normally see (with the exception that it would preg_replace the links with strong tags). While this is pretty logical, it comes with a couple strong disadvantages:

a) It requires people who want to contribute to documentation to submit patches... since a lot of really good writers are not coders, and since there are very few people contributing to documentation, the barrier to participation is something worth considering... coders are often not the best people to write end-user documentation.

b) There are security issues involved; while core modules would have a guarantee to be free of any nastiness, we would definitely not be able to apply this same strategy to contrib module documentation, which might contain who knows what. :P

So. Any ideas? ;)