New issue summary
Over on #2874508: Add tests for URLs in guide pages, I made a script to extract all URLs in all languages' source files, and a test that visits the URLs and checks that they have status 200 (OK).
Many URLs failed the test. Here is the list:
301 for http://advomatic.com/
===> https://www.advomatic.com/
301 for http://creativecommons.org/licenses/by-sa/2.0/
===> https://creativecommons.org/licenses/by-sa/2.0/
301 for http://drupal.org/security/contrib/rss.xml
===> https://drupal.org/security/contrib/rss.xml
301 for http://drupal.org/security/psa/rss.xml
===> https://drupal.org/security/psa/rss.xml
301 for http://drupal.org/security/rss.xml
===> https://drupal.org/security/rss.xml
301 for http://drupal.stackexchange.com/
===> https://drupal.stackexchange.com/
301 for http://git-scm.com
===> https://git-scm.com/
301 for http://pronovix.com
===> https://pronovix.com/
301 for http://pronovix.com/
===> https://pronovix.com/
301 for http://www.drupical.com/
===> https://www.drupical.com/
301 for http://www.goodfreephotos.com
===> https://www.goodfreephotos.com/
302 for https://api.drupal.org/api/drupal/core!LICENSE.txt/8
===> https://api.drupal.org/api/drupal/core!LICENSE.txt/8.2.x
301 for https://assoc.drupal.org
===> https://www.drupal.org/association
301 for https://drupal.org/pronovix
===> https://www.drupal.org/pronovix
301 for https://drupal.org/u/ifrik
===> https://www.drupal.org/u/ifrik
301 for https://drupal.org/u/jhodgdon
===> https://www.drupal.org/u/jhodgdon
301 for https://en.wikipedia.org/wiki/en:
===> https://en.wikipedia.org/wiki/Main_Page
301 for https://en.wikipedia.org/wiki/en:User:Severnjc
===> https://en.wikipedia.org/wiki/User:Severnjc
302 for https://twitter.com/drupalsecurity
===> https://mobile.twitter.com/drupalsecurity
301 for https://www.ars.usda.gov/is/graphics/photos/nov04/K11611-1.htm
===> https://www.ars.usda.gov/oc/images/photos/nov04/k11611-1/
301 for https://www.drupal.org/creating-drupal-test-sites
===> https://www.drupal.org/docs/develop/local-server-setup/how-to-create-a-t...
301 for https://www.drupal.org/cron
===> https://www.drupal.org/docs/7/setting-up-cron/overview
301 for https://www.drupal.org/documentation/backup
===> https://www.drupal.org/docs/7/backing-up-and-migrating-a-site/backing-up...
404 for https://www.drupal.org/documentation/blocks
301 for https://www.drupal.org/documentation/build/distributions
===> https://www.drupal.org/docs/7/distributions
301 for https://www.drupal.org/documentation/clearing-rebuilding-cache
===> https://www.drupal.org/docs/7/administering-drupal-7-site/clearing-or-re...
301 for https://www.drupal.org/documentation/install/create-database
===> https://www.drupal.org/docs/7/install/step-2-create-the-database
301 for https://www.drupal.org/documentation/install/modules-themes/modules-8
===> https://www.drupal.org/docs/8/extending-drupal-8/installing-modules
301 for https://www.drupal.org/documentation/modules/block
===> https://www.drupal.org/docs/8/core/modules/block/overview
404 for https://www.drupal.org/documentation/modules/blocks
301 for https://www.drupal.org/documentation/modules/breakpoint
===> https://www.drupal.org/docs/8/theming-drupal-8/working-with-breakpoints-...
301 for https://www.drupal.org/documentation/modules/filter
===> https://www.drupal.org/docs/8/core/modules/filter/overview
301 for https://www.drupal.org/documentation/modules/image
===> https://www.drupal.org/docs/8/core/modules/image/working-with-images
301 for https://www.drupal.org/documentation/modules/menu
===> https://www.drupal.org/docs/7/working-with-menus/overview
301 for https://www.drupal.org/documentation/modules/node
===> https://www.drupal.org/docs/7/nodes-content-types-and-fields/about-nodes
301 for https://www.drupal.org/documentation/modules/responsive_image
===> https://www.drupal.org/docs/8/mobile-guide/responsive-images-in-drupal-8
301 for https://www.drupal.org/documentation/modules/taxonomy
===> https://www.drupal.org/docs/7/organizing-content-with-taxonomies/organiz...
301 for https://www.drupal.org/documentation/performance
===> https://www.drupal.org/docs/7/managing-site-performance-and-scalability/...
301 for https://www.drupal.org/documentation/structure
===> https://www.drupal.org/docs/7/nodes-content-types-and-fields
301 for https://www.drupal.org/drupalcon
===> https://events.drupal.org
301 for https://www.drupal.org/getting-started/admin/reports
===> https://www.drupal.org/docs/7/monitoring-a-site/reports
301 for https://www.drupal.org/getting-started/before/overview
===> https://www.drupal.org/docs/7/understanding-drupal/overview
301 for https://www.drupal.org/getting-started/install-contrib/themes
===> https://www.drupal.org/docs/7/extending-drupal/installing-themes
301 for https://www.drupal.org/global-training-days
===> https://groups.drupal.org/node/512931
301 for https://www.drupal.org/glossary
===> https://www.drupal.org/docs/7/understanding-drupal/glossary
301 for https://www.drupal.org/node/14307
===> https://www.drupal.org/terms
301 for https://www.drupal.org/node/627158
===> https://www.drupal.org/docs/7/managing-users
301 for https://www.drupal.org/project/modules
===> https://www.drupal.org/project/project_module
301 for https://www.drupal.org/requirements
===> https://www.drupal.org/docs/7/system-requirements/overview
301 for https://www.drupal.org/security-team
===> https://www.drupal.org/drupal-security-team/general-information
301 for https://www.drupal.org/supporters/hosting
===> https://www.drupal.org/association/supporters/hosting
301 for https://www.drupal.org/theme-guide/8
===> https://www.drupal.org/docs/8/theming
301 for https://www.drupal.org/u/diana.lakatos
===> https://www.drupal.org/u/dianalakatos
I've pulled a few out that we shouldn't fix, because they are actual useful redirects vs. pages that have moved permanently:
302 for http://www.drush.org
===> http://www.drush.org/en/master/
302 for https://api.drupal.org
===> https://api.drupal.org/api/drupal
See also comment #2 for a couple of URLs we've figured out what to do with.
Original report about block-concept.txt
Near to the end, at the "Attributions" section there are 2 articles mentioned as original source of this topic.
https://www.drupal.org/documentation/modules/blocks["Working with blocks (content in regions)"]
https://www.drupal.org/documentation/blocks["Blocks"]As the further reading links section is commented out,
//==== Additional resources
then I'd suggest to replace the broken links in the Attributions paragraph with the following two working ones already pointing to the migrated Docs pages:
- Regular block module: https://www.drupal.org/docs/8/core/modules/block
- Custom block module: https://www.drupal.org/docs/8/core/modules/custom_block
Patch attached for convenience.
| Comment | File | Size | Author |
|---|---|---|---|
| #14 | urls.sed_.txt | 5.25 KB | jhodgdon |
| #14 | 2874238-manual-edits.patch | 9.34 KB | jhodgdon |
Comments
Comment #2
jhodgdonThanks for the issue and patch! This is really annoying. The documentation content is being migrated to new URLs, but I think it is ***supposed*** to have redirects. Obviously, this page doesn't.
Anyway... I think this is the page that was migrated:
https://www.drupal.org/docs/8/core/modules/block/overview
titled "Working with blocks". So I think the correct patch would be to replace the old URL with this one for the "Working with..." page. And I don't see anything on the custom block module page that was adapted into the text of this page, so I think we can just cut down to this one link. Thanks!
Also adding the translation needed tag to this issue (our first test of the new proposed process!)
Comment #3
jhodgdonAdding a related issue to create tests to verify all the other outside/documentation URLs in the User Guide.
Comment #4
jhodgdonSee also #2824046-61: Plan for URLs for core modules documentation because these pages should have had redirects. [Even if they do get redirects, we should still update the URLs in the User Guide, I think.]
Comment #5
eojthebraveThis looks like it's actually a typo in the link used in the guide. The page used in the guide is http://drupal.org/documentation/modules/blocks, but the page on d.o. is http://drupal.org/documentation/modules/block (which does not redirect to the appropriate place).
That said, since this is the D8 version of the guide, we should just link directly to the D8 page at https://www.drupal.org/docs/8/core/modules/block/overview like @jhodgdon mentions above.
Comment #6
jhodgdonOver on #2874508: Add tests for URLs in guide pages, I made a script to extract all URLs in all languages' source files, and a test that visits the URLs and checks that they have status 200 (OK).
12 URLs failed the test. So, I think we should repurpose this issue to fix all 13 of them, rather than having 13 separate issues. Adding the list of broken URLs to the issue summary.
I'm also going to add to the test to make sure the final URL was the same as the original, to detect successful redirects... more coming soon...
Comment #7
jhodgdonQuite a few more redirects -- mostly http to https. Adding these to the summary.
Comment #8
jhodgdonI looked in more detail and various of these URLs. I've moved some to the category of "fails but don't fix", and have figured out what we should do about some others. I haven't looked at all of them yet... there are a bunch of drupal.org links that, like the ones in the original issue summary here, we probably need to look at more carefully in context before just accepting the redirects...
Comment #9
baluertlThanks Jennifer for expanding the initial scope of this issue for the greater benefit of others. Also, your automated script makes much easier to have a full overview on the external URLs used in the User Guide, which is great. I just want to reflect on one link related to Hungarian:
https://hu.wikipedia.org/wiki/Tartalomkezel
Although it's listed in the "Links that fail test but we shouldn't fix" section, but I think we definitely needs to improve that. This appears in /source/hu/understanding-drupal.txt (at line 64) and the full URL is written like this:
https://hu.wikipedia.org/wiki/Tartalomkezel%C5%91_rendszerek
We just get used to that this tricky character (ő, Ő) has been always problematic, wherever it appears. :) Now I see that Wikipedia redirects to https://hu.wikipedia.org/wiki/Tartalomkezelő_rendszerek (note the lack of encoding), so do you think shall I change it to this one in our source file?
________
P.s. what a sense of humour... look, even Drupal.org's auto linker logic can't recognise it :)
Comment #10
jhodgdonBetter script (see #2874508-6: Add tests for URLs in guide pages)... pasting output into issue summary.
Comment #11
jhodgdonAh, just saw #9. I just need to modify my URL testing script so it counts % as a valid URL character. Now it doesn't flag the hu.wikipedia page as 404, and a couple of other URLs that apparently had % in them and got cut off by my script. Good catch! I'll commit the change to the script shortly on the other issue. Meanwhile, updating the issue summary here with the corrected list.
Balu: Will it cause you problems if I do a global search/replace, including HU language source, to fix the redirected URLs? I think the HU team is the most active, and I am not worried about any other languages, I think it will be easier to fix them all at the same time in one patch rather than making each language do the fixes separately.
Comment #12
baluertlIf I understand you correctly, feel free to replace the URLs also in HU source files even if they are translated already or not. Or did I missed something in the question?
Comment #13
jhodgdonThat was the question. I will replace URLs in all files then!
Comment #14
jhodgdonWell. Some of those redirects went to drupal 7 pages... and some of the references we used to write pages have gone away..
So, I spent some time looking for better resources, and made a sed script that did better redirects, after first editing manually the block-concept and block-regions files to take out things that were no longer there.
Phew!
Here's the script, plus the patch for the manual edits... I'm going to look over the full results in an hour or two and then run the URL checker again to see if there are more redirects etc. needed. Then hopefully make a commit. But uploading this now so it doesn't get lost...
Balu: If you have a moment, please look over the HU edits in the manual patch -- they may not be quite right as I don't read/write HU but I think they are OK...
Comment #15
jhodgdonOK, the script worked pretty well. I had to do one minor edit, and then when I ran the URL checker script again, I found just a couple of hits that I fixed in another sed script. I'm going to go ahead and commit this, mark it fixed, and send out an alert to the translator list.
Comment #16
jhodgdonComment #18
baluertlAs requested I also checked in source/hu/ some of the first newly replaced URLs from the 2874238-manual-edits.patch and then clicked through all the links source/hu/attributions.txt (this contains the most external links) to make sure whether if they redirects or not. Now I confirm that they look fine from my side too. Thanks for your efforts Jennifer!