Problem/Motivation

This module is built using the YoastSEO.js library. The integration with that library, is stored in a separate GitHub repository due to licensing reasons.

The YoastSEO.js module has gone through a significant overhaul in the past year. Among other things it has adopted React and Web Workers. While doing so it has significantly altered the structure of the library which has made integration difficult.

To make sure this module can use the latest features of the library, the used version must be updated to the new structure. I'd like to do this before a stable 2.0 release since it may be necessary to change data structures in the module to store some of the analysis data (e.g. the library provides both SEO and readability scores but only one of those is stored at the moment).

Proposed resolution

A lot of work has been done already on a separate branch to transform the on-page widget from a jQuery powered widget into one rendered with React.js. The reason this is done is to make it easier to add new features to the widget in the future (such as Social Media sharing previews).

The work will need to be continued and tested so that i can be implemented in a stable 2.0 version of this module. Part of the effort in upgrading the module was making sure that the Drupal UI could be used to translate feedback messages. This is made difficult because YoastSEO uses a different translation library that's incomatible with Drupal's.

Remaining tasks

Previously completed tasks

External library

#2726871: External library included without exception approval from the Licensing Working Group
#2748327: Extended ASCII Characters

Use the entity API to parse rendered entities

Many currently requested features are about multiple fields or unique field combinations. The goal is to move to parsing rendered entities instead of individual fields. This should help us achieve compatibility with things like the Media module using custom tags in the body field or image fields that are outside of the body.

#2587177: No images appear on the page
#2763999: Multiple field support doesn't recognize image fields
#2730333: Content Analysis for Entities, not just nodes
#2642438: Image fields and Yoast SEO
#2573899: Paragraphs module support

Offer better control over which content is scanned

We will move from a default of "Enabled for all content types" to "Enabled for none". Enabling of the Real-time SEO module for entities will be handled on the Real-time SEO configuration page, similarly to how this is also grouped for the metatag module.

#2576633: Offer control over which content types are scanned

Let metatag do what it does best

The metatag module is configured by putting tokens into the metatag fields. A common example is to set the meta description tag to use ‘[node:summary]’ which will use the value of the summary field for a node.

Currently the Real-time SEO module overwrites whatever is configured for the metatag field with its own values. This can lead to unwanted effects. The goal should be to parse whatever the metatag has been configured to by default. The option should exist on the configuration page to "Enable editing of metatag values on the entity edit page" which will provide the module's current behaviour.

#2820851: Support Node Summary
#2862967: Custom fields for meta tags result in error

Issue fork yoast_seo-2879061

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

About issue forks

Comments

Kingdutch created an issue. See original summary.

kingdutch’s picture

kingdutch
he/him
Primary language English
commented
kingdutch’s picture

kingdutch
he/him
Primary language English
commented
Related issues: +#2726871: External library included without exception approval from the Licensing Working Group, +#2748327: Extended ASCII Characters, +#2587177: No images appear on the page, +#2763999: Multiple field support doesn't recognize image fields, +#2730333: Content Analysis for Entities, not just nodes, +#2834751: Support Media CKeditor, +#2642438: Image fields and Yoast SEO, +#2573899: Paragraphs module support, +#2576633: Offer control over which content types are scanned, +#2636500: Should not change the values in my metatag fields unless I want it to, +#2820851: Support Node Summary, +#2862967: Custom fields for meta tags result in error, +#2839553: Add webtest for the configuration page, +#2652428: Provide support for Pathauto, +#2652432: Support directory and language path prefix in url, +#2867785: Translate the Real-Time SEO feedback into a different language
kingdutch’s picture

kingdutch
he/him
Primary language English
commented
Version: 8.x-1.x-dev » 8.x-2.x-dev
kingdutch’s picture

kingdutch
he/him
Primary language English
commented
Related issues: -#2726871: External library included without exception approval from the Licensing Working Group, -#2748327: Extended ASCII Characters, -#2587177: No images appear on the page, -#2763999: Multiple field support doesn't recognize image fields, -#2730333: Content Analysis for Entities, not just nodes, -#2834751: Support Media CKeditor, -#2642438: Image fields and Yoast SEO, -#2576633: Offer control over which content types are scanned, -#2636500: Should not change the values in my metatag fields unless I want it to, -#2820851: Support Node Summary, -#2862967: Custom fields for meta tags result in error, -#2839553: Add webtest for the configuration page, -#2652428: Provide support for Pathauto, -#2652432: Support directory and language path prefix in url, -#2867785: Translate the Real-Time SEO feedback into a different language

Moved issues from "Related" to "Child"

kingdutch’s picture

kingdutch
he/him
Primary language English
commented
Issue tags: +Needs issue summary update

There are currently two issues that are related to tests that I consider blocking for a stable release.

#2941878: Write test for title and description editing
#2941877: Write test for SeoManager::getEnabledBundles

I hope to be able to tag an alpha version soon. Most of the 2.0 goals have been committed to the dev branch already.

jazzper’s picture

jazzper commented

Thanks for all the work on this module!

Any news on a beta version? I'm working on a website with Paragraphs and Taxonomy and would love to use this module on those entities.

kingdutch’s picture

kingdutch
he/him
Primary language English
commented

Hi Jazzper,

Thanks for your interest! :)

Time has been an issue for the most part in the past year. In the past few weeks I've had time to work on some of the outstanding issues. There is currently a feature/react-widget branch which improves the way we use the YoastSEO.js library a lot. It also brings with it the possibility to translate the feedback using Drupal's translation tools.

I'm currently working on moving the data storage from the field itself into a separate entity using an Entity Reference field type. I want to complete this work before releasing a beta/stable version because it will have impact on the way we store data. However, it's a necessary change to make it possible to support multiple keywords which we can't yet do with the current field type.

Unfortunately that doesn't provide an actual ETA. The current alpha release for the 2.0 version should work with Paragraphs and Taxonomies however. I'm also planning on investing some time in ensuring there is an upgrade path between the alpha versions where data changes occur.

The current version for single keywords working with the new React widget could be released as the next alpha version but it needs some performance improvements and I hope that https://www.drupal.org/project/react will release a Drupal 8 version soon that we can depend on to fill the React dependency.

jazzper’s picture

jazzper commented

Great, thanks for the update!

dercheffe’s picture

dercheffe
he/him
Primary language German
Location Sersheim, Germany
commented

Is the Support Media CKeditor issue still critical for a new release, since media is in core?

Any news about a new release of this module?

vrwired’s picture

vrwired commented

I second comment #10 on dismissing/take care of #2834751: Support Media CKeditor and looking to support this module monetarily which I will act upon now in good faith.

kingdutch’s picture

kingdutch
he/him
Primary language English
commented
Issue summary: View changes
Issue tags: -Needs issue summary update

Updating the issue summary to more clearly define what I think is needed for a stable 2.0 release.

#2992481: Update YoastSEO JS to recent version is most crucial as it may require data changes in the module which may be a breaking change.

smustgrave’s picture

smustgrave commented

First thank you for this module!

We have a client that is looking forward to some of the features coming in 2.0 branch, especially the media images being detected.

Question is there a date being planned for the 2.0 release?

kingdutch’s picture

kingdutch
he/him
Primary language English
commented

There is no current date planned. The biggest issue is that the underlying YoastSEO.js library has gone through some major changes, so quite a bit of work is needed to bring those changes back into this module. A lot of work had been done, but that work is now no longer compatible.

The biggest hurdle is the way the YoastSEO.js library handles translation, which is incompatible with how Drupal does this. That's where the majority of the integration work has been spent and will need to be spent again.

That work has been done here: https://github.com/goalgorilla/RTSEO.js/tree/feature/react-presenter

zenimagine’s picture

zenimagine commented

It would be great if this module could preview Google results by integrating the Schema Metatag module.

For example, the price of a product, the stars of the reviews, the duration of a cooking recipe, ...

https://www.drupal.org/project/schema_metatag

norsargsyan made their first commit to this issue’s fork.

davidrobinson_pw’s picture

davidrobinson_pw commented
Issue summary: View changes

Happened to spot a broken link on the summary so just amended that - no protocol was on separate git repo link so it was shown relative to the issues url.

orkutmuratyilmaz’s picture

orkutmuratyilmaz
Primary language Turkish
Location İzmir
commented

We have some good news:

https://github.com/Yoast/wordpress-seo/issues/17899

thejimbirch’s picture

thejimbirch commented
Related issues: +#3474509: Consider working with Real-time SEO for Drupal maintainers

An alpha of the standalone YoastSEO.js has been released. This was a blocker per comment in #14
https://www.npmjs.com/package/yoastseo
https://github.com/Yoast/wordpress-seo/issues/17899

As this module has been selected to be in Drupal CMS, and will be featured prominently in the Advanced SEO recipe, I would love it if we could move this project forward. I know the delays with the changes in the underlying library have been frustrating and may have caused maintainer burnout.

What would be required to help move development of this module, and an update release forward?

A similar project has also been created on Drupal.org, yoast_analysis
https://www.drupal.org/project/yoast_analysis

I created an issue there to see if we could work together to create a unified solution.
#3474509: Consider working with Real-time SEO for Drupal maintainers

In addition to the 3 remaining roadmap issues, there are a dozen issues in RTBC, and another 10 that are in Needs Review on the 8.x branch at the time of this comment.

Thank you for your consideration and contribution!

kingdutch’s picture

kingdutch
he/him
Primary language English
commented

Hey everyone!

I'm excited that a release for the Yoast SEO library has been created, which unblocks some of the work.

2.x made some important changes vs 1.x. Most notably it provides better analysis support in how entities are rendered rather than looking at individual fields itself. It also better manages how the Real-Time SEO module attaches fields to entities. I initially wanted to use that momentum to make some more changes.

Most notably I wanted to change how the analysis results were stored to enable a few things:

1. Store both the SEO score and the reading ease score; currently only the SEO score is stored (the library has since also added an "inclusivity score"
2. Adjust storage and the widget in a way that allows multiple keywords to be analysed
3. Store these results in a way that allows creating overviews to enable insights into how pages rank (e.g. comparing pages for the same (related) keywords).
4. Decouple analysis from the form itself so that in the future it could also be used for on-page analysis.

Secondarily I want to ensure the webworker is used by the widget to fix issues like #3008802: The page freezes while the Real-Time SEO module runs script without adding a button the user needs to press manually.

Finally this module currently allows editing the title and description, but realistically a better UX can be provided by having a site builder provide a dedicated field for this and only rendering the result from the metatag module (that also ensures more complex logic can be included).

The biggest reason not to tag a stable 2.0 would be that these changes might cause changes to the code or storage which can break other modules if they rely on a specific structure. From a data perspective these changes can usually be made while preserving the data itself so that for actual site users there's be an upgrade path within the 2.x-* range of versions.

I think the next steps now that this module will make it into Drupal CMS would be to create a clear plan of which of these features would be needed for 2.0 and which should be pushed onwards to a 3.x version.

One of the things I hope to do in the coming week is to move development of the module from Drupal's GitLab to https://github.com/Kingdutch/drupal-real-time-seo. This is because the module requires both the Drupal code of the module itself as well as a custom JavaScript library which contains the widget: https://github.com/Kingdutch/RTSEO.js This allows both the module and library to be developed and versioned in a single repository (with the same version) and allows the Behat tests to run on GitHub (I've not been able to get those to work in Gitlab). The module code can be subtree split back to GitLab so that MRs here are still possible. The library code was previously split out in #2726871: External library included without exception approval from the Licensing Working Group. That should massively ease development.

I'll also be at DrupalCon Barcelona where I hope to have time to do some of the above things :)

thejimbirch’s picture

thejimbirch commented

Thanks for the update @kingdutch!

anybody’s picture

anybody
Primary language German
Location Porta Westfalica
commented
Issue summary: View changes

Please also mind #3475510: Make 8.x-2.x the default branch in GitLab on this road :)

volodymyr.umanets’s picture

volodymyr.umanets commented

Hi @kingdutch,
I can confirm that web worker prevents the page freezes.

Nevertheless, I did some research, and found the reason of those freezes, it is related mainly to findKeywordInFirstParagraph and KeywordDensityAssessment stuff.

In getSentences.js there is a code:

exports.default = function (text) {
	text = (0, _unifyWhitespace.unifyNonBreakingSpace)(text);
	let blocks = (0, _html.getBlocks)(text);

	// Split each block on newlines.
	blocks = (0, _lodashEs.flatMap)(blocks, function (block) {
		return block.split(newLineRegex);
	});

It takes sometime for it to execute on long strings.
I have no freezes even with 600kb body text, but have ones if set in SEO widget to use the default display mode, which also parses header and footer because of Layout Builder.

So if there is any base64 image it will freeze in order to do some operations by researcher. In my case it was 78kb long base64 string, making the researcher stuck.
Replacing it with svg ~10kb resolves the issue, but still a small delay is present.
So, without web worker the only solution to fix it without having preview button - use pre-defined display mode for SEO.

kingdutch’s picture

kingdutch
he/him
Primary language English
commented
Status: Active » Fixed

2.0 has been tagged today. Since I was unable to make the improvements that I wanted, I've merged the improvements contributed by the community, so that the module at least keeps moving forward.

Closing this issue and then we can see how we can bring in more improvements into future updates and realise some of the original plans through smaller steps :)

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.