Problem/Motivation

If there is a paragraph where there is no field data (perhaps related to https://www.drupal.org/project/paragraphs/issues/2741937), the total count does not match the actual count since the count query is using a different mechanism.

Proposed resolution

Count in the same way as the fetchall.

Patch to come shortly.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

scott_euser created an issue. See original summary.

scott_euser’s picture

Assigned: scott_euser » Unassigned
Status: Active » Needs review
FileSize
497 bytes

Given that the loading of all items to be updated is using a select query:

$query = $database->select('paragraphs_item_field_data', 'p');
$query->fields('p', ['id', 'parent_id', 'parent_type', 'parent_field_name']);
$query->range($sandbox['current'], $sandbox['current'] + $paragraphs_per_batch);
$result = $query->execute();

updated count to match:

$paragraphs = $database->select('paragraphs_item_field_data', 'p')
  ->countQuery()
  ->execute();
$sandbox['total'] = $paragraphs;
jasonlttl’s picture

Note, even with the patch I'm still seeing an infinite loop in cases where there are zero paragraph content entities.

JimCraner’s picture

Confirmed, we have zero paragraphs and the updatedb command hangs in a Divide by 0 loop on the _8018 update:

[notice] Update started: paragraphs_update_8018
[warning] Division by zero paragraphs.install:349
[notice] 0 Paragraphs processed.

bkosborne’s picture

Priority: Normal » Major

This seems pretty major to me. I wouldn't say critical, because if you don't have any paragraph entities, then it's clear you're not using the module.

David Radcliffe’s picture

I had the same problem, but since I had no paragraph entities, I was able to resolve the problem by uninstalling and reinstalling the module.

alberto56’s picture

Status: Needs review » Needs work

With the patch, I no longer get the division by zero, but I still get the infinite loop:

Performing paragraphs_update_8018                                    [ok]
Performing paragraphs_update_8018                                    [ok]
Performing paragraphs_update_8018                                    [ok]
Performing paragraphs_update_8018                                    [ok]
Performing paragraphs_update_8018                                    [ok]
Performing paragraphs_update_8018                                    [ok]
Performing paragraphs_update_8018                                    [ok]
Performing paragraphs_update_8018                                    [ok]
Performing paragraphs_update_8018                                    [ok]
Performing paragraphs_update_8018                                    [ok]
Performing paragraphs_update_8018                                    [ok]
Performing paragraphs_update_8018                                    [ok]
Performing paragraphs_update_8018                                    [ok]
Performing paragraphs_update_8018                                    [ok]
Performing paragraphs_update_8018                                    [ok]
Performing paragraphs_update_8018                                    [ok]
Performing paragraphs_update_8018                                    [ok]
Performing paragraphs_update_8018                                    [ok]
Performing paragraphs_update_8018                                    [ok]
Performing paragraphs_update_8018                                    [ok]
if you don't have any paragraph entities, then it's clear you're not using the module.

In my continuous integration server, I have this module enabled with an empty database using a previous schema, and my C.I. server attempts to run the updates, to make sure no error occurs. So in my case, I do want to have the module enabled for the CI server even though there's no content.

websiteworkspace’s picture

I've just experienced the same error when updating the paragraphs module just now:

1 - the first update attempt (using composer), went into an infinite loop that issued the following output endlessly before being canceled.


> [warning] Division by zero paragraphs.install:349
>  [notice] 0 Paragraphs processed.

2 - After aborting with Ctrl-C, I run composer, and then drush update again and the following output was issued:



 ---------------------- ----------- --------------- ---------------------------
  Module                 Update ID   Type            Description
 ---------------------- ----------- --------------- ---------------------------
  paragraphs             8018        hook_update_n   Make the parent fields
                                                     revisionable.
  paragraphs             8019        hook_update_n   Update the field storage
                                                     definitions of the parent
                                                     fields.
 ---------------------- ----------- --------------- ---------------------------


 // Do you wish to run the specified pending updates?: yes.

>  [notice] Update started: paragraphs_update_8018
>  [error]  Cannot add field paragraphs_item_revision_field_data.parent_id: field already exists.
>  [error]  Update failed: paragraphs_update_8018
 [error]  Update aborted by: paragraphs_update_8018
 [error]  Finished performing updates.


scott_euser’s picture

So two remaining issues to add to the patch:

  1. If total is zero, rest of update continues to be attempted but shouldn't.
  2. If already stuck in endless loop, reattempt fails because addField() is called without a preceding ! hasField() check

Will update the patch as soon as I am behind a computer.

scott_euser’s picture

Updated patch now to cover cases where no Paragraphs exist and where user has run updb, got caught in endless loop, arrived here, and applied patch (ie, allowing them to re-run without fatal error).

corE’s picture

3-20-19 [not sure which section to post this in],

I have been getting a failed paragraphs update from 1.6 to 1.7 and my log has multiple entries of "Warning: Division by zero in paragraphs_update_8018()":

The following updates returned messages:
paragraphs module
Update #8018

Failed: Drupal\Core\Database\SchemaObjectExistsException: Cannot add field paragraphs_item_revision_field_data.parent_id: field already exists. in Drupal\Core\Database\Driver\mysql\Schema->addField() (line 414 of ...core/lib/Drupal/Core/Database/Driver/mysql/Schema.php).

The database update says I may need to "update your database manually". I manually rolled it back to 1.6 - Still showing errors on status page though.

scott_euser’s picture

Applying the patch in https://www.drupal.org/project/paragraphs/issues/3041868#comment-13032310 should solve that for you @corE

blueblot’s picture

same problem as in #8
tried the latest patch but no luck.

priyanka1989’s picture

Same problem as #11

paragraphs module

Update #8018

Failed: Drupal\Core\Database\SchemaObjectExistsException: Cannot add field paragraphs_item_revision_field_data.parent_id: field already exists. in Drupal\Core\Database\Driver\mysql\Schema->addField() (line 414 of /Users/DevAdmin/Sites/devdesktop/cummins-test/docroot/core/lib/Drupal/Core/Database/Driver/mysql/Schema.php)

cedrichillion’s picture

Same issue and still not being able to update after applying https://www.drupal.org/project/paragraphs/issues/3041868#comment-13032310

Subsequently, trying to run drush entity-updates gives me this error message:

>  [error]  Drupal\Core\Entity\EntityStorageException: Exception thrown while performing a schema update. SQLSTATE[42S22]: Column not found: 1054 Unknown column 'parent_id' in 'where clause': SELECT 1 AS expression
> FROM
> {paragraphs_item_field_data} t
> WHERE parent_id IS NOT NULL
> LIMIT 1 OFFSET 0; Array
> (
> )
>  in Drupal\Core\Entity\Sql\SqlContentEntityStorage->wrapSchemaException() (line 1491 of /home/vagrant/code/drupal/sites/xxxx/web/core/lib/Drupal/Core/Entity/Sql/SqlContentEntityStorage.php).
alesr’s picture

You get into an endless loop because of this line
$sandbox['#finished'] = ($sandbox['current'] / $sandbox['total']);
If $sandbox['total'] is 0, it will never finish.

So probably you don't have any paragraphs in your database if you get this error.
Patch #10 didn't work for me either, so here is patch #16.

bander2’s picture

Status: Needs review » Reviewed & tested by the community

16 fixes it for me.

scott_euser’s picture

Status: Reviewed & tested by the community » Needs work

Fixes to other issues we're removed in #16. Please build from existing patches and provide interdiffs.

alesr’s picture

Status: Needs work » Needs review
ahebrank’s picture

I'll put in a vote for the fix in #16 addressing the primary cause of this problem and as such should be hotfixed. If deploying between staged environments, that check should be sufficient.

I *think* the rest of #10 and possibly stuff like #15 are secondary effects from 8018 failing the first time, which may be worked around by restoring the db from a backup (or live/test environment), applying #16, and updating again.

#19 is probably OK, but the fact that there are two levels of bugs at work makes this tougher to review.

Jon Pollard’s picture

For me, patch #19 didn't work - left my site dead. Patch #16 however has done the trick. The site has no paragraph content but does have the module installed.

azinck’s picture

#10 has the right idea of needing to fix the query that comes up with the total count because it's possible to have a mis-match between the number of paragraph entities and the number of records in the paragraphs_item_field_data table, but the count query provided in #10 is still wrong (caused only one batch to run, rather than updating all paragraph records).

I've fixed that here, and have incorporated the other fixes from #19 since they look right to me.

jurgenhaas’s picture

Status: Needs review » Reviewed & tested by the community

Patch from #23 applies nicely and works as expected.

scott_euser’s picture

Thanks for updating it!

I *think* the rest of #10 and possibly stuff like #15 are secondary effects from 8018

Well part of 10 handles paragraphs not cleaned up which result in a count mismatch as per the original issue summary. The rest are cleanup on failed updates yes.

hunterbuchanan’s picture

Another vote of confidence for #23 patch, seems to work great. Thank you all!

alasserr’s picture

Hi,
I've encounterd this problem as I had no paragraphs.

I just replaced the line 349 of paragraphs.install with this one:

$sandbox['#finished'] = ($sandbox['total'] == 0) ?? ($sandbox['current'] / $sandbox['total']);

See, if no paragraphs are to process ($sandbox['total']) then no process is run.

Have a good day

Adrien

ro-no-lo’s picture

Btw. if someone, like me, was stuck with a situation that update 8019 is the latest according to the keyvalue schema entry for paragraphs. The situation for me was, that the revision table still had no entries in parent_id, parent_type and praent_field_name, which in turn resulted in problems down the road where paragraph executes ->getParentType() (or something like that).

Anyway, I looked at the .install updates and wrote this sql, which copies the fields from one table to the revision table, which shall be done in 8018.

UPDATE paragraphs_item_revision_field_data pi inner join (
  SELECT p.id, p.parent_id, p.parent_type, p.parent_field_name 
  FROM paragraphs_item_field_data p
) as src ON pi.id = src.id
SET 
  pi.parent_id = src.parent_id,
  pi.parent_type = src.parent_type,
  pi.parent_field_name = src.parent_field_name
  ;

You just have to execute this once. If you wanna ultrashure, you may add IS NULL checks for the fields. Anyway the update 8019 should then be executed without problems. Therefore you have to set the 8018 schema value in the keyvalue store for paragraphs to make it work for the update.php call.

blueblot’s picture

#23 patch works fine. Thx!

ccshannon’s picture

I actually thought I had created some Paragraphs (I have several Paragraph bundles/types I am testing with), but I got the infinite loop problem, but I guess I hadn't created any, or had deleted the ones I had created for testing. Patch #23 fixed the issue and I got this in the output from drush:

[notice] Update started: paragraphs_update_8018
 [notice] No Paragraphs to be processed.
 [ok] Update completed: paragraphs_update_8018

So, just to note, again, that it is not unusual to be developing a project that has Paragraphs configured but no actual Paragraphs in the DB.

Great that people jumped on this and released patches, so thanks to @azinck.

TomSaw’s picture

#23 worked. Dankeschön!

fonant’s picture

Patch #23 worked for me. Thanks!

Florent Jousseaume’s picture

Hello,

The #23 works fine for me.

Thank you !

  • miro_dietiker committed deeebd5 on 8.x-1.x authored by azinck
    Issue #3041868 by scott_euser, alesr, azinck, blueblot, priyanka1989,...
miro_dietiker’s picture

Status: Reviewed & tested by the community » Fixed

Thank you all for the quick improvements proposals and reviews.

Committed #23, will create a new release shortly.

astringer’s picture

I was not able to updb, I also had 3 entity/field definition mismatches.

I patched with #23.

It allowed me to drush updb, but I am still getting mismatched entity/field definitions for one of the three fields -- Parent ID field.

I ran drush entup and got the following error:

Drupal\Core\Entity\EntityStorageException: Exception thrown while performing a schema update. SQLSTATE[42S22]: Column not found: 1054 Unknown column 'parent_id' in 'where clause': SELECT 1 AS expression                                           
[error]
FROM
{paragraphs_item_field_data} t
WHERE parent_id IS NOT NULL
LIMIT 1 OFFSET 0; Array
(
)
 in Drupal\Core\Entity\Sql\SqlContentEntityStorage->wrapSchemaException() (line 1491 of /app/web/core/lib/Drupal/Core/Entity/Sql/SqlContentEntityStorage.php).
Failed: Drupal\Core\Entity\EntityStorageException: !message in Drupal\Core\Entity\Sql\SqlContentEntityStorage->wrapSchemaException() (line 1491 of /app/web/core/lib/Drupal/Core/Entity/Sql/SqlContentEntityStorage.php).  

I just took over managing this site, and it appears there are no paragraphs in the database, but I don't know this module very well, and would just like it to update — so I can figure out why there is a paragraph type set up but apparently no paragraphs....later.

Anyone know how to cleanly and easily clear this?

David Radcliffe’s picture

If there is no paragraph content then the quickest solution is to uninstall and reinstall Paragraphs.

ccshannon’s picture

Thanks, @David Radcliffe. I have several Paragraph Types custom configured (though, yes, they are exported to .yml), and was worried that might be an issue, so I admit, I didn't try.

Farnoosh’s picture

Thank you #37. On Core 8.6.13 and lightning 3.207 with no paragraph content, reinstalling Paragraphs module fixed the issue.

azovsky’s picture

The last dev version (or #23 patch) of the module not working, unfortunately. Still an endless loop on `Performing paragraphs_update_8018`.

Berdir’s picture

Are you sure it's endless? How much data do you have? it can take a while if you have a lot of paragraphs.

azovsky’s picture

Actually, I'm not sure... I have a lot of content! E.g. the paragraphs_item_field_data table has ~ 32K rows. In this case how many "updates" can I see? 32000/50=640 ? But I see ~ 2K "Performing paragraphs_update_8018 [ok]" lines in my CLI in ~ 30 min.! I don't think is it okay.

Berdir’s picture

No, that doesn't seem OK, but you will need to debug it somehow to figure out why it doesn't finish, there might be something off with your data, some kind of inconsistency. I'd suggest you open a new issue and try to provide more details.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.