It seems like the schema and DB index don't square up here, which AFAICT both prevents messages being delivered and prevents the DB update running (if there are existing entries).

#3031714: Database row deletions are inefficient added a unique index to mid but did not have a corresponding update to message_digest_schema() to make that column a serial.

In Drupal\message_digest\Plugin\Notifier\DigestBase::deliver() we trigger a DB insert without setting mid, which is fine if mid is a serial and unique, OR if mid is not a serial and can store duplicates.

AFAICT, this won't work. If the schema update is applied (which can only happen if there are no existing message_digest entries in the DB anyway?), then future digest messages will not be inserted to that table.

Is this something which only works on MySQL perhaps? It still seems like it would not behave correctly.

When there's no pre-existing data in message_digest

Running message deliveries throws a lot of integrity constriant violation exceptions:

[error] Drupal\Core\Database\IntegrityConstraintViolationException: SQLSTATE[23505]: Unique violation: 7 ERROR: duplicate key value violates unique constraint "message_digest__mid__key"
DETAIL: Key (mid)=(17) already exists.: INSERT INTO message_digest (receiver, entity_type, entity_id, notifier, timestamp, mid) VALUES (:db_insert_placeholder_0, :db_insert_placeholder_1, :db_insert_placeholder_2, :db_insert_placeholder_3, :db_insert_placeholder_4, :db_insert_placeholder_5); Array()

When there's already data in message_digest

DB is Postgres, and we have existing entries in the message_digest table. The column mid is not unique; far from it:

nzmathsd8=# select count(*), mid from message_digest group by mid having count(*) > 1;
1924	1
1926	2
1926	3
1926	4

I don't think we can rely on {message_digest}.mid being unique based on the schema, but 022e5b2 merged in #3031714: Database row deletions are inefficient treats it as such.

vagrant@example:/vagrant$ drush updatedb
 -------- ------------ --------------- ---------------------------------------- 
  Module   Update ID    Type            Description                             
 -------- ------------ --------------- ---------------------------------------- 
  messag   8102         hook_update_n   Add unique index to mid field.          
  e_dige                                                                        
  st                                                                            
 -------- ------------ --------------- ---------------------------------------- 

 Do you wish to run the specified pending updates? (yes/no) [yes]:
 > yes

>  [notice] Update started: message_digest_update_8102
>  [error]  SQLSTATE[23505]: Unique violation: 7 ERROR:  could not create unique index "message_digest__mid__key" 
> DETAIL:  Key (mid)=(1) is duplicated.: ALTER TABLE {message_digest} ADD CONSTRAINT message_digest__mid__key UNIQUE (mid); Array
> (
> )
>  
>  [error]  Update failed: message_digest_update_8102 
 [error]  Update aborted by: message_digest_update_8102 
 [error]  Finished performing updates. 

Comments

xurizaemon created an issue. See original summary.

xurizaemon’s picture

Issue summary: View changes
xurizaemon’s picture

Issue summary: View changes
xurizaemon’s picture

Issue summary: View changes
xurizaemon’s picture

Issue summary: View changes

(sorry for all the edits, I should use preview)

xurizaemon’s picture

Status: Active » Needs review
StatusFileSize
new1.49 KB

Here's my proposed solution, which removes the unique index (I don't think that was what we wanted) and replaces it with a regular index.

It seems like a single message could be delivered to multiple people, which would have multiple id and share a single mid?

This should support that use case. Would appreciate feedback from @jhedstrom & @kevin.dutra!

Status: Needs review » Needs work

The last submitted patch, 6: message_digest-3060135-6-index_not_key.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

xurizaemon’s picture

Status: Needs work » Needs review
StatusFileSize
new1.54 KB

Status: Needs review » Needs work

The last submitted patch, 8: message_digest-3060135-8-index_not_key.patch, failed testing. View results

alorenc’s picture

Hi Chris,

I agree with you about the index, it should not be unique as one message can have many subscribers.
It looks like the unit fails because the 4 param expects to be the table specification, not the whole plugin structure.
I suggest something like that:

$spec = message_digest_schema()['message_digest'];

$schema = \Drupal::database()->schema();
$schema->dropUniqueKey('message_digest', 'mid');
$schema->addIndex('message_digest', 'mid', ['mid'], $spec);

echo15’s picture

StatusFileSize
new967 bytes

Hello there.
I've made small changes to @xurizaemon patch from comment #8 accordint to @alorenc suggestion.

echo15’s picture

StatusFileSize
new1.03 KB

Sorry. The wrong patch was attached to comment 11.

echo15’s picture

Status: Needs work » Needs review

Please review patch in #12 comment

zviryatko’s picture

StatusFileSize
new1.21 KB

For users who already did update and 8102 failed these patches will not work, 'cause 8102 will block all further updates, but for all others who didn't have data by some reason and update applied successfully it needs to drop the unique key, so better to merge #8 and #12 (i.e. commenting out 8102 and drop index with the fix mentioned in #10).

Status: Needs review » Needs work

The last submitted patch, 14: message_digest-3060135-14.patch, failed testing. View results

echo15’s picture

@zviryatko I have the latest version of the module.
So 8002 is already in source code.
#12 patch if for users that have the latest version of module.

BTW patch 14 failed.

echo15’s picture

Status: Needs work » Needs review

Please review patch in #12 comment

anthonyf’s picture

StatusFileSize
new680 bytes

The latest patches no longer apply vs. 8.x-1.x. I rerolled it to include only the new update hook.

anthonyf’s picture

StatusFileSize
new1020 bytes

My previous patch was missing the schema change to make mid index non-unique. This one adds that change back in.

  • jhedstrom committed a6922c0 on 8.x-1.x authored by anthonyf
    Issue #3060135 by xurizaemon, echo15, anthonyf, zviryatko:...
jhedstrom’s picture

Status: Needs review » Fixed

Sorry for the long delay. This fix makes sense, as I don't think mid was ever intended to be unique (it could be but would depend on use cases/senders, etc.)

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.