D7 to D8: FieldException: Attempt to create a configurable field of non-configurable field storage ldap_user_puid_sid.

This is likely a duplicate of #2837074: ldap_user: Support UI installation for non-English languages. Are you really on alpha3?! beta4 would need the patch but it's in dev.

Hi

If dropping tables is an option for you I'd recommend dropping user__ldap_user_prov_entries and removing the ldap_* columns in users_field_data. Cache clearing and calling the code from ldap_user_preinstall() should fix your issue.

Out of curiosity, did you perform a regular migration with Drupal Migrate and that created those fields for you? If that's the case, could you post a DESCRIBE of those tables in your database? I'd like to compare the structure to a working 7.x site to see if I there is something trivial to do about this and avoid that.

I'm experiencing the same problem, with the same error message. I'm running Drupal 8.6.1 and I used drush to migrate from Drupal 7 into Drupal 8. I've dropped the table and columns mentioned in #8 multiple times, both with and without the patch added, and the error remains.

Here is the DESCRIBE information for both tables before running the ldap_user module install and after the install (with columns intact).

Let me know if you'd like additional information.

Before enabling ldap_user

mysql> describe users_field_data;
+--------------------------+------------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+--------------------------+------------------+------+-----+---------+-------+
| uid | int(10) unsigned | NO | PRI | NULL | |
| langcode | varchar(12) | NO | PRI | NULL | |
| preferred_langcode | varchar(12) | YES | | NULL | |
| preferred_admin_langcode | varchar(12) | YES | | NULL | |
| name | varchar(60) | NO | MUL | NULL | |
| pass | varchar(255) | YES | | NULL | |
| mail | varchar(254) | YES | MUL | NULL | |
| timezone | varchar(32) | YES | | NULL | |
| status | tinyint(4) | YES | | NULL | |
| created | int(11) | NO | MUL | NULL | |
| changed | int(11) | YES | | NULL | |
| access | int(11) | NO | MUL | NULL | |
| login | int(11) | YES | | NULL | |
| init | varchar(254) | YES | | NULL | |
| default_langcode | tinyint(4) | NO | | NULL | |
| rh_action | varchar(255) | YES | | NULL | |
| rh_redirect | varchar(255) | YES | | NULL | |
| rh_redirect_response | int(11) | YES | | NULL | |
+--------------------------+------------------+------+-----+---------+-------+
18 rows in set (0.00 sec)

mysql> describe user__ldap_user_prov_entries;
+------------------------------+------------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+------------------------------+------------------+------+-----+---------+-------+
| bundle | varchar(128) | NO | MUL | | |
| deleted | tinyint(4) | NO | PRI | 0 | |
| entity_id | int(10) unsigned | NO | PRI | NULL | |
| revision_id | int(10) unsigned | NO | MUL | NULL | |
| langcode | varchar(32) | NO | PRI | | |
| delta | int(10) unsigned | NO | PRI | NULL | |
| ldap_user_prov_entries_value | varchar(255) | NO | | NULL | |
+------------------------------+------------------+------+-----+---------+-------+
7 rows in set (0.04 sec)

After enabling ldap_user

mysql> describe users_field_data;
+--------------------------+------------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+--------------------------+------------------+------+-----+---------+-------+
| uid | int(10) unsigned | NO | PRI | NULL | |
| langcode | varchar(12) | NO | PRI | NULL | |
| preferred_langcode | varchar(12) | YES | | NULL | |
| preferred_admin_langcode | varchar(12) | YES | | NULL | |
| name | varchar(60) | NO | MUL | NULL | |
| pass | varchar(255) | YES | | NULL | |
| mail | varchar(254) | YES | MUL | NULL | |
| timezone | varchar(32) | YES | | NULL | |
| status | tinyint(4) | YES | | NULL | |
| created | int(11) | NO | MUL | NULL | |
| changed | int(11) | YES | | NULL | |
| access | int(11) | NO | MUL | NULL | |
| login | int(11) | YES | | NULL | |
| init | varchar(254) | YES | | NULL | |
| default_langcode | tinyint(4) | NO | | NULL | |
| rh_action | varchar(255) | YES | | NULL | |
| rh_redirect | varchar(255) | YES | | NULL | |
| rh_redirect_response | int(11) | YES | | NULL | |
| ldap_user_puid_sid | varchar(255) | YES | | NULL | |
| ldap_user_puid | varchar(255) | YES | | NULL | |
| ldap_user_puid_property | varchar(255) | YES | | NULL | |
| ldap_user_current_dn | varchar(255) | YES | | NULL | |
| ldap_user_last_checked | int(11) | YES | | NULL | |
| ldap_user_ldap_exclude | tinyint(4) | YES | | NULL | |
+--------------------------+------------------+------+-----+---------+-------+
24 rows in set (0.01 sec)

mysql> describe user__ldap_user_prov_entries;
+------------------------------+------------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+------------------------------+------------------+------+-----+---------+-------+
| bundle | varchar(128) | NO | MUL | | |
| deleted | tinyint(4) | NO | PRI | 0 | |
| entity_id | int(10) unsigned | NO | PRI | NULL | |
| revision_id | int(10) unsigned | NO | MUL | NULL | |
| langcode | varchar(32) | NO | PRI | | |
| delta | int(10) unsigned | NO | PRI | NULL | |
| ldap_user_prov_entries_value | varchar(255) | NO | | NULL | |
+------------------------------+------------------+------+-----+---------+-------+
7 rows in set (0.01 sec)

mysql>

Awesome, thanks.

So I think I understand this issue a bit better now and supposedly the database is fine but Drupal is apparently unhappy about where this field is registered. My assumption is that the fields are now FieldStorage instead of BaseDefinitions.

I'd like to ask for two more things to get a better of idea of the problem:

The output of this:
drush sqlq "SELECT value FROM key_value WHERE name like '%ldap_user_puid_sid'" | php -R 'print_r(unserialize($argn));

And could you please also add the yaml files in the migration pertaining to ldap_user_puid_sid?

It'd certainly be possible to write a script which cleans the relevant parts from the database and solves it that way but ideally we'd keep the information to allow for migrations with intact mappings since not all sites are set up to match Drupal users with LDAP accounts opportunistically. I think the data within those fields is still close enough that data from 7 should work in 8 but I haven't actually tried a migration.

However, I've not found an easy path to solve this problem. It seems to me that we'd need to something like what SqlContentEntityStorageSchemaConverter() is doing where we copy the data to a temporary field and then set it back when the new definition is installed.

If've attached an experimental patch based on the assumptions above and I'd be interested to hear if that helps and whether it actually deletes the data. (Definitely backups beforehand.) You might need to put a "return;" Just after the start of ldap_user_entity_base_field_info() if your site is already in an inconsistent state until that hook has run.

Grahl, I've followed the steps you listed above. I migrated the site via drush, and I didn't find any yaml files within my site from the migration containing a reference to ldap_user_puid_sid.

Below is the code from the SELECT statement from before I tried to enable the ldap_user module.

drush sqlq "SELECT value FROM key_value WHERE name like '%ldap_user_puid_sid'" | php -R 'print_r(unserialize($argn));'
Array
(
    [user__ldap_user_puid_sid] => Array
        (
            [description] => Data storage for user field ldap_user_puid_sid.
            [fields] => Array
                (
                    [bundle] => Array
                        (
                            [type] => varchar_ascii
                            [length] => 128
                            [not null] => 1
                            [default] => 
                            [description] => The field instance bundle to which this row belongs, used when deleting a field instance
                        )

                    [deleted] => Array
                        (
                            [type] => int
                            [size] => tiny
                            [not null] => 1
                            [default] => 0
                            [description] => A boolean indicating whether this data item has been deleted
                        )

                    [entity_id] => Array
                        (
                            [type] => int
                            [unsigned] => 1
                            [not null] => 1
                            [description] => The entity id this data is attached to
                        )

                    [revision_id] => Array
                        (
                            [type] => int
                            [unsigned] => 1
                            [not null] => 1
                            [description] => The entity revision id this data is attached to, which for an unversioned entity type is the same as the entity id
                        )

                    [langcode] => Array
                        (
                            [type] => varchar_ascii
                            [length] => 32
                            [not null] => 1
                            [default] => 
                            [description] => The language code for this data item.
                        )

                    [delta] => Array
                        (
                            [type] => int
                            [unsigned] => 1
                            [not null] => 1
                            [description] => The sequence number for this data item, used for multi-value fields
                        )

                    [ldap_user_puid_sid_value] => Array
                        (
                            [type] => varchar
                            [length] => 255
                            [binary] => 
                            [not null] => 1
                        )

                )

            [primary key] => Array
                (
                    [0] => entity_id
                    [1] => deleted
                    [2] => delta
                    [3] => langcode
                )

            [indexes] => Array
                (
                    [bundle] => Array
                        (
                            [0] => bundle
                        )

                    [revision_id] => Array
                        (
                            [0] => revision_id
                        )

                )

        )

)

After I applied the patch from #12, I still got the same error:

The website encountered an unexpected error. Please try again later.Drupal\Core\Field\FieldException: Attempt to create a configurable field of non-configurable field storage ldap_user_puid_sid. in Drupal\field\Entity\FieldConfig->getFieldStorageDefinition() (line 318 of core/modules/field/src/Entity/FieldConfig.php).

I ran the SELECT statement again, and got this result:

drush sqlq "SELECT value FROM key_value WHERE name like '%ldap_user_puid_sid'" | php -R 'print_r(unserialize($argn));'
Array
(
    [users_field_data] => Array
        (
            [fields] => Array
                (
                    [ldap_user_puid_sid] => Array
                        (
                            [type] => varchar
                            [length] => 255
                            [binary] => 
                            [not null] => 
                        )

                )

        )

)

@cajewell: Your feedback is a bit unclear on when you installed the patch. If you installed it before you enabled the module: The proposed patch is only an update hook for now and will not run in installation (we can add it there if it works). You can force it to run by adding ldap_user_update_8308(); to the install hook or running it directly with drush. If you enabled the module and then patched the update hook should be recognized correctly and run on updb but if your site is now in an unusable state you should be able to get it to that point by adding a return as indicated above (or removing ldap_user_entity_base_field_info() outright).

That the output is different and several fields are missing without running the update hook is..interesting. Not sure how that could be the case.

P.S.: If you use the <code> tag to report your output it is much easier to read. You can retroactively edit your comments to apply that formatting.

@grahl, I installed the patch before trying to enable the ldap_user module. Once I enable the ldap_user module, the site is unusable and I can't get the module to run the patch via the ui or drush. Drush returns this error:

Drupal\Core\Field\FieldException: Attempt to create a configurable   [error]
field of non-configurable field storage ldap_user_puid_sid. in
Drupal\field\Entity\FieldConfig->getFieldStorageDefinition() (line
318 of
/home/caj/websites/www7/htdocs/core/modules/field/src/Entity/FieldConfig.php)
snip

The rest of the error is quite long, but I can share it with you if you'd like. I didn't try the other options you listed above, I'm not sure how to implement them. I could test them for you if you can give me more details.

I have found a workaround for this issue, for anyone just at the beginning of the migration process. Duplicate the Drupal 7 site, and turn off all ldap modules. Then go to the uninstall tab and uninstall all ldap modules. This removes all the tables from the Drupal 7 database. After migration, ldap can be added and configured on the Drupal 8 site without issue.

Unfortunately I can't spare the time at the moment to try to set up a new site for testing the migration.

Attached is an updated patch if someone wants to run this on installation and my comments were misunderstood. One might have to move the update hook at the beginning of the install hook if Migrate has already installed those field definitions before activating LDAP (not sure).

I'd appreciate test feedback.

@grahl
Thank you for providing the patches!
I tried both patches on installation on a migrated site from D7. The site always returns the error described above when I enable the ldap_user module.

Once I uninstall the ldap_user module and run "Drush entity" the error is gone. Any idea why I can't install the ldap_user module?

Maybe I applied the patches wrong, so here is a short description on the two ways I applied them:
1.1. composer require drupal/ldap
1.2. Apply patch
1.3. drush en ldap_servers ldap_user authorization
1.4. Cache rebuild

2.1. composer require drupal/ldap
2.2. drush en ldap_servers ldap_user authorization
2.3. apply patch
2.4. drush updb
2.5. Cache rebuild

Both times I get the following error:

Drupal\Core\Field\FieldException: Attempt to create a configurable field of non-configurable field storage ldap_user_puid_sid. in Drupal\field\Entity\FieldConfig->getFieldStorageDefinition

I hope you can spare the time to take a look into patching this because I am almost done with the migration and it would be really labour intensive to redo the migration just to make the ldap server work.

If you need any logs, please let me know!

Another thought on the side:
Why does the module ldap_user create columns in users_field_data? The D7 LDAP Configuration was stored in tables with the exact same names as the columns now. I think that just deleting the old tables won't solve the issue. I've dropped them manually and still experienced the same issue as before.

I had some similar problem. I also migrated site from D7 to D8 with LDAP installed. This was creating similar problem for me, Attempt to create a configurable field of non-configurable field storage ldap_user_puid_sid.

I tried to delete the said tables from #8 directly from database, but I still could not access the site except for configuration pages. Navigating elsewhere would show that error again. I got atleast my site to work again by playing with these following settings before installing LDAP on Drupal 8 site.
I went to admin > config > account settings. In the manage field and manage form display tab, I deleted all the fields that had ldap. Then in the database, I deleted all the tables that had ldap stored, some cache tables and migrate tables. After this, I installed the LDAP module again and the site did not break. Please make database backup before changing anything.

did anyone found a solution for this?, just ran into this with a migrated D7 to D10 site...

also getting this in a D7->D10 migration with LDAP. will attempt removing LDAP from D7 before the user migration.