Currently the name field for the default active store is a 255 character varchar.

- Is this long enough? I can see some situations where this could get pretty long, especially when appending dynamic data like view names or node types and the like.

- If we make it a lot longer, does this affect performance of selects on this table? We're going to be selecting on this field a lot.

Discuss.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

sun’s picture

Title: Size of name field in default active store » Size of name field in default active store can be too small
Project: » Drupal core
Version: » 8.x-dev
Component: Config system » configuration system
Priority: Normal » Critical
Issue tags: +Configuration system

1) ACK. We should definitely discuss this. Potential option: Force-limit config object names.

2) NACK.

Bumping to critical, since this question is pretty essential.

gdd’s picture

Well I think there's no doubt it has to be longer. The field is our primary key so it will be indexed, and I don't think it will affect performance as long as we can leave it a varchar. As of MySQL 5.0.3 varchars can be up to 65,535 so its just a matter of how long we want to go. Chx just pointed out to me that Oracle has a limit at 4096 characters, so we should probably take that as our upper limit unless other supported databases are worse.

Anonymous’s picture

i don't think we're looking at this from the right angle.

as we've decided to force filenames into the base API, the constraint here is actually filesystem file name length.

gdd’s picture

Status: Active » Closed (won't fix)

and having researched this a bit, it appears that max file name length on many still-popular filesystems is ... 255 characters.

I think we should close this and see if it becomes an issue. If it does, we can reopen and investigate some other options. One way or another though, we'll have this limit for the file names.

sun’s picture

Priority: Critical » Normal
Status: Closed (won't fix) » Active

I don't think that won't fixing this issue is the appropriate action.

Post-Barcelona, we're not only talking about filenames, but also about context parameters within filenames.

I just tested how many characters 255 characters actually are and was only able to reach ~100 characters for a config object name without context parameters; i.e., leaving another ~100 for those. But of course, that was only a quick test.

The most concerning parts actually are the rather unpredictable lengths of machine names within config object names. E.g., taking a field instance config object, there's at least

field_name (32) + entity_type (32) + bundle (128) [err...128!?] == 192 chars

alone for these identifiers; i.e., not even counting the static config name parts in between, which are likely going to be along the lines of

field..entity_type..bundle. == 28 chars

i.e., 220 chars in total — without any context parameters

gdd’s picture

I had forgotten about the context parameters, yes. I can't believe bundle machine names are 128, that seems a little excessive. However I'm sure there are other places we might run into this.

I'm open to suggestions. One thought put forth by chx was putting each piece of the config name into a directory. So instead of

image.style.large.yml

you might get

image/style/large.yml

Which would give us 255 characters at every level of nesting. I really hate this though. Would we then force module developers to also build this structure into their provided config? I think we would have to for consistency, and that would be pretty irritating. The DX around supplying config is already not great.

Other ideas?

xjm’s picture

So I went back and forth about this.

  • I think chx's solution is the only way we can guarantee that 255 characters are available for every namespace component.
  • However, I really think that change would kill not only CMI's DX but also the UX.
  • We could bake something into drush or even a shell script into core to automatically read and write these paths.
  • However, one of the big wins in CMI currently is that it works well for everyone, even poor folks on shared hosting (which is a lot of Drupal sites). Right now all they have to do is find and copy that one file, which they can do even with FTP or browser-based hosting managers.
  • @merlinofchaos, @heyrocker, and I even discussed a UI feature that would simply give the user a downloadable, properly-named config file. Then all the user has to do is browse to the hashed config directory and drop that file in place. Anyone can do it. Drupal is easier.

    Using a nested directory structure would totally kill that.

So, I personally think we need to find a way to live with 255 characters. Make namespace components short. Cap machine names at a sane length. Put additional machine-name generating API in core at a low level to enforce shorter lengths. Make bundle names shorter if we have to.

We discussed what an upgrade path to shorten the bundle name field would look like.

  1. Truncate bundle names that are over X characters. (64?)
  2. When there are duplicates:
    my_really_stupid_long_bundle_name_whatever_stuff_bunnies becomes my_really_stupid_long_bundle_name_whatev_01
    my_really_stupid_long_bundle_name_whatever_stuff_frogs becomes my_really_stupid_long_bundle_name_whatev_02
  3. Since bundle names are in code all over the place (this is the yucky part), we add a database table that maps old bundle names to new. It's gross, but I think it's the edge case. Way better to live with that than killing the usability of CMI for everyone.

Not sure how to handle the context bit. Maybe we hash it and list the context hashes in the main file? Maybe contexts get a subdirectory (one, no nesting) and we live with that slight decrease in UX with the assumption that people building sites complicated enough to use it can figure out how to use an automated tool to put the file in the right place? I don't know.

Thoughts?

sun’s picture

This has a decent amount of implications on the rest of the system. There has to be a definite answer and definition before release, so tagging accordingly.

The final answer will slightly depend on other issues; e.g., #1479492: [policy] Define standards for naming and granularity of config files

The most straightforward solution would be to say that configuration object names have a maximum length of 250 ASCII characters (255 minus dot minus 3-4 (yml|xml|json) file extension. This restriction would have to be enforced in Config::save() with strlen() (enforcing ASCII bytes length) and throwing an exception if it exceeds that length.

That would leave the problem to the rest of the system to deal with. But of course, it's not that simple — machine name maximum lengths have to be actually shortened/limited all over place.

Before we think about an advanced mechanism for shortening + mapping old/longer machine names, we should gather actual data from D7 production sites, to make sure we're not introducing a mapping mechanism for a very rare edge-case.

Additionally, context overrides would likely have to be moved into sub-directories; i.e.:

./language=de/system.site.yml
./domain=example.com+language=fr/system.site.yml
./system.site.yml
gdd’s picture

Putting the context in subdirectories is not a bad idea, it only complicates the DX for a small percentage of sites. I just referred that issue here for further discussion.

neclimdul’s picture

the syntax of those directories is a bit terrifying but it seems like a pretty clever idea putting the "context" first making domains of stores(domains as in logical groupings). That seems possibly more useful then the additional values on the key and it makes finding any system.site settings as easy as running find . -name system.site.yml or doing some similar directory scans. So there's a possible win for some class of user.

David Strauss’s picture

I've read over this issue with some guidance from Greg. Here's what I've concluded:

There isn't a good case for the basic (non-contextualized) configuration path to get split out with directories. The closest I see here is for fields and bundles, and even that case has 20+ characters of headroom. Extensive directories and recursion are a definite DX/UX penalty I'd like to avoid.

It might be a good idea to split out the context into a folder. It's clearly possible for a module to approach a 255-character configuration path and for a site to use context that pushes the combined length too far. Two pieces of code that behave within documented limits shouldn't be possible to combine into something that fails.

The current context patches (#1616594) make my eyes glaze over in their complexity, especially the additions to bootstrap. I'd prefer to start with something like config($name, $context = ''). That way, we can use folders for context, minimize effect on how modules ship default configuration (a big DX issue with deep directories), and still avoid context+config exceeding 255 characters.

sun’s picture

FWIW, I'd also be fine with calling this fixed, on the grounds that it's only the context stuff where the length can become problematic, and so that should be using one dedicated sub-directory per context.

If we all agree on that, then what I'd like to see in this issue is a patch that enforces the maximum length of 250 ASCII/single-byte characters [NOT unicode/multibyte length, i.e., intentionally not using the drupal_strlen() wrapper], e.g., in Config::save() and bails out with an uncaught exception if the maximum length is exceeded.

That is, because this should fail very visibly, as we want to prevent a situation in which Developer A on OS/platform/storage A says "works for me" and User B on OS/platform/storage B says "WTF, not at all."

And of course, this should be covered by a test. ConfigCRUDTest is probably a good home for that.

joachim’s picture

> I can't believe bundle machine names are 128, that seems a little excessive.

There's no limit set on entity types and bundle names. See #1709960: declare a maximum length for entity and bundle machine names.

xjm’s picture

xjm’s picture

Assigned: Unassigned » xjm

I'll do as suggested in #12 and file a patch + tests to throw an exception when trying to save a configuration object with a name over 250 characters.

xjm’s picture

Status: Active » Needs review
FileSize
3.18 KB
2.03 KB
sun’s picture

Hm. Perhaps we should get in #1701014: Validate config object names first - or merge the two? They both seem to add validation for the same thing.

xjm’s picture

Sure, I'll add this to that issue and standardize a bit.

xjm’s picture

Title: Size of name field in default active store can be too small » [META] Length of configuration object name can be too small
Status: Needs review » Active

#16 is now included in the other patch. Re-scoping this issue to continue to track all the issues that affect the object name length.

xjm’s picture

Pancho’s picture

Issue tags: -Release blocker

After re-scoping, no longer a "Release blocker".

xjm’s picture

The tag "Release blocker" doesn't make sense anyway; that's what the "critical" issue status means. :)

jibran’s picture

Status: Active » Needs review
Issue tags: -revisit before beta, -Configuration system

#16: config-1186034-16.patch queued for re-testing.

Status: Needs review » Needs work
Issue tags: +revisit before beta, +Configuration system

The last submitted patch, config-1186034-16.patch, failed testing.

jibran’s picture

Issue tags: +Needs reroll

Tagging.

mtift’s picture

Title: [META] Length of configuration object name can be too small » Length of configuration object name can be too small
Issue tags: -revisit before beta, -Needs reroll

This doesn't feel like a meta issue to me, so I'm removing "meta" from the title. There are definitely other related issues, but we already have a meta issue for those -- and because this issue concerns installation, it seems like this issue should be a child issue of #1776830: [META-1] Installation and uninstallation of configuration provided by a module that belongs to another module's API.

Also, this issue does not really need a re-roll because the code from #16 was incorporated into #1701014: Validate config object names (although the class name changed from ConfigNameTooLongException to ConfigNameException).

The fact that we've gotten by since this issue was opened in June 2011 suggests to me that this character limit might not be a problem -- at least in Drupal core -- and that we can continue to enforce the 250 character limit in configuration file names.

mtift’s picture

Then again, maybe it isn't related to #1776830: [META-1] Installation and uninstallation of configuration provided by a module that belongs to another module's API because it doesn't concern one module's connection to another module's API.

mtift’s picture

xjm’s picture

Issue summary: View changes
Issue tags: +Naming things is hard