[Meta] Document format/content of various YML files

I don't think I ever filed an issue for it, but libraries.yml is not the only YAML file in Drupal that could use better documentation. I shared the following idea with others a couple of times:

Let's create a /core/docs directory and create YAML files to document YAML files in there; e.g., /core/docs/libraries.yml

These YAML docs files resemble the full schema of the actual corresponding YAML file. They use inline comments above each node/property to summarize/describe it and point out possibly existing constraints (characters, length, etc).

If multiple, completely different variations of a node/property are possible, then docs file simply specifies the same property more than once.

In other words, /core/docs/libraries.yml would go like this: (incomplete, just to get the point across)

# Declares asset libraries supplied by an extension.

# The name of the library. Referenced in PHP code.
# All libraries are anchored and always referenced by extension, so the
# extension name does not need to be repeated in library names.
# For custom libraries, the name should denote its purpose and should be
# prefixed with 'drupal.' if the library depends on Drupal.
# Common examples are 'drupal.admin' and 'drupal.front'.
drupal.admin:
  # Specifies a custom version; defaults to the extension version.
  version: VERSION
  # JavaScript asset files to load for this library.
  js:
    # Each key is the path to an asset file, relative to the extension
    # directory. Each value are options for the file. Files are loaded in
    # the specified order.
    js/filter.admin.js:
      # Denotes that the file is minified already.
      minified: true
    js/filter.admin2.js: {}
  # CSS asset files to load for this library.
  css:
    theme:
      css/filter.admin.css: {}
  # Dependencies to load before this library.
  dependencies:
    - core/jquery
    - core/drupal
    - core/jquery.once
    - core/drupal.form

# For external libraries, the name should be the proper, lower-underscored
# name of the library. The name should be prefixed with the library name
# of the framework (delimited by a dot), if any.
jquery.ui.dialog:
  remote: https://github.com/jquery/jquery-ui
  version: 1.10.2
  license:
    name: MIT
    url: https://github.com/jquery/jquery-ui/blob/1.10.2/MIT-LICENSE.txt
    gpl-compatible: true
  js:
    assets/vendor/jquery.ui/ui/jquery.ui.dialog.js: {}
  css:
    component:
      assets/vendor/jquery.ui/themes/base/jquery.ui.dialog.css: {}
  dependencies:
    - core/jquery.ui
    - core/jquery.ui.widget
    - core/jquery.ui.button
    - core/jquery.ui.draggable
    - core/jquery.ui.mouse
    - core/jquery.ui.position
    - core/jquery.ui.resizable

How do you propose that people would magically discover that this documentation exists?

Also, this is really a coding standards issue (we need to decide how to document YML formats and files), so adding proper issue title and tag.

How do you propose that people would magically discover that this documentation exists?

How do you propose that people would magically discover that core.api.php and other documentation exists?

I don't see a difference. Documentation for PHP files lives in some special PHP files. Documentation for YAML files lives in some special YAML files.

Well if you go to api.drupal.org, you don't have to know about the special PHP files. There are topics for Hooks, etc. All I'm trying to say (and not saying it very well) is that we need to think about what to do on api.drupal.org to make sure developers are aware of this documentation. Which means mentioning this convention in a topic, or something like that.

I considered YAML schema files as well. I do think we should have properly defined schemata for these files (e.g., to implement validators).

However, a schema file is strictly restricted to describing the schema only. You are not able to document alternative use-cases, as demonstrated in #1; e.g., the declaration of an internal/custom asset library looks vastly different to an external asset library.

We can certainly reference a YML file being used in documentation in a @defgroup topic, so documenting things in a YML file is not a show-stopper. Just something to consider.

I think I'm in favor of writing the documentation in /core/docs/libraries.yml or similar as per sun's original comment and then referencing them in the @docblock comments. My initial reaction is that it makes more sense to write the documentation for YAML files in a YAML file, that way they serve not only as documentation of the properties but also as examples of the syntax/etc that can be copy/pasted and modified for your own use case.

Can someone make a formal proposal for what this documentation would look like and how someone would discover it?

I could be a bit off base here so bear with me. But here's my current thinking.

Right now, all of the things we use YAML files for are already likely to be high-level topics on the D8 documentation. https://api.drupal.org/api/drupal/groups/8

So, we could:

1. Create core/docs/example.routing.yml which serves as comprehensive documentation for the options that can be used in the routing.yml file. This would be pretty much akin to have the primary documentation for hook_menu() be the @docblock for hook_menu() which lists all the possible options you could put into your menu items array.
2. We update the topic @docblocks that reference these YAML files to point to the provided example.*.yml file. This page for instance could be updated to point to https://api.drupal.org/api/drupal/core%21includes%21menu.inc/group/menu/8, maybe replacing this part, "See https://www.drupal.org/node/2092643 for more details about *.routing.yml files ..." of the current documentation on that page. The YAML file would have canonical documentation, unlike what we have no where some properties are documented in the topic @docblock, and some on the linked d.o. page, but neither contains everything.
3. We make the necessary changes to api.drupal.org to allow for the display of the contents of a YAML file inline on the page when viewed. Example: https://api.drupal.org/api/drupal/core%21modules%21action%21config%21sch... , could be as easy as just toggling the view source open by default on YAML pages? (Note, this page was just selected because it's the first YAML file I found on api.d.o)

I think this sounds like a reasonable plan, with the possible exception of item 3 (I just don't know if it's feasible/practical/a good idea to make the "view source" be toggled open in the API module just for YML files).

Probably we can make the wording of item 2 something like "blah blah blah is documented in the source of this example file: example.routing.yml, which can be found in the core/docs directory", to take care of this problem?

I think it makes it easier to read/scan api.drupal.org if the info is just there and you don't have to click to expose it. But either way works for me.

So then, here's a list of .yml files we'll need to create for this. There may be others, but this is what I'm aware of right now. Should we open a new issue for each? Or do them all here?

- core/docsexmaples.breakpoints.yml
- core/docsexamples.info.yml
- core/docsexamples.libraries.yml
- core/docsexamples.links.action.yml
- core/docsexamples.links.contextual.yml
- core/docsexamples.links.menu.yml
- core/docsexamples.links.task.yml
- core/docsexamples.permissions.yml
- core/docsexamples.routing.yml
- core/docsexamples.services.yml

Where do we draw the line between this and examples.module?

Also, please do not use docsexmaples, I won't be the only one to read that docSEXamples

I don't think we actually need all of those. There is documentation elsewhere on how to make contextual links, action links, menu links, and task links. ...

So... Actually, the more I think about this, the more I'm unsure that this is even the best way to go. I think writing @defgroup topics for things like "Using libraries" would make more sense than trying to document this in an example .yml file.

For instance, we already have:
https://api.drupal.org/api/drupal/core!includes!menu.inc/group/menu/8
That tells you how to do the links things, as well as routing. We also have a ton of examples in Core of *.routing.yml and the *.links.*.yml files. Why would we need another one?

So maybe we should just do something similar for the other YML files as well -- make sure there's a topic about them that describes how to create them, and rely on Core to have many examples... and the Examples project if we need more details?

Or do we really need extensive documentation like in comment #1 for each of these? I just don't see the value...

So... Actually, the more I think about this, the more I'm unsure that this is even the best way to go. I think writing @defgroup topics for things like "Using libraries" would make more sense than trying to document this in an example .yml file.

Well, there are different kind of usecases if you ask me.

A @defgroup topic is really helpful in case you want to kinda dive deeper into the topic, understand backgrounds etc.
For many usecases though examples really help, as you just need to copy and adapt them for your usecases. Examples like libraries.yml are potentially hard, even
for people which belongs there. Having an example right in your face, reduces the amount of potential problems (see #2389203: Validate the CSS categories in libraries.yml files.)

So I would support your idea to extensively document features in @defgroups, but still provide some easy to use examples.
I would argue that in examples you can more easy grasp the entire spectrum of possiblities.

I think that's fine, but if we have decent handbook documentation elsewhere, does this issue need to be critical?

We do not generally provide detailed easy-to-use examples of things in Core, beyond the files that Core actually uses. So we have examples of *.libraries.yml files in core, as well as many other types of yml files... I don't think we really should need to provide more. In some documentation, we've done things like "See class Foo for an example of how to do this"; we could do something like that.

And if a more detailed example is needed, I think our philosophy in the past has been to put this into the Examples project.

RE #22, I do not believe we have any good documentation for the libraries yml files on drupal.org or anywhere else at this time.

I would handle this like we do .info files. Have a nice handbook page with copy/paste examples + documentation of what's there, link to it from whatever the most relevant API topic page is on api.d.o.

Sure, on #24. Generally, the idea for API documentation in Drupal Core is:
- Make sure there is a topic (@defgroup in a core api.php file) that gives you the bare minimum documentation for that API -- not necessarily all the details, but at least the basic syntax/usage/etc.
- Make sure the topic is linked from the landing page on api.drupal.org for Drupal 8, or that there is a logical path to go through to find it (e.g., if you were documenting the API for making menu links, it should either be in a topic with "menu" in the name that is listed on the api.d.o landing page, or if you go to the "menu something" topic that is listed there, that page should link to this other topic that has this information in it).
- From the "bare minimum" @defgroup topic, link to more detailed documentation on d.o. The page on d.o should give you more details, in more of a tutorial style.

Right now we have neither d.o pages nor topics for at least some .yml files. That is why this issue is critical.

I think the next step is probably to look into which yml files need information and make a list of topics and d.o pages we need to make, so I'll do that shortly and post a separate comment.

OK, here's a list of the types of yml files we have, and what documentation we have about them:

(a) *.info.yml (module/theme/profile .info files) Hm... https://api.drupal.org/api/drupal/core!modules!system!core.api.php/group... links to the theme guide and module developers guide, and there is documentation in both about *.info.yml files. The link to the install profile/distribution guide leads you to a page that is more about how to use them than how to create one; should probably be linking to https://www.drupal.org/node/159730 instead, and that has docs on the *.info.yml file). So... We also don't have any docs on api.d.o for the syntax of the module, theme, and profile .info.yml files. Is that OK or not? Anyway, we should fix the install profile link for sure, and decide whether api.d.o should have documentation itself on the syntax of *.info.yml files for modules, themes, and distributions (I think it probably should have at least a little?).

(b) These next 5 are all documented on https://api.drupal.org/api/drupal/core!includes!menu.inc/group/menu/8 :
*.links.menu.yml
*.links.task.yml
*.links.action.yml
*.links.contextual.yml
*.routing.yml

(c) *.services.yml - documented on https://api.drupal.org/api/drupal/core!modules!system!core.api.php/group...

(d) *.libraries.yml - https://api.drupal.org/api/drupal/core!modules!system!theme.api.php/grou... mentions how to attach libraries, but there is no documentation on how to define a library. Should add that.

(e) *.permissions.yml - mentioned on https://api.drupal.org/api/drupal/core!modules!system!core.api.php/group... but there is no documentation about the format. Should add that.

(f) *.breakpoints.yml - As far as I can tell, there is zero documentation on api.d.o for these files or breakpoints in general. We probably need to add a topic about breakpoints and responsive stuff.

(g) These next ones are documented on https://api.drupal.org/api/drupal/core!modules!system!core.api.php/group... with links to more details on d.o:
config/schema/*.schema.yml
config/install/*.yml
(there are also some views configs in test modules that are not in config/install directories but instead are in *_test_views directories, but I'm not going to worry about them -- they're config files anyway)

Filed 4 child issues for the missing docs, changing this to a Meta.

Dicussed with @xjm, @webchick, @catch and @effulgentsia.

We decided that the only critical part of this is #2390239: No information about format of permissions.yml in permissions topic since it concerns security - so I'm making that critical. #2390241: No documentation on how to define a library is major since defining libraries is an important task for a developer. Leaving #2390247: No documentation on responsive / breakpoints on api.d.o at normal.

@jhodgdon happy to discuss this is you think want I've done is a bad idea.

Sounds good to me. We can also make this issue (since it is a meta and the child issues have been prioritized) Normal?

What about having dummy documentation files, the same way we have foo.api.php? So we'd have api.routing.yml, api.permissions.yml, and so on?
Within those, same general idea as in api.php: documentation of the structure, followed by a sample.

RE #31, we could do that, but since the *.yml file are parts of APIs, and require you to also define classes (controllers, entities, whatever), it seems like they can be documented more logically as part of an API topic. In any case, we already have documentation for nearly all of them done; there is only one child issue open.

> In any case, we already have documentation for nearly all of them done; there is only one child issue open.

I can't find comprehensive documentation on the various menu.yml files.

This is nice: https://www.drupal.org/node/2122241 but not all properties are covered.]

It *looks* like the properties and their defaults are in the code in Drupal\Core\Menu\MenuLinkManager::$defaults.

From api.drupal.org d8 landing page, click on User interface > Menu entries, local tasks, and other links. This will take you to
https://api.drupal.org/api/drupal/core!lib!Drupal!Core!Menu!menu.api.php...
which has good documentation on the various menu.yml files, I think.

It's good, yes, but it's not *complete*.

For a menu link, it shows these properties:

  title: 'Recent log messages'
  parent: system.admin_reports
  description: 'View events that have recently been logged.'
  route_name: dblog.overview
  weight: -1

MenuLinkManager::$defaults shows that all these properties can be in a menu link definition in the yml file:

  protected $defaults = array(
    // (required) The name of the menu for this link.
    'menu_name' => 'tools',
    // (required) The name of the route this links to, unless it's external.
    'route_name' => '',
    // Parameters for route variables when generating a link.
    'route_parameters' => array(),
    // The external URL if this link has one (required if route_name is empty).
    'url' => '',
    // The static title for the menu link. If this came from a YAML definition
    // or other safe source this may be a TranslatableMarkup object.
    'title' => '',
    // The description. If this came from a YAML definition or other safe source
    // this may be be a TranslatableMarkup object.
    'description' => '',
    // The plugin ID of the parent link (or NULL for a top-level link).
    'parent' => '',
    // The weight of the link.
    'weight' => 0,
    // The default link options.
    'options' => array(),
    'expanded' => 0,
    'enabled' => 1,
    // The name of the module providing this link.
    'provider' => '',
    'metadata' => array(),
    // Default class for local task implementations.
    'class' => 'Drupal\Core\Menu\MenuLinkDefault',
    'form_class' => 'Drupal\Core\Menu\Form\MenuLinkDefaultForm',
    // The plugin ID. Set by the plugin system based on the top-level YAML key.
    'id' => '',
  );

OK. I think that we should leave that topic as it is -- the topics on api.drupal.org are meant to cover the basics, not be totally comprehensive, and I think that what is there covers the 90% of module developers use case.

Our policy has been to link those topics to more comprehensive docs on drupal.org. That topic links to
https://www.drupal.org/developing/api/8/menu
whose subpages are meant to give full details. So, the full list of properties cited in #36 should probably be documented on https://www.drupal.org/node/2122241

We can either file an issue in the Documentation project to do that (it could be a child of this issue), or if you have time, maybe you can go ahead and edit it? Thanks!

> Our policy has been to link those topics to more comprehensive docs on drupal.org.

Those are good too, but I feel that not documenting YML files within our codebase is a step back from our previous documentation standards.

We discussed this already in this issue, and came to the conclusion that is in the issue summary. Are you saying you want to start the discussion over again?

Yes, I think this needs to be reconsidered.

I think the reasons we have for documenting PHP code and hooks within our codebase also hold for YML files. The more expository docs we have on d.org are definitely needed, but they don't cover everything, and it's much easier for them to get out of date.

There's only one child issue left, maybe better to close the issue?

Also it looks no longer major

My proposal at #2823463: Devise an extensible way to document extensible data structures would cover this issue.

Also, I think this is a massive shortcoming in our docs on D8, so I'm not sure about downgrading the priority.