I am very honored to be granted the privilege of taking the lead on the module packaging and dependency system. After collaborating with Adrian and acquainting myself with past discussions and work already contributed, I'm very excited to get this functionality in place. Without your support we will not be successful so I urge you to please give feedback and suggestions and most of all your support so that we can speed this contribution up to completion.

You can visit the following links if you want to familiarise yourself with where we are at today.

http://drupal.org/install-system-overview - Adrian's initial overview of the install system.
http://drupal.org/node/34198 - Link from dikini (Vlado) about his presentation at DrupalCON.
http://dikini.net/node/46 - The presentation and notes.
http://dikini.net/tags/drpkg - drpkg tag
http://lists.drupal.org/pipermail/development/2006-July/017908.html - latest discussion on drupal_dev

Focus: This task is only focusing on package descriptors. The layout and content of the file which will define a module's meta data and its dependencies.

Outcome: The successful outcome of this task is an agreed upon standardisation of package descriptor files; what these files look like, where they are stored, their content, etc. and generated with as much information as we can possibly obtain patched to the repositories.

This is the first step in getting the dependency system in place and if we take the whole project in smaller bite sized chunks to chew on and digest it will aid in keeping it simple and easier to stay focused and finalise as soon as possible.

You are still encouraged to continue posting ideas and comments that you may have regarding the dependency system, packaging, post installation process, updating process, module administration, module loading process or anything related on drupal_dev and the relevant forums, we will be paying attention to these and your information will aid in streamlining this project so that we can complete the outstanding tasks before code freeze.

Current assumptions:
We have decided to use the .ini file format as php has built in support for parsing these.
Each module is placed in its own directory and has a package.ini file as descriptor.
By using a modified subset of the drpkg subsystem files are being generated as initial descriptors for modules.

Generated content:
module name (from module file)
repository (contrib,core)
description (hook_help admin/modules#description)
path (directory on repository)
dependencies (module namespace in function names)

Questions:
How are we going to describe themes?
Any advice on obtaining more dynamic information from current modules?
Any meta data that should be added to the descriptors?
Any meta data that should be removed from the descriptors?

Examples of a few package.ini files are attached for your review. We are looking forward to your suggestions and feedback.

Nick

CommentFileSizeAuthor
#11 meta.tar.gz2.42 KBnickl
descriptor.tar.tgz748 bytesnickl
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Steven’s picture

PHP's ini parsing functions contains way too much voodoo magic and hidden syntaxes. PHP constants are filled in, various keywords are replaced (e.g. 'yes', 'on', ...), keywords are reserved and even boolean expressions are evaluated. On the other hand, PHP's ini syntax is restrictive. According to the docs, all values containing "non-alphanumeric characters" must be surrounded by quotes.

Judging from the example files, the idea is to put varied content in the ini fields including pieces of text. INI files as defined by PHP are not suitable for this as they have no support for multi-line values.

Why not just use our own format? I propose the following... no hidden replacements, a much more forgiving syntax when it comes to quotes, newlines and spaces, and support for multi-line values:

$theme = <<<DOC
// Comments are supported
  // with single-line PHP syntax

Key:Value pairs
With: Spaces
On :Either
Side : Work like you expect them to // with comments at the end too

Double-Quotes : "Can be used to \"surround\" a value"
And are : "needed to use two slashes inside a value // like this"
Double-Quotes : are returned as part of the value " when not at the beginning "

Single-Quotes : 'Can be used to \'surround\' a value'
And are : 'needed to use two slashes inside a value // like this'
Single-Quotes : are returned as part of the value ' when not at the beginning '

Multi-line values: "Are supported when you use quotes
to surround them."

The value:
can be placed on the next line too

This works: 
"For multi-line
values as well."

DOC;

if (preg_match_all('@^\s*((?:[^:/]|/(?!/))+?)\s*:\s*(?:("(?:[^"]|(?<=\\\\)")*")|(\'(?:[^\']|(?<=\\\\)\')*\')|([^\n]*?))\s*($|//)@ms', $theme, $matches, PREG_SET_ORDER)) {
  foreach ($matches as $match) {
    list(, $key, $value1, $value2, $value3) = $match;
    $value = stripslashes(trim($value1, '"')) . stripslashes(trim($value2, "'")) . $value3;
    print("$key => $value\n");
  }
}

Morbus Iff’s picture

Why package.ini and not modulename.ini?

Morbus Iff’s picture

The "repository" seems like a useless bit of info, especially when it's controlled by the user. Can't we just whitelist, instead, all of the core modules, and assume everything else is contrib? That seems the smartest way to save space and logic, otherwise we're gonna have data duplication across all of contrib (stating that yes, modules in contrib, are, um, in contrib).

Also, I'll throw another vote out for disliking .ini files.

How are we handling translations?

Morbus Iff’s picture

The "maintainer" name could be inferred from the owner of the project, again making this another bit of housekeeping that we don't need to keep in the .ini files - look up the "path" in the drupal.org database, find the project that matches that path, and suck out the maintainer from the owner of the project. Downsides: assumes all modules have a project (which, in the case of some of mine, don't), and assumes that Drupal.org usernames are "good enough".

I'm presuming the "size" and "MD5" would be autogenerated by the cvs packaging script?

I'm not sure I like "installnotes" - are we replacing the stronger INSTALL.txt?

Can't "path" be autogenerated by the cvs packaging script too?

merlinofchaos’s picture

This issue is related to http://drupal.org/node/76340 where I've been working on UI redesign.

The UI is dependent on the layer beneath it, particularly in how things are implemented. I've successfully implemented a simple dependency system already.

joe.murray’s picture

Eventually we want to be able to identify contrib modules that play nice together, and those that don't. Often those responsible for modules know that already for other popular contrib modules. Letting module maintainers state some this early on would be useful - at least warnings could be generated when building an install known to not work, and a format could be established that would later allow more automated test suites to state modules have been verified as working together to a certain standard.

pwolanin’s picture

A question about the initial description:

"dependencies (module namespace in function names)"

Does this mean trying to guess modules based on function names, or will there be some grand mapping of all function names (in core and contrib) to their containing module?

Such a mapping actually might be useful to find modules that are incompatible because of (inadvertant) use of the same function names.

gopherspidey’s picture

Sorry if this has been answered or discussed in the past, but why are ini files used.

Why don't we just have a _meta function in the the .install file that returns an array of terms. Then we don't have to read and parse the file manually, we just call the function when we want the information.

So thing simple like this.

function drupal_meta () {

$ret = array();

$ret['repository'] = 'core';
$ret['maintainer'] = 'maintainer name';

......

return $ret;
}

Morbus Iff’s picture

spidey: in a nutshell, the .ini files would be parsed and read as part of drupal.org, and executing arbitrary PHP in hundreds of contrib modules on a production website which serves as a developer forge is not safe in any way, shape, or form.

gopherspidey’s picture

Morbus Iff: Thanks for the explaination. I knew that I was missing something.

nickl’s picture

FileSize
2.42 KB

Thank you for all the comments and feedback.
We have progress:
We've implimented Steven's parser and the file format currently looks like this:

description: UI for the views module
name: views UI
package: views
version: 0.9.1
drupal: 4.8
dependencies: views
links: administer = admin/views,configure = something/else

This is the minimum content as implimented on the admin modules screen. You are still welcome to suggest any additions to this file and to see them in operation by following this thread: http://drupal.org/node/76340

Module descriptors are stored in their module directories as modulename.meta.

Attached are a few meta descriptors needed to test the current admin modules screen patch.

Please don't hesitate to make more comments your input is valued and appreciated.

pwolanin’s picture

How much of this is generated automatically, and how much by hand?

merlinofchaos’s picture

At the moment, the entire .meta file needs to be generated by hand. There are 2 reasons for this.

1) Automatic generation of some of this data is tricky and would be quite a bit of code AND
2) project.module has other things that need to happen before any of this can be done. WHich means
3) manual generation of the .meta file is currently the best way to get improvements into 4.8.

nickl’s picture

Status: Active » Closed (fixed)

Done and comitted to head : http://drupal.org/node/80952