This issue is about considering to use or not Doctrine ORM(Object-Relational Mappers) for entity mapping to database schema.

Problem/Motivation

We have built over the past home made Drupal ORM for Entity management. Right now, we are thinking about replacing hook_entity_info() arrays with annotations, requiring entities to be plugins. We should use the plugin system sparingly as it will set the tone for community.

Drupal as of version 6, implements an ORM. Hooks can add data to nodes from anywhere they want, whether it maps to a database table or not. However, the ORM is "naked": there is no abstraction on data access. Everything is loaded at once into a single big blob data that gets returned from node_load().

Drupal as of version 7 improved a lot this. The Entity API matches the entity structure to the relational database and adds an abstracted layer for CRUD operations, but, the loading of an entity is still not optimized.

Drupal in its 8-dev version changed the entity_info() arrays into annotations, so this is a good step and a good direction. Entities are currently re-factored against the plugin system, using a EntityManager to govern persistence. It is exactly how Doctrine ORM actually work.

Because an ORM is more robust, it can do a lot more optimization under the hood. Complex relationships can be modeled in a database-independent way, even tying to non-SQL systems like the file system, Services, etc.

  • The Entity system is really complex, you have to first extend a base class or implement the EntityInterface.
  • Entities themselves contains reference to controllers, via annotations, this should not appear in a data object.
  • Annotations are not used on a per-field basis, but as a meta-data stack which helps the entity manager to understand how to deal with the Entity.

Refactored solution

  • Does not need to extend a base class or to implement an interface.
  • Generates Physical Data Model from classes themselves.
  • No need for Schema API anymore in the long run.
  • Uses annotations the correct way.
  • Have a meta-data caching system included.

Entities are generated and looks like the following:

<?php

namespace Drupal\entity;

use

Doctrine\ORM\Mapping as ORM;

/**
 * Node
 *
 * @ORM\Table(name="node")
 * @ORM\Entity
 */
class Node
{
 
/**
   * @var integer
   *
   * @ORM\Column(name="nid", type="integer", nullable=false)
   * @ORM\Id
   * @ORM\GeneratedValue(strategy="IDENTITY")
   */
 
protected $nid;

 

/**
   * @var \Drupal\entity\User
   *
   * @ORM\ManyToOne(targetEntity="Drupal\entity\User")
   * @ORM\JoinColumns({
   *   @ORM\JoinColumn(name="uid", referencedColumnName="uid")
   * })
   */
 
protected $user;
}
?>
<?php

namespace Drupal\entity;

use

Doctrine\ORM\Mapping as ORM;

/**
 * User
 *
 * @ORM\Table(name="users")
 * @ORM\Entity
 */
class User
{
 
/**
   * @var integer
   *
   * @ORM\Column(name="uid", type="integer", nullable=false)
   * @ORM\Id
   * @ORM\GeneratedValue(strategy="IDENTITY")
   */
 
protected $uid;

 

/**
   * @var string
   *
   * @ORM\Column(name="name", type="string", length=60, nullable=false)
   */
 
protected $name;

 

/**
   * @var string
   *
   * @ORM\Column(name="pass", type="string", length=128, nullable=false)
   */
 
protected $pass;

}

?>

Remaining tasks

Decide if we want to go in this direction or not.

Resources

Comments

Sylvain Lecoy’s picture

Issue summary:View changes

Detailed 'arrays'.

pounard’s picture

+1

pounard’s picture

Issue summary:View changes

detailed 'do this'.

Crell’s picture

Status:Active» Postponed

I stumbled across this via Google in response to a twitter discussion. Bizarre... :-)

You forgot a link:

http://www.garfieldtech.com/blog/orm-vs-query-builders

This is also not even remotely close to possible for Drupal 8. You're talking about a wholesale replacement of the entity system. We're already most of the way through doing that in Drupal 8; doing it a second time with a month to feature freeze is not worth discussing.

There's no "9.x" version option, so marking postponed. Given how different Entity API and Doctrine are in their design (from what I understand of Doctrine, which is admittedly only cursory), it's probably a won't fix.

webchick’s picture

I've been doing this for D9 issues.

webchick’s picture

Issue summary:View changes

typo

tim.plunkett’s picture

Version:8.x-dev» 9.x-dev
Sylvain Lecoy’s picture

Experimental work has started here: http://drupal.org/project/doctrine on the 7.x-1.x branch.

I implemented a Driver which from the Schema API + Entity API can translate to Doctrine meta data API. I am able to load a user entity from the doctrine entity manager created through a dependency injection container in Drupal 7.

You might check the code out as I am working on it. Now trying to complete the Schema + Entity API Driver for doctrine for further integration with ToMany relationships and 'foreign keys' column analysis.

axel.rutz’s picture

hot stuff. but surely the "orm as vietnam" paper is a must-read:
http://blogs.tedneward.com/2006/06/26/The+Vietnam+Of+Computer+Science.aspx

pounard’s picture

Very interresting article! Thanks

Sylvain Lecoy’s picture

Yes it is the problem of ORM, even if it was written back in 2006 this article is still valid.

But this issue is about creating and maintaining our own vietnam API versus using an existing vietnam API. :-)

Sylvain Lecoy’s picture

Some updates here: I was able to correctly translates the drupal_schema() to a conceptual data model understandable by Doctrine, thus the DatabaseDriver is now capable of reverse engineer the conceptual model to generates entities.

I am currently blocked by the way search module is conceptually defining relationships and getting this error:
Doctrine\ORM\Mapping\MappingException: It is not possible to map entity 'SearchDataset' with a composite primary key as part of the primary key of another entity 'SearchIndex#sid'.

It is because search_dataset table defines a composite primary key (sid, type), which is a subset of the primary key defined by search_index (word, sid, type). The foreign key defined by search_index (sid, type) is a part of the primary key, and it cause troubles as doctrine does not know how to map these entities together.

There is one solution which is to ignore this table, waiting for an upstream fix of the schema to fit doctrine's expectations, or to use a driver that I was writing - the Schema API Driver - which maps only entities tables and try to reverse engineer only tables from declared entities. This approach has the following drawback, entities must be declared through Entity API to be recognized by Doctrine.

The problem I encountered also is from a physical PoV (e.g. from a database perspective) Doctrine were unable to differentiate a OneToOne (bidirectional) relationship from a ManyToOne (unidirectional) relationship. They both are translated physically by the following:

User
id
address_id (FK)
Address
id

What is expressed is: "This entity has a property that is a reference to an instance of another entity". In other words, using a ManyToOne is the way to map OneToOne foreign key associations: the relation is treated like a toOne association without the Many part.

For the OneToOne (unidirectional) relationship its easier: the foreign key is constrained by a UNIQUE property.

Sylvain Lecoy’s picture

Long story short: Drupal Schema API cannot be translated as a CDM understandable for doctrine to generate entities.

What I propose here is to declare only the entities which will be managed through doctrine, that is for instance, entities declared through Entity API.

Sylvain Lecoy’s picture

Generated entities are now working:

You might check the result here: http://drupalcode.org/project/doctrine.git/tree/HEAD:/modules/entity/lib....

Here is a subset of the User object:

<?php

namespace Drupal\entity;

use

Doctrine\ORM\Mapping as ORM;

/**
 * User
 *
 * @ORM\Table(name="users")
 * @ORM\Entity
 */
class User
{
 
/**
   * @var integer
   *
   * @ORM\Column(name="uid", type="integer", nullable=false)
   * @ORM\Id
   * @ORM\GeneratedValue(strategy="IDENTITY")
   */
 
protected $uid;

 

/**
   * @var string
   *
   * @ORM\Column(name="name", type="string", length=60, nullable=false)
   */
 
protected $name;

 

/**
   * @var string
   *
   * @ORM\Column(name="pass", type="string", length=128, nullable=false)
   */
 
protected $pass;
}
?>

The following code has been generated tahnks to the EntityAPIDriver, which for each entities defined through entity API, walk through the tables defined by the Schema API and generate proper fields and relationships reading the foreign keys meta-data.

Here is the interesting part in the Node entity: the relation to its author.

<?php

namespace Drupal\entity;

use

Doctrine\ORM\Mapping as ORM;

/**
 * Node
 *
 * @ORM\Table(name="node")
 * @ORM\Entity
 */
class Node
{
 
/**
   * @var integer
   *
   * @ORM\Column(name="nid", type="integer", nullable=false)
   * @ORM\Id
   * @ORM\GeneratedValue(strategy="IDENTITY")
   */
 
protected $nid;

 

/**
   * @var \Drupal\entity\User
   *
   * @ORM\ManyToOne(targetEntity="Drupal\entity\User")
   * @ORM\JoinColumns({
   *   @ORM\JoinColumn(name="uid", referencedColumnName="uid")
   * })
   */
 
protected $user;

}

?>

The variable name 'user' is inferred from the Foreign Table Name, singularized if the association is a toOne relationship, and pluralized if the association is a toMany relationship.

Sylvain Lecoy’s picture

Issue summary:View changes

Updated issue summary.

Sylvain Lecoy’s picture

It is not feature complete by the way: unique constraint is missing, and I need to test what happen when you subclass Node with an Article class for instance.

Sylvain Lecoy’s picture

Issue summary:View changes

Added entity classes preview.

Sylvain Lecoy’s picture

Issue summary:View changes

Added further motivations.

Sylvain Lecoy’s picture

Issue summary:View changes

typo

Sylvain Lecoy’s picture

Status:Postponed» Active

Battle plan:

  • Polishing Drupal 7 integration and release something with tests and quality stuff.
  • Port it to Drupal 8 as a separate module.
  • Start investigating Drupal 9 possibilities.
Sylvain Lecoy’s picture

Issue summary:View changes

Added doctrine project.

axel.rutz’s picture

@sylvain: i'm really excited about your efforts!

Sylvain Lecoy’s picture

Created a roadmap for a 1.0 release here: #2243015: Roadmap 1.0.

I am actively looking for a co-maintainer to help implement the remaining issues.

Module usage is growing slowly but people are clearly using it (see page statistics).

Please contact me if you want to contribute with me, I already put guidelines in the issues, I will do intensive code review and help you if you have any questions.

axel.rutz’s picture

In above url i can only find the placeholder.
My workload is already way too high but i might help here or there.
Maybe i'll write a rules integration (like we have for EFQ)... ;-)

Sylvain Lecoy’s picture

Hello axel.rutz,

If you want to take one of these issues (which are actually child of the placeholder: I've updated the issue, thanks :))

Main challenges are revision and multi-lingual support, which can be approached by two different ways: bottom-up or top-down (see following issues).

For now the Entity API driver is 80% functional, and not yet automated tested. The Field API driver task can be a great way to enter in the Doctrine/Drupal field at the same time. General idea is to see fields as entities, with a special care to the cardinality in case its a 1-1 relationship.

Sylvain Lecoy’s picture

Issue summary:View changes