This issue is about considering to use or not Doctrine ORM(Object-Relational Mappers) for entity mapping to database schema.
Problem/Motivation
We have built over the past home made Drupal ORM for Entity management. Right now, we are thinking about replacing hook_entity_info() arrays with annotations, requiring entities to be plugins. We should use the plugin system sparingly as it will set the tone for community.
Drupal as of version 6, implements an ORM. Hooks can add data to nodes from anywhere they want, whether it maps to a database table or not. However, the ORM is "naked": there is no abstraction on data access. Everything is loaded at once into a single big blob data that gets returned from node_load().
Drupal as of version 7 improved a lot this. The Entity API matches the entity structure to the relational database and adds an abstracted layer for CRUD operations, but, the loading of an entity is still not optimized.
Drupal in its 8-dev version changed the entity_info() arrays into annotations, so this is a good step and a good direction. Entities are currently re-factored against the plugin system, using a EntityManager to govern persistence. It is exactly how Doctrine ORM actually work.
Because an ORM is more robust, it can do a lot more optimization under the hood. Complex relationships can be modeled in a database-independent way, even tying to non-SQL systems like the file system, Services, etc.
- The Entity system is really complex, you have to first extend a base class or implement the EntityInterface.
- Entities themselves contains reference to controllers, via annotations, this should not appear in a data object.
- Annotations are not used on a per-field basis, but as a meta-data stack which helps the entity manager to understand how to deal with the Entity.
Refactored solution
- Does not need to extend a base class or to implement an interface.
- Generates Physical Data Model from classes themselves.
- No need for Schema API anymore in the long run.
- Uses annotations the correct way.
- Have a meta-data caching system included.
Entities are generated and looks like the following:
<?php
namespace Drupal\entity;
use Doctrine\ORM\Mapping as ORM;
/**
* Node
*
* @ORM\Table(name="node")
* @ORM\Entity
*/
class Node
{
/**
* @var integer
*
* @ORM\Column(name="nid", type="integer", nullable=false)
* @ORM\Id
* @ORM\GeneratedValue(strategy="IDENTITY")
*/
protected $nid;
/**
* @var \Drupal\entity\User
*
* @ORM\ManyToOne(targetEntity="Drupal\entity\User")
* @ORM\JoinColumns({
* @ORM\JoinColumn(name="uid", referencedColumnName="uid")
* })
*/
protected $user;
}
?>
<?php
namespace Drupal\entity;
use Doctrine\ORM\Mapping as ORM;
/**
* User
*
* @ORM\Table(name="users")
* @ORM\Entity
*/
class User
{
/**
* @var integer
*
* @ORM\Column(name="uid", type="integer", nullable=false)
* @ORM\Id
* @ORM\GeneratedValue(strategy="IDENTITY")
*/
protected $uid;
/**
* @var string
*
* @ORM\Column(name="name", type="string", length=60, nullable=false)
*/
protected $name;
/**
* @var string
*
* @ORM\Column(name="pass", type="string", length=128, nullable=false)
*/
protected $pass;
}
?>
Remaining tasks
Decide if we want to go in this direction or not.
Comments
Comment #0.0
Sylvain Lecoy CreditAttribution: Sylvain Lecoy commentedDetailed 'arrays'.
Comment #1
pounard+1
Comment #1.0
pounarddetailed 'do this'.
Comment #2
Crell CreditAttribution: Crell commentedI stumbled across this via Google in response to a twitter discussion. Bizarre... :-)
You forgot a link:
http://www.garfieldtech.com/blog/orm-vs-query-builders
This is also not even remotely close to possible for Drupal 8. You're talking about a wholesale replacement of the entity system. We're already most of the way through doing that in Drupal 8; doing it a second time with a month to feature freeze is not worth discussing.
There's no "9.x" version option, so marking postponed. Given how different Entity API and Doctrine are in their design (from what I understand of Doctrine, which is admittedly only cursory), it's probably a won't fix.
Comment #3
webchickI've been doing this for D9 issues.
Comment #3.0
webchicktypo
Comment #4
tim.plunkettComment #5
Sylvain Lecoy CreditAttribution: Sylvain Lecoy commentedExperimental work has started here: http://drupal.org/project/doctrine on the 7.x-1.x branch.
I implemented a Driver which from the Schema API + Entity API can translate to Doctrine meta data API. I am able to load a user entity from the doctrine entity manager created through a dependency injection container in Drupal 7.
You might check the code out as I am working on it. Now trying to complete the Schema + Entity API Driver for doctrine for further integration with ToMany relationships and 'foreign keys' column analysis.
Comment #6
geek-merlinhot stuff. but surely the "orm as vietnam" paper is a must-read:
http://blogs.tedneward.com/2006/06/26/The+Vietnam+Of+Computer+Science.aspx
Comment #7
pounardVery interresting article! Thanks
Comment #8
Sylvain Lecoy CreditAttribution: Sylvain Lecoy commentedYes it is the problem of ORM, even if it was written back in 2006 this article is still valid.
But this issue is about creating and maintaining our own vietnam API versus using an existing vietnam API. :-)
Comment #9
Sylvain Lecoy CreditAttribution: Sylvain Lecoy commentedSome updates here: I was able to correctly translates the drupal_schema() to a conceptual data model understandable by Doctrine, thus the DatabaseDriver is now capable of reverse engineer the conceptual model to generates entities.
I am currently blocked by the way search module is conceptually defining relationships and getting this error:
Doctrine\ORM\Mapping\MappingException: It is not possible to map entity 'SearchDataset' with a composite primary key as part of the primary key of another entity 'SearchIndex#sid'.
It is because search_dataset table defines a composite primary key (sid, type), which is a subset of the primary key defined by search_index (word, sid, type). The foreign key defined by search_index (sid, type) is a part of the primary key, and it cause troubles as doctrine does not know how to map these entities together.
There is one solution which is to ignore this table, waiting for an upstream fix of the schema to fit doctrine's expectations, or to use a driver that I was writing - the Schema API Driver - which maps only entities tables and try to reverse engineer only tables from declared entities. This approach has the following drawback, entities must be declared through Entity API to be recognized by Doctrine.
The problem I encountered also is from a physical PoV (e.g. from a database perspective) Doctrine were unable to differentiate a OneToOne (bidirectional) relationship from a ManyToOne (unidirectional) relationship. They both are translated physically by the following:
What is expressed is: "This entity has a property that is a reference to an instance of another entity". In other words, using a ManyToOne is the way to map OneToOne foreign key associations: the relation is treated like a toOne association without the Many part.
For the OneToOne (unidirectional) relationship its easier: the foreign key is constrained by a UNIQUE property.
Comment #10
Sylvain Lecoy CreditAttribution: Sylvain Lecoy commentedLong story short: Drupal Schema API cannot be translated as a CDM understandable for doctrine to generate entities.
What I propose here is to declare only the entities which will be managed through doctrine, that is for instance, entities declared through Entity API.
Comment #11
Sylvain Lecoy CreditAttribution: Sylvain Lecoy commentedGenerated entities are now working:
You might check the result here: http://drupalcode.org/project/doctrine.git/tree/HEAD:/modules/entity/lib....
Here is a subset of the User object:
The following code has been generated tahnks to the EntityAPIDriver, which for each entities defined through entity API, walk through the tables defined by the Schema API and generate proper fields and relationships reading the foreign keys meta-data.
Here is the interesting part in the Node entity: the relation to its author.
The variable name 'user' is inferred from the Foreign Table Name, singularized if the association is a toOne relationship, and pluralized if the association is a toMany relationship.
Comment #11.0
Sylvain Lecoy CreditAttribution: Sylvain Lecoy commentedUpdated issue summary.
Comment #12
Sylvain Lecoy CreditAttribution: Sylvain Lecoy commentedIt is not feature complete by the way: unique constraint is missing, and I need to test what happen when you subclass Node with an Article class for instance.
Comment #12.0
Sylvain Lecoy CreditAttribution: Sylvain Lecoy commentedAdded entity classes preview.
Comment #12.1
Sylvain Lecoy CreditAttribution: Sylvain Lecoy commentedAdded further motivations.
Comment #12.2
Sylvain Lecoy CreditAttribution: Sylvain Lecoy commentedtypo
Comment #13
Sylvain Lecoy CreditAttribution: Sylvain Lecoy commentedBattle plan:
Comment #13.0
Sylvain Lecoy CreditAttribution: Sylvain Lecoy commentedAdded doctrine project.
Comment #14
geek-merlin@sylvain: i'm really excited about your efforts!
Comment #15
Sylvain Lecoy CreditAttribution: Sylvain Lecoy commentedCreated a roadmap for a 1.0 release here: #2243015: Roadmap 1.0.
I am actively looking for a co-maintainer to help implement the remaining issues.
Module usage is growing slowly but people are clearly using it (see page statistics).
Please contact me if you want to contribute with me, I already put guidelines in the issues, I will do intensive code review and help you if you have any questions.
Comment #16
geek-merlinIn above url i can only find the placeholder.
My workload is already way too high but i might help here or there.
Maybe i'll write a rules integration (like we have for EFQ)... ;-)
Comment #17
Sylvain Lecoy CreditAttribution: Sylvain Lecoy commentedHello axel.rutz,
If you want to take one of these issues (which are actually child of the placeholder: I've updated the issue, thanks :))
Main challenges are revision and multi-lingual support, which can be approached by two different ways: bottom-up or top-down (see following issues).
For now the Entity API driver is 80% functional, and not yet automated tested. The Field API driver task can be a great way to enter in the Doctrine/Drupal field at the same time. General idea is to see fields as entities, with a special care to the cardinality in case its a 1-1 relationship.
Comment #18
Sylvain Lecoy CreditAttribution: Sylvain Lecoy commentedComment #19
elvis2 CreditAttribution: elvis2 commentedI currently use symfony and really like doctrine. There are a few known problems though. Doctrine, with lots of schemas, is slow. When adding annotations on top, those annotations need to be read and cached.
I would be concerned about using doctrine on a large build. BUT, I could see a reverse engineering usage, to pull into Drupal to create schema files (yml now?).
Just a thought...
Comment #20
Sylvain Lecoy CreditAttribution: Sylvain Lecoy commentedHave a look at my work in Doctrine I wrote a driver for Drupal Schema API.
http://cgit.drupalcode.org/doctrine/tree/lib/Drupal/doctrine/Mapping/Ent...
and
http://cgit.drupalcode.org/doctrine/tree/lib/Drupal/doctrine/Schema/Sche...
The idea was to not use annotations (while entirely compatible) but schema API as meta-data. Of course both scenarios would need caching to be effective.
Comment #21
catchThis needs an explanation of how things like:
field translation
field UI
views
and etc. would be handled. If they can't be then it's won't fix.
Comment #22
Sylvain Lecoy CreditAttribution: Sylvain Lecoy commentedPartially answered in:
Missing views, but after 2 years I have lost interest in this. Maybe someone will take the lead on this issue one day !
Comment #23
giorgio79 CreditAttribution: giorgio79 commentedPossibly a dupe now given #2846366: Improve Drupal's Database Abstraction Layer Extensibility and Capabilities
Comment #25
xjmThis is a minor-only addition. Since 8.9.x and 9.0.x are now in beta, I'm moving this to 9.1.x. Thanks!
Comment #32
catchhttps://drupal.org/project/doctrine hasn't been updated for seven years.
In the meantime, we've had to fork Doctrine code into core, to support use-cases they no-longer wanted to #2631202: Doctrine no longer supports SimpleAnnotationReader, incorporate a solution into core.
Given this, the prospect of trying to replace the entity system with Doctrine, which we might then have to fork back into core, seems like a non-starter to me. Any work would need to start again with the 2023 version of the entity system compared to 2023 Doctrine too. Marking this 'won't fix'.