I've started a project for sharing entity metadata, which provides entity level full CRUD and access functions as well as info about all contained entity properties. As all this information is also needed to created services for entities I think it would make totally sense to build the default services 7.x core integration on top of that module.

Well I know that having a dependency is never optimal, but in D7 we will have the plugin manager in core so installing a module is not that a problem. Also with relatively popular modules like rules and others depending on it a lot of people will have it already.. :)
But by having this metadata shared in a module has advantages - that way modules providing entities just need annotate metadata once and any module relying on it can make use of it, so we can avoid lots of code duplication!

I've done some code that provides resources for all entities at http://github.com/fago/services - it's working fine.
I'm using this as a base for rules web services integration, which allows drupal sites to react on events of remote drupal sites. Anyway I'd love to see services building upon that :)

Comments

fago’s picture

Anything new about that? Let's avoid duplicating all that code for implementing entity CRUD & access if possible.

gdd’s picture

Unfortunately I have not gone very deep in 7.x at this point. What will probably happen is that we will simply port 6.3 to a 7.3 and then look into getting more 7-specific features into a 7.4. I also want to look into the page rendering stuff as well.

fago’s picture

Personally, I do think the d6 <-> d7 transition is a good opportunity to break APIs, as module authors have to adapt their code anyway. Having to different services APIs (7.3 + 7.4) might pose transition troubles when multiple modules rely on different versions.

Anyway, I'll try to convince you. First off, please read my blog post, which introduces entity metadata in general.

As I've written there I think Services would be a natural fit to use entity metadata. I think it could benefit much from it, here some advantages coming to my mind:

  • Benefit from existing module integration. Entity metadata can already do full CRUD for all core entities + it determines access for that. Modules introducing new entities are likely to integrate with entity metadata fast, as it's need to make use of it with Rules.
    But apart from this entity level integration, there is property level integration. It incorporates property level view + edit access restrictions. For fields, field access is incorporated, but there is also access info for regular properties. E.g., changing the node published status is up to users with 'administer nodes' permission. Shouldn't Services with regular session auth respect that too?
  • Having metadata available for services surely pays off. E.g. you could use it to automatically generate Service API documentation. Servers could use it too, e.g. the SOAP server could generate proper WSDL.
  • Exposed data structures would be more simple and correspond to the metadata. E.g. a simple text field value won't sit in $node->field_name[$langcode][0]['value'], instead $data['field_name'] does it. Oh and it would also take over text processing, if activated. Thus the real internal data structure gets hidden and is streamlined to make more sense. Also that way a module changing its internals, need not be reflected in a Service change!

To actually do the integration, I see two ways to go. 1., depend on Entity Metadata. 2., don't depend on it and ship services integration with Entity Metadata, but don't provide another one. That way modules could choose whether they need core-resources or not. Probably most modules would need it anyway, so probably 1) is better.

Disadvantages:

  • A new dependency, the Entity API. However, lots of sites will have it anyways as Rules + modules relying on it (ubercart, drupal commerce,..) will need it anyway, as well others that already make use of it (media, profile2, searchapi, ..). Also we have the plugin manager in core - installing a module shouldn't be so much work any more ;)
  • As described in the third point above, the exposed data structure would change. However when migrating from D6 to D7 it would change anyway.

Unrelated to that, when introducing the dependency you could also make use of the entity crud API to simply turn the services endpoints in exportable entities + make them fieldable. I do it that way for rules too.

voxpelli’s picture

Subscribing - but no opinion on this yet

fago’s picture

Some thoughts about possible ways to integrate with Entity Metadata:

1.) Use Entity Metadata to provide Services integration to avoid duplicated code. This is what I've done in my github repo.
2.) Allow modules to optionally attach metadata to their resources + make it available to servers/via the API. This should be rather easy and not involve much changes - maybe it's even possible without any API changes at all (only additions). This is needed for modules to make really use of the metadata, in particular to achieve the improvements mentioned in the second point above.
3.) Furthermore we could improve the way Services describes data structures to be compatible with the way Entity metadata describes them - which is a bit different (but not much). This would ease integration between both modules and would make it totally compatible with Rules (as it also uses that way) + it would be consistent with the wsclient module. Establishing a unique way for describing data in Drupal makes much sense to me and would help to improve the DX.

kylebrowning’s picture

Status: Postponed » Active

Certainly happen to commit any patches to the 7.x branch that are reasonable.

Edit: fixes typo

kylebrowning’s picture

Marking this as postponed until we get to a 4.x branch.

I think its way to late to start changing APIS this vastly. Ill gladly start a branch once the git migration is done.

kylebrowning’s picture

Status: Active » Postponed
voxpelli’s picture

Once this issue gets reopened D7 will have been stable for a while and I then think we should have a discussion on how this issue is best solved and look at how the community at large has begun to solve this problem. I know there has been some discussions related to the Entity API and the problem it tries to solve and there has probably been even more by then.

I also think that if anyone has the time it would be good if work on this issue began as a contrib module of its own just like the REST Server and the Services OAuth module was prior to being incorporated in Services 3.x. That would enable a lot more flexible development of it.

klausi’s picture

In the meantime fago and I created the restws module to demonstrate what we have in mind. It especially targets Drupal entities and how they can be exposed as resources according to the REST principles.

voxpelli’s picture

@klausi: I added #1042512: Why not use Services Module? - I can't see why you chose not to use Services at all? Can you explain it for me in that issue?

fago’s picture

Status: Active » Postponed

I also think that if anyone has the time it would be good if work on this issue began as a contrib module of its own just like the REST Server and the Services OAuth module was prior to being incorporated in Services 3.x. That would enable a lot more flexible development of it.

Duplicating all resources based upon the entity API looks weird to me and would probable confuse most services users. Anyway, as outlined in #3 providing resources based upon the entity API is just the first step. But the further steps can only be achieved with API changes, so a services-addon module doesn't help here. E.g. it's not possible to utilize property information of a resource in a services rest formatter, as the API doesn't provide the formatter with that.

So "restws" tries to realize the ideas outlined in #3, thus it builds strongly upon the Entity API while it concentrates on providing RESTful services. If the approach proves to be successful, maybe service 4.x can pick it up for its RESTful stuff. Maybe it just smooths the way for some of its design decisions, or maybe not. We'll see.

You can find more thoughts about restws in my blog post.

voxpelli’s picture

@fago: I fail to find any request for API changes in this issue. I thought this issue was entirely about creating new resources based on the Entity API. Could you post new issues for any API change you need in Services? Such things are easily lost in big issues like this one.

Edit: Btw - duplicating the Services module looks weird to me and probably confuses most services users.

jherencia’s picture

Sub.

marcus_clements’s picture

I suggest that if this happens that the resource definition be extended to include return object type information gleaned from entity metadata - see a related issue for Services 6.x-3.x

fago’s picture

ad #13:
As described in #12, making use of the entity API for resources is just the very first step, for which you can find the code on github (as you know). But as the decision of Services is to not rely upon the entity API for now, rolling patches for any architectural changes that would integrate much more deeply with the entity API is obviously pointless.

ad #15: Exactly.

marcingy’s picture

Status: Postponed » Closed (won't fix)

Closing as this issue is never going to move rightly or wrongly