Why not use Services Module?

+1 we could really get this into the already existing services module, no need to fork.

This is easily handled by creating your own Resources and just as an FYI those could easily be brought into core services

This is definitively not a fork of services as we go after fundamentally different design goals. Instead of describing each entity type as resource, we want to leverage the full potential of the Entity API that already contains property information, CRUD functions etc. This generic approach is capable of dealing with entity types that do not even exist yet or that come with any third party module. Furthermore we do not believe in the concept of endpoints - resources should always be available on the same URL path pattern. /node/1.json gives me the JSON representation of a node, /user/1.xml gives me the XML representation of a user, /profile/1.pdf gives me the PDF representation of a user profile. This is simple, robust and eases client development.

Maybe this module can be an inspiration for services 4.x, but I don't see that it fits to the design of services 3.x right now. I don't want to piss you brave services developers off, but rather suggest a radically different approach how RESTFul web services can be exposed in Drupal. However, I'm open to discussion and look forward to talk to you guys at the Drupal Developer Days Brussels next week.

I think authentication should be ignored completely in a core REST service module and should be handled by separate modules. HTTP is a stateless protocol, so an authentication module must always read the credentials somehow from the HTTP request and perform a user login. I see currently 3 common ways for REST services:

1) Session cookie: Drupal handles that for us and does the user login. Nothing to do here, yay!
2) HTTP Authorization header: see restws_basic_auth.module (a submodule of this project), 8 lines of code to login the user.
3) OAuth: A module could read the access_token from the URL, map it to a user account and do the login.

The authentication must be done at the beginning of page request, then the RESTful web services module is able to work with users and the permission system. I see no problem even when all 3 authentication methods are active, a resource representation (/node/123.json) can live at the same place in any case. Am I missing something?

I'm pretty sure you had your reasons for developing services as it now is, no offense.

I think it is a good idea to handle authorization completely with the user/permission system. For an OAuth module this could mean to create a new (sub)user that has restricted permissions and that is used to log into when an access_token is passed. That would protect the user's privacy. I can see that this could lead to problems with the "own" permissions (e.g. "edit own content") ... *sigh* yeah, OAuth is not trivial.

No:
1) Neither is the code nor the intended functionality a duplicate of the services module.
2) Athorization via the user/permission system is just one aspect among the differences between restws and services. Using entity metadata from entity.module instead of explicit resource information for entities is another. Convention over configuration (i.e. no explicit endpoints) also. We know that in Drupal the HTML representation of a node lives at /node/123, so we also want to have the JSON representation there (either by sending "Accept: application/json" in the HTTP header or by appending ".json" to the URL). As I said, we have some fundamentally different design decisions here.

Coincidentially Jeff Eaton just made a blog post about Drupal 8 and missing APIs:

The second missing API is a built-in foundation for web services. Drupal's early automatic support for standards like RSS and BlogAPI helped push it ahead of competition that required grafted-on solutions. As next generation standards for data exchange emerge, though, we risk falling behind. In Drupal 8, we can build on the foundation of the Entity API: automatically exposing all Drupal entities in a RESTful fashion would dramatically simplify the work of building desktop and mobile apps, complex cross-site publishing tools, and more.

That is exactly what we are trying here with restws.

Actually klausi, you can use ACCEPT headers for application/xml or application/yaml or json in services 3.x and you can even do things like node/1.php for serialized options.
I dont see how it is fundamentally different.

You can already do everything you have described in services, and we have an extensible system for adding any type of Response format or auth system as well as the ability to write your own resources.

DId you even look at services 3.x?

@kylebrowning: I was not able to create an endpoint in services that serves resources at "/" or "<front>", can you explain how I can do it?

@voxpelli: What do you mean by Google-like APIs? Of course "artificial" resources should be possible at any path, but well known Drupal entities should live at their default path.

The Services module need more contributors - there's already more to be done there than the current contributors has time to do. Why not instead of spending time on duplicating Services help the Services guys out in getting that module as good as possible?

As you know I've started contributing to Services 7.x-3.x a year ago. I tried to move Services in that direction I think it should go, but without any results (see #745046: make use of entity metadata).

ad #10:
>DId you even look at services 3.x?
See above.

See also #745046-12: make use of entity metadata for why we are not building upon services.

As voxpelli said, Services is for more than just entities (although it doesn't disallow them at all) and thus has to support a much wider range of use cases. Multiple endpoints was a widely requested feature for people implementing their own product-specific APIs. This allows them to expose versioned APIs, as well as having some features be authenticated and some not. Also note that by default Services uses Drupal authentications invisibly just like Restws, additional authentication methods are optional. While Services 3 does optionally support alternate server implementations, we are squarely focused on REST for the most part.

Ultimately I don't really care who does what or whatever. We have got an amazing release with 3.x and people will use whatever modules they want and that is as it should be. We will fully support Entities in the next dot-release or two most likely, and what would have been nice is if this effort could have been focused on making that happen, but no big. Entities are obviously a big part of Drupal 7 but I don't believe for a second that we are moving to a world where everything in Drupal core and contrib for all use cases ever is entitized, and if you want to support USERS rather than THEORIES then acting as such is incredibly short sighted. That's just my opinion tho, and if there's anything there is plenty of room for in Drupal it is opinions. Best of luck on the project.

Entities are obviously a big part of Drupal 7 but I don't believe for a second that we are moving to a world where everything in Drupal core and contrib for all use cases ever is entitized, and if you want to support USERS rather than THEORIES then acting as such is incredibly short sighted.

While restws strongly builds upon entities and the APIs provided by the entity API, it doesn't force resources to be entities either.

As voxpelli said, Services is for more than just entities (although it doesn't disallow them at all) and thus has to support a much wider range of use cases. Multiple endpoints was a widely requested feature for people implementing their own product-specific APIs. This allows them to expose versioned APIs, as well as having some features be authenticated and some not.

I'm sure there are situations you need that, however we haven't seen the need for it in our use-cases, so we prefer to keep things as simple as possible. Also having some resources authenticated and others not works fine with a single "endpoint".

That's just my opinion tho, and if there's anything there is plenty of room for in Drupal it is opinions. Best of luck on the project.

Agreed! Wish you (and the services team) all the best too! I'm sure both projects can learn and benefit from each other, so in the end Drupal will win.

"Agreed! Wish you (and the services team) all the best too! I'm sure both projects can learn and benefit from each other, so in the end Drupal will win."
or be fragmented...

Fago and I had a chat yesterday in IRC about the endpoint URL issue; although I can't speak for him, I think we nailed down a useful idea regarding the URLs. Versioning via different endpoint URLs is pretty much standard at this point, although it's also standard to have /api as the url of the current api version. Using native entity URLs by default makes moving TO a versioned API very very difficult, but it would be possible to supply optional menu callbacks on the entities themselves, aliasing those to the normal API callback URLs.

For example, node/%node/json could simply be an optional pointer to api/node/%node/json.

@eaton: You still wouldn't be able to support the old version, would you? With Services you can have multiple versions of an API deployed at the same time - that solved our versioning problem completely.

Not with the code that's present in this module. It was more a discussion about the philosophy of having explicit endpoints versus node/%node/$format style endpoints. I'm only just sinking my teeth into the complexities of the various web services issues ;-)

Subscribe.

@eaton:
I've opened #1046470: Prepare for versioned services for that, so let's discuss that there. I'm still unsure about what's best way to proceed though.

@voxpelli: This is not about criticism services' approach. With multi-server support I agree with you, you'll need multiple endpoints. But this is about finding the best way to support versioned APIs in RestWS.

#23, Again it just seems like you are recreating the wheel since services already does this.

> Entities are obviously a big part of Drupal 7 but I don't believe for a second that we are moving to a world where everything in Drupal core and contrib for all use cases ever is entitized

My feeling on this is that on the one hand, having a standardized system of services for entities is great. Yay standardization and consistency.

But having *just* services for entities can't be the whole picture. Consider: when was the last time you built a Drupal site where every public-facing URL was a node, user, or taxonomy term, and showing nothing more than that entity?

We're always building sites that show multiple things at once, with complex relationships. In the same way, we're always building services that pull out some bizarre combination of data from the site, depending on use cases often unique to the site in question.

So a service system that has only CRUD operations on entities seems to be to be a very nice demonstration, but not that useful practically.

Maybe we haven't been clear on this, bur restws will of course support also "artificial" resources that developers create on demand. I'm currently working on such a resource and will publish an example how it can be done with restws (basically you just have to implement your custom resource controller class and register it with the restws_resource_info hook).

So don't worry ;-)

P.S.: Jep, that would be something for restws.api.php which does not exist yet.

"bur restws will of course support also "artificial" resources that developers create on demand"

Which makes this a COMPLETE fork of the existing services solution.

fago-- and klausi--

Yeah this has gone from very interesting alternate idea to essentially a fork with the added feature set. It's quite obvious that the maintainers just want to do their own thing for the sake of doing their own thing and there is nothing we can do besides continue the broader community support behind the Services effort. Good luck with the project.

>Yeah this has gone from very interesting alternate idea to essentially a fork with the added feature set.

a) The feature set has not been changed at all since we've published the project.
b) We've already explained the different design decisions as well as the difference of the functionality of the modules, all of that is still valid.
c) As there is no common source of the code base, there is no fork.
d) Sad, that the discussion has moved to "that level" - I see no purpose to continue with that.

A. What does that have to do with ANYTHING?
B. Not once have you explained any design decisions that differ from what Services can already achieve. They are NOT valid in the sense that you continue to ignore you can do this with Services and we would love your help to do this in 4.x #1044162: Move to Services, initial port and #745046: make use of entity metadata
C. Its not a fork in the sense you are using the same code, its a fork in the sense that you are attempting to provide everything services already provides. You chose to NOT combine efforts just because we said this feature cannot be in 3.x. Im sorry you got butt hurt about that but it can be in 4.x. We proved you can do entities with Services here #1044162: Move to Services, initial port.
D. Uhmm ok.

Whatever Its cool, ill just make sure to create a project and import Voxpellis code. That will probably get included into core Services in 4.x

Have fun reinventing the wheel.

Sorry, didnt mean to switch this.

For the record, I'll list some differences to services (again). Maybe we can use this for a documentation page later.

* You cannot expose the node resource at /node/1.json with Services (at least I could not do it and nobody showed otherwise).
* Resources are not automatically exposed in Services (you have to create endpoints). In RestWS you just enable the module and setup user permissions.
* RestWS is authentication agnostic while Services provides authentication integration (read: if you need OAuth go to Services, at least now)
* Services is resource API version agnostic (for a new API version a new endpoint is created), while RestWS plans to implement versioning differently (see #1046470: Prepare for versioned services)
* RestWS is strictly REST principles only and does not contain any code for message-oriented service types (read: if you need SOAP, XML-RPC etc. go to Services). This makes the code base smaller and easier to maintain.
* Any custom entity is automatically available via a uniform URL interface (/entity_type/entity_id) in RestWS, no configuration needed. Incoming data is handled by the Entity API wrappers. Insert all the Entity API advantages here.

I can see that you are interested in integrating the Entity API for Services 4.x and we will of course help. However, I don't think that even this step will render RestWS useless nor obsolete.

* RestWS has data property information baked into the API, such that formatters can/have to make use of it. This allows formatters for e.g. to render resource references differently, as required for the format. Also formatters can format a resource by making use of any additional information available about a resource, as the RDFa example shows by applying the RDF mapping the core RDF module provides for entities.

Fago, #33 is achieved by changing your Content-Type header to application/json or application/yaml or xml. Alternatively you can make your own formatters, alter existing formatters and extend others. It even has support for calling an endpoint as node/1.json or node/1.xml if you dont want to send headers.

How is #33 or for that matter, anything else listed different from Services?

>How is #33 or for that matter, anything else listed different from Services?

The architecture is different than in services. Services has resources, models and formatters - but restws just has resources and formatters, whereas the formatter has full control / knowledge of the context. That enables the formatter to lookup the RDF mapping for the resource in question and use it to generate RDF that takes the mapping into account.
Then as said multiple times, the availability of property information to formatters enables them to take the info into account, e.g. for formatting resource references as ideal for a certain format.

ad #34:
Well, as your patch has started to show we could handle the requests on our own and then pass the request on to services, where then our resource implementations are invoked. But that way Services isn't a great help any more for what we are doing, as we'd have to do most parts ourselves anyway. Next, being bound to the services API would form an obstacle for us when formatting the resource in the way we want it (see above). So, as Services doesn't help us, we don't use it (for now).

"Why not using Services?" Clearly the central theme in the responses is that using services would add extra complexity. This is not to say that services is overly complex, but that it is possible to build a simpler module by accepting certain assumptions about how the service will work. This is not illogical, hardly a duplication of work, and has precedent in the community (see emf vs. campaignmonitor for one of several examples).

The only concern I see is a fragmentation of the user base and support modules (see the variety of video related modules in drupal 6). Hopefully this will not be an issue!

Thanks for the great work Klausi and Fago, and also to everyone who has worked on services.

"Why not using Services?" Clearly the central theme in the responses is that using services would add extra complexity. This is not to say that services is overly complex, but that it is possible to build a simpler module by accepting certain assumptions about how the service will work. This is not illogical, hardly a duplication of work, and has precedent in the community (see emf vs. campaignmonitor for one of several examples).

The only concern I see is a fragmentation of the user base and support modules (see the variety of video related modules in drupal 6). Hopefully this will not be an issue!

Thanks for the great work Klausi and Fago, and also to everyone who has worked on services.

So, this project sounds like Views_datasource

Yes and no. It is similar to views_datasource as it also exposes content in various formats (as far as I can see). However, RESTWS goes beyond that: you can also create, update and delete content via an interface. And it is not only limited to content (nodes), as all kind of entities known to the system can be exposed. Also artificial resources (as shown in restws.api.php) can be CRUD enabled.

The comment from Jeff Eaton scares me..

@klausi nice simple solution for REST-only, keep up the good work!

Additionally Services module uses form submission functions for inserting new data, makes it impossible for images/files etc

Its not impossible at all go look at the .test files. Uploading files to a node is easy.

I think you're wrong kyle, services node_resources only let you add a file and a node as seperate things, you still need to connect them somehow right? How do i submit the file/image for a content type node along with all the other fields in the same submission?

Please don't let the negative feedback from the Services developers stop this useful project. I spent the better part of a day trying to get Services to run. It was large, unreliable, complex and has virtually no useful (that is, not out of date) documentation. I eventually got it running for a sample "endpoint" using the supplied specialty "noteresource" module outlined in http://drupal.org/node/1246470 with the corrections noted in the Comments and some other fixes. But I realized that to make it work for my project, I would have to write a bunch of custom code. And all I want it to do is expose some node data as a REST service.

I used this module and Entity and had it up and running in 15 minutes. Simple, reliable (so far anyway ;-) and completely understandable. I've already done the demo version for my boss to review. So thanks for this and for all your work.

Thanks for the hard work on this great alternative!

I'm all for working on alternative solutions! You should update the project page though, because Services module in the 3.x branch does do Restful. Full CRUD credentials are covered.

> I'm all for working on alternative solutions!

I fully agree :), especially as I think there is one big difference in terms of entity create/ update -- Entity metadata Vs form API

From my own experience, I prefer restws for entity CRUD as it passes my data to entity_metadata_wrapper() and not to a form_submit() e.g. populating a date-field using wrapper is much easier, and IMO more correct via wrapper then FAPI.

I've been playing with the services module for a few days to see if I can use it for a project. I ran into an issue. The Services module does not return custom fields when you request a resource. It might work for nodes, but support for comments is not there and that's what I need. When requesting a comment resource, I want the full resource including any custom fields attached and the data they contain.

So I discovered the restws module today. It works as expected right out of the box.

Thanks!

people interested in this discussing might want to join forces on creating a common basis for services in drupal 8.

see the following posts by klausi on g.d.o:

• RESTWS for Drupal 8
• A RESTful entity web API for Drupal 8