Problem/Motivation

Support for exportables and Features module integration is a key requirement for facilitating Commerce development and in particular the production of Commerce-based distributions. See the handbook page on exportable configuration for background.

In Drupal contrib, there are two main APIs that facilitate exportables and Features integration:

  • Ctools provides a schema-based API for designating data tables as exportable. The Features module includes support for Ctools exportables. Documentation is available on how to implement exportables with an accompanying admin UI.
  • Entity API provides entity-based exportables and Features integration. Given the extensive use of entities in Commerce, this approach seems to be a good fit. The Profile2 module provides an example of Entity API based exportables integration. Documentation is available on Entity API based exportables and their accompanying admin UI.

Both Ctools and Entity API are current dependencies of Commerce 7.x, so leveraging one or the other's exportables support would not introduce a new dependency.

It is also possible to provide Features integration directly through implementing Features hooks. However, for consistency with other Drupal entity types, it's strongly preferable to begin with providing exportables support rather than jumping straight to Features support. Using an existing exportables API like Entity API or Ctools provides several key advantages:

  • Provides a now-familiar set of UI cues and operations:
    • Marks individual items as "default" (in code and unmodified), "overridden" (in code but overriden locally), and "normal" (created but not yet in code).
    • Provides export, import, and revert operations.
  • Provides Features support with no custom Features specific code.

Currently the Commerce features module provides some interim Features integration based on directly integrating Features hooks.

However, exportables support and Features integration looks to be complicated by unusual architectural patterns in Commerce, as summarized in #9. The usual pattern, as with e.g. Views and Context, is that UI modules serve as pure UI additions to data structures defined and managed by backend API methods. In Commerce, however, UI modules create their own data structures and implement entity hooks, providing entity and bundle information. This architecture is at odds with the basic assumptions of exportables and Features APIs.

Proposed resolution

Features integration is being developed on an interim basis in Commerce features, which provides some Features integration based on integrating Features hooks.

A longer term solution may include building on Entity API's exportables support, supplemented if needed with Ctools support for non-entity exportables (if such exist).

Remaining tasks

  • Address the issue of data structures in UI features. Consider moving data structures from UI to API modules. Issue: #1467986: Remove schema definition from UI modules in Commerce.
  • Choose between Entity API and Ctools as basis for exportables support.
  • Choose between:
    • Develop directly in Commerce (either in an existing or in a new branch).
    • Develop first in commerce_feature contrib module, via extensive alter hook work, and then move to Commerce core.
    • Develop in new contrib module providing a set of alternative UI modules based on Entity API or Ctools exportables UIs, and then move to Commerce core, replacing existing UI modules.
  • Implement work, likely in a new branch of Commerce; see #1467180: Open 2.x Drupal Commerce branch.

User interface changes

Whether based on Entity API or Ctools, UI changes would include new cues as to the current status of individual items (default, overridden, normal) and available operations (export, import, revert).

API changes

An Entity API based approach would extend the existing entity definitions to leverage Entity API. A Ctools based approach would extend existing hook_schema() implementations.

Original report by EclipseGc

Due to the nature of working with Drupal in general, we should probably start putting together a list of things that will make Drupal Commerce more features compatible. Big wins for all of this can probably be found with some strategic use of ctools exportables and wouldn't force people to explicitly use features module to take advantage of the benefits.

I was discussing this with rfay and he asked me to create an issue for this. I've started a little of this work from a conceptual stand point trying to make checkout panes more features friends. This code can be found here: http://drupalcode.org/sandbox/eclipsegc/1253600.git/blob/refs/heads/mast... But the module itself is super short and as an example of just how easy implementing ctools for some of these features could be.

I think the first major step in this direction would be parsing through all of drupal commerce's code and determining what would and what would not make a good exportable. Info hooks are often a good map to either an exportable or plugin type, and starting there would not be a bad idea.

Comments

rszrama’s picture

Is that the easiest way to move forward? What about http://drupal.org/project/commerce_features?

EclipseGc’s picture

I'd have to dig through all the code, but I think 2 points.

1.) We don't want to be forced to use features module. Code committed to d.o would probably benefit from NOT using features if it's not absolutely necessary, and I think that some of the more 'polished' drupal commerce ecosystem modules will probably be interested in not having a features dependency.

2.) Furthermore, I know from talking with Randy he'd like to see this in 1.x, and I know from talking with you Ryan that the changes I would propose here are probably not 1.x material in your mind since I'd want to actually change certain hook from returning arrays to objects and other such similar "api" changes. I'm honestly not interested in that particular debate, 1.x/2.x I'll let you two hash that out, I'm just saying we can get these sorts of exportables if we just do a little schema work and a little hook work in probably quite a few cases with no new dependencies, no new modules, just dotting our i's and crossing our t's in regard to ctools compatibility. Also, in the one instance I cited in my code, we'd actually get to remove quite a few lines of code from drupal commerce. I suspect there are other such places that I've simply not had the chance to find yet.

In any event, whether we go direct features compatibility or ctools exportables, we still need to define what all those areas look like, and if that means we do features for commerce 1.x and ctools for commerce 2.x that's completely cool by me, but I think we definitely want to focus in on making the various components of commerce that SHOULD be exportable (and already are in some cases) EASILY exportable. Yes you can store a checkout pane in code, no it's not readily obvious how to do that if you're not willing to dig into the code a bit. We can solve that, and that's really all I'm proposing here.

rfay’s picture

Commerce Features is great, and I think probably it will live on, but the fact is it has to jump through all kinds of hoops because we don't use ctools exportables. If we changed to using ctools exportables it would have very little to do.

rfay’s picture

Title: Features Integration Checklist » Features Integration Checklist (ctools exportables for every entity)
rszrama’s picture

Ok, but it's not for entities, right? Just info structures?

EclipseGc’s picture

Title: Features Integration Checklist (ctools exportables for every entity) » Features Integration Checklist

Actually digging through the code today:

hook_commerce_currency_info()

I know Ryan and I have discussed this one before. I assume a need for new currencies exists as the world changes and this seems pretty straight forward in that regard. The only thing that I'm giving weird glances is the numeric_code since I'm unsure what the canonical source for such a thing would be. Moving to export on this would give us the ability to easily provide a UI for generating new currencies if necessary. Again totally unsure about the numeric_code though so that's something to discuss at least.

hook_commerce_order_state_info()
hook_commerce_order_status_info()

Both look uncomplicated and straight forward.

hook_commerce_line_item_summary_link_info()

Not 100% sure what I'm looking at here, and with its need for access controls it may require something a bit more extensive than just simple exportables integration.

hook_commerce_order_state_info()

Again this looks pretty straight forward from the core implementations I'm seeing.

hook_commerce_order_status_info()

Commerce Checkout's version of this looks significantly less straight forward than Commerce Order's. Commerce order leads me to believe this would be relatively simple to do, commerce checkout leads me to believe that work done to hook_commerce_checkout_page_info() would translate over to this info hook which might mean this isn't a good candidate for exportability. Still not clear from reading the code.

hook_commerce_customer_profile_type_info()

Unsure about the ramifications on this one. It doesn't look problematic, from the use case that's in core though. However it does appear to be another way to generate checkout panes.

hook_commerce_line_item_type_info()

Looks like it's the commerce_line_item's version of bundle handling. I've been wanting to do essentially this with nodes for a while to prove it will work, I think this is a potentially awesome win from an exportables perspective if we can get it working. I don't actually know of any reason it shouldn't work, and it would probably translate over to product types rather easily in a 2.x implementation and give us a good foundation for doing exportable product types.

hook_commerce_checkout_pane_info()

I've basically done this one already in the code I mentioned in the initial post, however after further review, I think that checkout_panes are probably a much better candidate for a ctools plugin architecture based on what all I'm seeing that they do and how they're implemented. I'd love to actually dive into this one a bit more. Plugin doesn't mean that there isn't some sort of exportable plugin wrapper around some element of it. A single plugin can actually represent MANY implementations simultaneously through some pretty awesome voodoo built into the plugin system, which makes writing a single plugin that handles 'exportables' of a like type of content_pane rather straight forward. I think this could be a superb use case for things like the commerce_fieldgroup_panes module.

hook_commerce_checkout_page_info()

Appears to be fueling the checkout wizard. If we're going to make panes a plugin and/or exportable, this seems another big win from an exportables/plugins perspective. I'd need to dig in further to figure out exactly which it should be, but if a page is just a holder for panes (i.e. no custom elements that aren't defined as panes) then it can probably just be exportable.

hook_commerce_payment_totals_row_info()

Due to the amount of logic I'm seeing in the implementations of this hook, it's probably not exportable. Should either stay as it is, or become a plugin. There is only one implementation to look at though so I can't say I'm getting a real deep look at what the hook can do beyond this single case.

hook_commerce_payment_method_info()

Guessing based on context that payment_methods are probably best represented by a plugin.

hook_commerce_payment_transaction_status_info()

Take it or leave it, this seems very DX to me.

hook_commerce_price_component_type_info()

Guessing that if this maps to anything in ctools it's also a plugin.

hook_commerce_product_type_info()

Exactly what I was talking about earlier. I think this is a prime target for exportables.

hook_commerce_tax_type_info()
hook_commerce_tax_rate_info()

Seem fairly straight forward at first glance, probably good candidates.

pcambra’s picture

Commerce features covers product types and tax types and rates with a features perspective, not ctools, I'm not really sure if ctools is enough for entities but it seems by the comments of EclipseGc that it would be enough.

Entity API implements a class EntityAPIControllerExportable that extends EntityAPIController for providing exportables out of the box http://drupalcode.org/project/entity.git/blob/refs/heads/7.x-1.x:/includ... (from line 509) and rules is based on that http://drupalcode.org/project/rules.git/blob/refs/heads/7.x-2.x:/include.... Docs: http://drupal.org/node/1021526

Maybe it would make sense to rely on that instead of implementing ours.

We may want to play around with all these stuff in a 2.x branch of commerce_features prior to include it in commerce itself.

EclipseGc’s picture

We could certainly play with it in a contrib module, but we'd end up doing a LOT of schema alter work to make that happen, and I think that planning with exportables in mind in a 2.x branch might push us to reorganize the schema just a little (schema is often in the _ui module and ctools exportables sort of expects ui modules to actually be ui, not ui + schema... we could debate whether that's a good or bad thing, I'm simply stating what "is").

rszrama’s picture

Status: Active » Postponed

@EclipseGc - I don't think we'll have a problem w/ respect to schemas in UI modules. The data structures themselves are all defined and managed in the API modules; the UI modules simply have tables that allow for form based creation of new bundles / data bits (for things like tax rates). Those tables come with those modules, because technically we'd see the definition of these things in the database through UI modules as secondary to the primary definition of things directly in hooks. The UI modules simply implement those hooks and load the necessary data from the database to define stuff.

Since we don't have a 2.x branch yet, I'm marking this postponed. As soon as it's up, we'll set 'er active again.

EclipseGc’s picture

Ryan, ok that sounds good. I just wanted this issue to exist as a place we can discuss and plan for 2.x in this regard.

rszrama’s picture

Sweet, and do you think we should make this one particularly about Ctools Exportables? It would make sense to me to have other issues to evaluate other areas of Ctools integration. And is it technically called Ctools Exportables? Or are we rather just making these plugin types that would then be exportable by virtue of being plugins?

EclipseGc’s picture

Yes 'ctools exportables' should be descriptive enough. Your question is a little difficult to answer. I can see ctools integration being a wide enough topic to cover a few issues depending on what all we decide to use, however the boundary between plugins and exports gets foggy in a situation like this one.

Something is "exportable" by virtue of the fact that we added a few lines to the appropriate schema, and thought through the schema so that it matches what we need (which should probably be the next task in relation to these hooks). However, sometimes we end up with things like content_panes which in and of themselves are probably better as plugins... this can become confusing since a single plugin can provide multiple options by wrapping exportable items. That being said, from an 'identification' stand point, I think this can all reside in a single issue, but breaking it up into tasks we might want to treat it very differently.

Continuing on this topic, exportables can be given a UI from ctools by implementing an export_ui plugin. This is very useful as it builds all the import/export/revert/clone/etc features for us, however this is technically unnecessary and we CAN roll our own. I wouldn't suggest it though. Export_ui plugins are pretty easy to write for most things.

pcambra’s picture

So roadmap for 2.x includes replacing commerce features by ctools exportables?

Is the plan to rely on entityAPI exporables support or are we going to implement our own?

EclipseGc’s picture

Entity API is something I've not worked with extensively. Does the 'export' export individual entities (say product x and y). If so, I personally have no desire to try to replace such a tool. If (on the other hand) it's exporting bundle information or information about the entity definition itself, then I'd be much more interested in putting something ctools related there, though I'm certainly not sold on that either direction.

rszrama’s picture

@pcambra, there is no set roadmap yet, so don't throw in the towel just yet. ; )

Right now it's very much a "here are some possibilities, let's talk them through" kinda thing.

EclipseGc’s picture

@pcambra

Yes what Ryan said, I am certainly in no place to mandate, I'm just trying to illustrate the possibilities and their implications.

pcambra’s picture

I'm still going to provide basic support for entity types in commerce features module, don't worry :)) (I've got everything baked/half baked except orders)

But if we decide to move forward to ctools, I'll rather put "evolutive" effort here instead of in features :)

emarchak’s picture

Has any progress been made on this? It looks like this thread has died out.

nedjo’s picture

Title: Features Integration Checklist » Exportables and Features integration for Drupal Commerce

From a quick assessment of the state of this issue, it seems:

  • Emphasis should be on exportables support, rather than direct Features integration. Exportables support offers many advantages that we don't get by jumping straight to Features support.
  • Rather than custom implementations, the focus should be on deciding between the two leading exportable APIs: Entity API and Ctools.
  • Of these, Entity API seems to be the closest match, given the strong emphasis on entities in Commerce.
  • The current Commerce UI modules approach as summarized in #9 presents a barrier to effective implementation of exportables support, through either Entity API or Ctools, because it is significantly at odds with standard Drupal architecture. Refactoring for exportables support should include bringing all hook_schema() implementations into API modules, rather than UI ones.
  • Introducing exportables support should probably include conversion to the exportables UI as available through either Entity API or Ctools, in place of the custom UIs provided in Commerce.
  • While some steps could be taken in commerce_features, proper exportables support will require significant refactoring of Commerce and so will need to be done directly in Commerce, probably in a new branch.
  • Exportables support is not necessary or appropriate for all Commerce data types. Support should focus on entities and bundles that (a) are appropriately created and edited through the UI and (b) qualify as configuration rather than e.g. content.

I've updated the summary for the issue. Please review and make any corrections.

nedjo’s picture

Issue summary: View changes

Add issue summary.

nedjo’s picture

Issue summary: View changes

Add note on export, import, and revert operations.

nedjo’s picture

Issue summary: View changes

Link to Entity API admin UI documentation.

nedjo’s picture

nedjo’s picture

nedjo’s picture

Issue summary: View changes

Add link to #1467180.

nedjo’s picture

Issue summary: View changes

Add link to #1467986.

nedjo’s picture

Issue summary: View changes

Note existing dependencies on Entity API and Ctools.

nedjo’s picture

Issue summary: View changes

Add link to ctools exportables documentation.

Anonymous’s picture

Since nedjo posted this issue on the #1467180: Open 2.x Drupal Commerce branch, and DC2 is waiting for D8 alpha, i would propose using the yaml configuration files for configuration. see: https://groups.drupal.org/node/155559

j0rd’s picture

Looking for this myself to package up my work from dev and push to production and vice versa.

Since we're not going to get exportable support for DC 1.x it seems, I'm curious what other people are using to push things like

  • Currency Configuration
  • Payment Gateway Configuration
  • Checkout page configuration
  • Product Configuration
  • Line Item Configuration
  • Order Configuration
  • Commerce CardOnFile configuration

Currently I'm having to do this in custom hook_updates() or by passing a text file to my colleagues explaining what changes I've made and how to replicate them. Last two days, I've had to do this on 4 machines to bring them up to speed and they'll quickly get out of sync.

bojanz’s picture

@j0rd
commerce_features, mostly.
payment getaway configuration is super tricky because it's embedded in the rule, though.

pcambra’s picture

Commerce cart expiration has an example on how to alter rules in code, i.e. to change settings http://drupalcode.org/project/commerce_cart_expiration.git/blob/refs/hea...

j0rd’s picture

thanks for the tips boys. I'll take a look.

j0rd’s picture

Issue summary: View changes

Link to new handbook page on exportable configuration, http://drupal.org/node/1477970.

kenorb’s picture

bojanz’s picture

Status: Postponed » Closed (outdated)

We've done what we could in 1.x.