note: this thread is meant for discussion. separate issues have been or will be filed as applicable

These are my notes from the Drupalcon Chicago Salesforce BoF.
Jackson River and Node One are both using forked versions of the module, and expressed interest to unfork their work and rejoin the contrib path (please correct me if I have misstated someone's position).

This thread is not meant to serve as a

The BoF identified several key pain points inhibiting wider adoption of Salesforce Suite

  • smart queueing & batching
    ie. Queueing outbound synch requests into batches, and optimizing each batch to eliminate redundant transactions. This was identified as the single biggest limitation with the existing system. especially for large organizations, synching in realtime is prohibitively expensive in terms of time and resource consumption. queueing and batching synchronization operations are critical to optimizing resource consumption. Beyond resource management, smart queueing buys us failure-handling in terms of API limitations and network outages
  • bi-directional
    ie. "Pull" from salesforce to Drupal in batches, instead of outbound message / notifications-based. Again, with large datasets, resource consumption is a critical issue; and sending / receiving messages in real-time is not as efficient as pulling data in batches.
  • complex cross object mapping
    various ideas were discussed about many to many salesforce object relationships, as discussed in #1056630: Bug: sf_user allows creating many-to-many relationships between SF and Drupal objects, even as it assumes they're one-to-one
  • rewrite and refactor taking more advantage of existing tools. the 7.x branch gives us a good opportunity to implement some sweeping improvements.
    existing tools discussed included:
    views - as a fieldmap builder to provide a more familiar interface
    ctools - for CRUD, OOP, other awesomeness
    feeds - to replace import handling

Comments

kostajh’s picture

Thanks for taking these notes Aaron. If folks could publish their forked work in drupal.org sandboxes and post links here, that would be helpful to the rest of us in figuring out how to move forward with merging (or not) the code.

aaronbauman’s picture

Title: [meta] Drupalcon BoF Notes » [meta] Drupalcon 7.x Roadmap

I think 7.x offers an opportunity for rethinking our approach, refactoring code, and tuning the solution.
A straight port from 6.x is a good place to start, but I don't think it addresses long term concerns.

One of my takeaways from the leaner, faster Drupalcon session was that the one-module-per-feature approach is not optimal. We should consider a more-includes, fewer-modules approach, a la views and ctools. Specifically, sf_* modules should be consolidated into a core sf package, and the files included as needed. Permissions and administration can be used to show/hide or enable/disable these features as applicable.

Renaming / repurposing thread to open for more general discussion.

aaronbauman’s picture

Title: [meta] Drupalcon 7.x Roadmap » [meta] 7.x Roadmap
robin monks’s picture

My work on a 7.x direct port is living here for the time being, http://drupal.org/sandbox/robinmonks/1096182 . I'm of the opinion that it would be ideal to make 7.x-1.x a direct port to get something working quickly -- and then perform the refactoring in 7.x-2.x.

This lets us maintain a more stable 1.x for general use while we refactor -- and also ensure a clean upgrade path from 1.x to 2.x when we're ready to start pushing those changes into the wild.

/Robin

robin monks’s picture

Just a note that I'm still working on this in sandbox, http://drupal.org/sandbox/robinmonks/1096182 -- currently focusing on getting object mapping working again and would love help to make that go faster -- with just one person the cycle of port, test, debug, test, research, test, port next feature -- it goes rather slowly.

/Robin

EvanDonovan’s picture

@Robin: I think this is a good plan - better to have something stable for people using 7.x now than to rebuild the system from scratch, essentially, which is what it seems would be necessary for the more ambitious proposals to work.

EvanDonovan’s picture

Since Robin's port was several months old, I re-ran the modules through Coder Upgrade tonight and created a new 7.x-2.x dev branch (no project node as yet, since I would like to get it working first). I think this will make collaboration easier than working in a sandbox. Also, it will make it easier to track the progress on commits, and the forward/backward porting of issues (once we get to that point).

My goals for this port are simply to have feature parity with the current 6.x-2.x, as per Robin's earlier comment (and to track new commits as they come), but to do this using entities rather than the duplication of logic which we have in 6.x in sf_node/sf_user. This will involve me merging sf_entity back into the 7.x-2.x branch shortly, and then combining anything specific to sf_node and sf_user into it, as well as changing the schema of the module accordingly (as per the current 7.x-1.x).

My two main questions at this point are:

1) What should we do about the current 7.x-1.x issues in the queue? I think that branch is essentially a "dead end" at this point (I certainly won't be working on it further), yet people are trying to use it now. It will be at least several weeks before there is a stable 7.x-2.x that we can recommend.
2) What about Jackson River and the other groups who expressed interest in a fuller refactoring of the module for 7.x than what I plan to be doing? Should there be a 7.x-3.x as well? If so, who would take point on that, and would it be based off the 7.x-2.x once that is stable, or would it be basically from scratch?

EvanDonovan’s picture

Category: feature » task
Priority: Normal » Major

Just started taking a look at the queue & realized that there are a lot of open 7.x-1.x issues. I think this is causing confusion, especially in light of #1197906: Release New 7.x-2.x Branch (as -dev).

Aaron & Kosta, would you mind if I were to warn people that 7.x-1.x was about to be deprecated, and then mark them as "won't fix" once there is a 7.x-2.x dev release?

EvanDonovan’s picture

Version: 7.x-1.x-dev » 7.x-2.x-dev
EvanDonovan’s picture

Now that 7.x-2.x is out (though not ready for production use), I think the main things will be to finish converting all the modules to the new schema, write export/import handlers for all mainstream kinds of fieldable fields (at least the core ones), convert Salesforce Import and Salesforce Queue to use Drupal queue, possibly separate the fieldmapping UI into its own module, possibly do two-way sync rules on the field level.

There are issues for some of those items. I'll add the links later.

We should also contact Jackson River to see if they will a) start testing 7.x-2.x, and b) contribute any of their expertise. It would be great if we could have a common solution for 7.x moving forward, whether that is the 7.x-2.x branch or a later 7.x-3.x branch that would incorporate larger changes like switching to OAuth authentication and the REST API (rather than SOAP), or even Views 3 integration for the UI (I think some ideas were tossed around at Drupalcon regarding that, but I can't remember the details right now).

EvanDonovan’s picture

Title: [meta] 7.x Roadmap » [meta] 7.x Long-Term Roadmap

I've contacted Jackson River and some of the other people who were present at the Drupalcon BoF. Hopefully we can hear some more about what people would like to see moving forward.

I'm going to create a separate issue for my more short-term roadmap.

EvanDonovan’s picture

kostajh’s picture

Status: Active » Closed (fixed)