Drupal migration/architecture

By cjchalmers on 18 May 2016 at 17:00 UTC

Hi,

first post and new to Drupal so pls excuse some naive questions

The following concerns the latest version of Drupal 8

I have joined an organisation that uses Drupal for static content and the teams are building seperate apps (twig + symfony) and microservices to deliver dynamic data coming out of Elastic search. Problem with this approach is that the architecture is becoming overly complex for the domain that it's in - basically it's a job site! And time to market for simple changes is too long. Developers avoid Drupal due to lack of knowledge and/or understanding how it works. I keep hearing "It's faster to do myself" or "it shall never perform with Drupal".

I have proposed we standardise on drupal, build up the knowledge (probably with some outside help) and only where necesarry build extra services for specific tasks. We could then leverage various modules such as elastic search etc to improve the finding experience

My concerns though are twofold:

1. We currently have approx 100 DB tables for recruiters, applicants and products. Would this model then all be seperate content types in Drupal and can this be modelled properly? IE with inheritance & relations etc? Are we not going to end up with a huge DB? I was looking into the Entity API to see how this would help me and am looking for some pointers on best practices

2. We currently use various backed (restful) services such as Salesforce and several home grown. Is it the idea that Drual copies the data from the external services into it's own DB? Again similiar concern to nr 1.

A concern from the developers is that by introducing Drupal we now step back into a monolithic solution whereas we were trying to move away from that (monolithic & services both have their pros & cons). How would the new module structure help us to spread the load across different servers?

Can anyone give me insight into this? We are probably looking at a site with approx 3M registered users though per day perhaps only 10k are logging in updating profiles and searching for jobs.

rgds

Colin

Comments

_

bander2 commented 18 May 2016 at 18:03

We currently have approx 100 DB tables for recruiters, applicants and products. Would this model then all be seperate content types in Drupal and can this be modelled properly? IE with inheritance & relations etc?

Drupal is very good at modeling data. The paradigm is a little different than your traditional MVC though. And there is no inheritance (a hairy topic in an MVC anyway) but there are relations using the Entity reference module for D7 (built in for D8).

Are we not going to end up with a huge DB?

Probably, but it sounds like it's pretty big already.

I was looking into the Entity API to see how this would help me and am looking for some pointers on best practices

If your data is not really "content" then using the entity api to model it is a good way to go. If you use content types like you mentioned above you get some overhead (they all expose a page, have an author and publish settings etc.). Entities are a lower level concept. Content types (aka node types) are a type of entity, but you can create others.

We currently use various backed (restful) services such as Salesforce and several home grown. Is it the idea that Drual copies the data from the external services into it's own DB?

You can take a look at the Salesforce Suite module for more information. I am not familiar, but I see stuff like:

Salesforce Mapping (salesforce_mapping): Map Drupal entities to Salesforce fields, including field level mapping.
Salesforce Push (salesforce_push): Push Drupal entity updates into Salesforce.
Salesforce Pull (salesforce_pull): Pull Salesforce object updates into Drupal on cron run.

That tells me that Drupal is syncing data between itself and salesforce. And yes generally Drupal likes to work with data that is in it's own database but there are some modules that make Drupal work with external data like Remote Entity and External Entities. Dependin on the work you are doing, setting up some simple web services might be all that's needed.

Also, BTW I don't think you'll want to use Drupal 8 for this. The elasticsearch and salesforce modules have not been released for 8 and it is likely that many other modules you will need have also not been released for 8.

Whether this is a good idea or not depends on what kind of microservices your organization is creating. If you are going to move everything to Drupal and still plan to do a lot of development on Drupal then it's probably not worth it. A Symfony development project has a way shorter time to market in my experience than Drupal. The power of Drupal is that many things that require development in Symfony are a simple configuration update in Drupal.

For instance, when I see "... deliver dynamic data coming out of Elastic search." I think this is as simple as configuring the elasticsearch module and creating a new View. It can literally be done in minutes.

- Brendan