It looks like Default Content as said on the project page is being deprecated in favor of UUID features.

Since we are in RC status should we be switching from using this module before release. Module has been mostly work except for the annoying duplicate files issue. ^_^

Wondering about any opinions on this.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

nedjo’s picture

We should strongly consider switching to Migrate instead of UUID Features. Commerce Kickstart provides good examples. Advantages of Migrate include:

  • Facilitates migration/data import as well as default content creation.
  • Has extensive support for many data types.
sylus’s picture

Agreed with nedjo I am using Migrate + Querypath with great success for migrating default content.

populist’s picture

Title: Default Content is being deprecated » Implement Migrate Module for Panopoly Demo Content

I think we should get a good solution here and think migrate is good. In terms of the profile, we might end up supporting both Default Content AND Migrate for awhile since I dont want to remove any modules prior to release. However, since nothing in Panopoly Core has Default Content requirements we are good.

saltednut’s picture

Unsure why you guys want to involve heavy migrate scripts. Isn't the use case for migrate to take data/info from another source and inject it into Drupal?

I've done some research with regard to this matter over the past few weeks and I've found migrate/deploy to both somewhat heavy-handed solutions for something simple like deploying a few demo nodes (which is what panopoly_demo currently does).

If you do go with migrate, I'll be interested to see how maintainable this is.

I do think you could use node_export, as its quite simple and meets the current use case.

Also, sorry for the duplicate issue at: #1903672: defaultcontent is no longer being worked on - switch to new demo content? - I didn't find this until after I had posted.

nedjo’s picture

Isn't the use case for migrate to take data/info from another source and inject it into Drupal?

Yes--but then again that's pretty much exactly what we're doing with default content.

Main reasons I'm favouring Migrate here:

  • It's easy and straightforward to implement.
  • It supports a simple hand-editable format (CSV).
  • It's been successfully implemented and road tested by the most-used D7 distro (Commerce Kickstart), providing a clear model to work from.
  • There's often a need to default more than nodes. We might need file entities with specific fields, taxonomy terms, or various other entity types. While most alternatives are severely limited, Migrate covers pretty much everything and can easily be extended as needed.
  • Facilitating migration is a key need in distros. The same work that's needed for default content also gets migrate support, for free.
saltednut’s picture

Facilitating migration is a key need in distros. The same work that's needed for default content also gets migrate support, for free

This is an excellent point that I had not been considering. A migrate example would be a good way to double-up feature and function, both delivering the demo content and providing the roadmap for distro users.

Thanks for your response @nedjo.

populist’s picture

Project: Panopoly » Panopoly Demo

Moving to this to the Panopoly Demo queue since its not part of the main Panopoly distribution

saltednut’s picture

@populist - here is a reference for importing demo content from a CSV using migrate if you want to go that route (working well for us). http://drupalcode.org/project/df.git/tree/HEAD:/modules/df/df_import

Thanks again @nedjo for showing me the light!

mrfelton’s picture

The thing about Migrate is that its actually conceptually different to Demo content as defined by the Open Apps Standard which states:

One of the hardest things about installing new functionality in Drupal is that often times you never know quite how to use it. The demo content functionality of Apps aims to lower the barriers to using an App by providing some default content that can be easily enabled and disabled. Demo content can be implemented in any way you see fit. Some options are:

  • Manually create nodes, taxonomy, menus, etc and track their individual ids so that they can be removed later.
  • Use Feeds to import/export content from included data files (see: Ideation app)
  • Use Default Content to give Nodes machine names and export them with Features
  • Use UUID Features to export content into Features for inclusion.

There are countless ways to crack this nut; the real goal here though is that you can turn this content on AND OFF. This allows a user to play with content to see how it works and when they are done, disable the module and have all of that installed content disappear so they can start building their own content.

The point here is that demo content can be enabled and disabled. A key differentiator between what defaultcontent (or uuid_features) and Migrate does is that with the former the content is removed when the demo module is disabled. With migrate that is not the case as the content just imported into the database. So in some ways something like uuid_features works better as a democontent module. Although I guess Migrated content could be rolled back when the demo content module is disabled.

mrfelton’s picture

Status: Active » Needs review
FileSize
861.22 KB

Attached patch revamps the demo modules default content to use the Migrate framework. This is much more robust than defaultcontent (which is pretty much depreciated anyway). It defines a single Migrate class for the Page nodes. When the demo content option is enabled the migrations are processed. When the demo content is disabled the Migrations are rolled back, thus deleting the demo content.

Some benefits are:

  • Easy to read and main migration classes
  • Easy to read and main csv data files
  • Default images stored as files instead of encoded data
  • Uses the robust Migrate framework to handle importing and removing demo content.

This patch depends on #2105433: Add Migrate modules to Panopoly Core to help facilitate better demo content patterns which pulls the Migrate modules into the profile.

mrfelton’s picture

Slight update to move the migrate enable/disable to module_enable and module_disable

mrfelton’s picture

saltednut’s picture

Status: Needs review » Reviewed & tested by the community

This looks great! Its pretty much the exact model followed by DF and Kickstart.

My only critique comes with the actual migration class. It looks like some strange tab indentation.

+++ b/import/panopoly_demo.node.incundefined
@@ -0,0 +1,53 @@
+  function csvcolumns() {
+    $columns[0] = array('title', 'Title');
+		$columns[1] = array('image', 'Image');
+    $columns[2] = array('body', 'Body');

I also like the additional menu callbacks, however, they do not take into consideration scope outside this limited migration. If Panopoly ever ships with more than just the 'lovely vegggies' content or does user imports, etc, then these callbacks could be refactored to take dynamic arguments.

All in all, its a major improvement and I'm happy to see this.

mrfelton’s picture

Fix for minor code style issues attached.

mrfelton’s picture

Assigned: Unassigned » mrfelton
Category: Bug report » Task
Issue summary: View changes
dsnopek’s picture

Will this work with Migrate 2.6-rc1 as well? It'd be great to depend on that version rather than 2.5.

dsnopek’s picture

Ok, I began attempting to test this patch so that I could verify if it also worked with Migrate 2.6 or not. However, I'm having a crazy amount of trouble just getting the patch to apply, and decided to ask for some advice. :-)

Patching with patch -p1 < 1869972.14-panopoly_demo-migrate.patch works, but leaves out the binary files, so the images don't work.

I tried patching with git apply 1869972.14-panopoly_demo-migrate.patch, but I get the following errors:

1869972.14-panopoly_demo-migrate.patch:492: trailing whitespace.
 * Implements hook_link_alter(). 
error: cannot apply binary patch to 'import/images/bell-peppers-et-al.jpg' without full index line
error: import/images/bell-peppers-et-al.jpg: patch does not apply
error: cannot apply binary patch to 'import/images/great-veggies.jpg' without full index line
error: import/images/great-veggies.jpg: patch does not apply
error: cannot apply binary patch to 'import/images/lovely-vegetables.jpg' without full index line
error: import/images/lovely-vegetables.jpg: patch does not apply
error: cannot apply binary patch to 'import/images/vegetables-are-great.jpg' without full index line
error: import/images/vegetables-are-great.jpg: patch does not apply

Looking up the "full index line" error, gives some advice for generating the patch with the index info, but no advice for how I can apply it when it's missing:

http://stackoverflow.com/questions/17152171/git-cannot-apply-binary-patc...

Any ideas? Unfortunately, I haven't done much with patches containing binary data...

Thanks!

mrfelton’s picture

Status: Reviewed & tested by the community » Needs review
FileSize
1.62 MB

Updated patch attached (patched using --full-index --binary this time). Could you test please David? This patch also includes an update hook that deletes the old defaultcontent nodes and reimports the new default content using migrate for those users upgrading from an older version of Panopoly.

dsnopek’s picture

The patch worked great this time - thanks!

I did a fresh install and everything worked except:

  1. Everywhere there is a single quote (ie. ') it has a backslash in front of it (ie. \'). I can see these in the CSV file too.
  2. Previously, there were menu items created for each of the demo nodes - with this patch there aren't.

I'll test an update in a moment...

dsnopek’s picture

Status: Needs review » Needs work

I did an upgrade which worked great! All the old content was removed and the new content was created.

But I noticed a few more minor issues:

  1. There were a few unicode characters that no longer come through, for example: Jícama becomes J?cama on the "Lovely Vegetables" node
  2. Previously, only two nodes were marked as "Featured": "Content demo" and "Great vegetables" - however, now they all are! This won't demonstrate the distinction between featured and not.
  3. It would be great if the "created" date that was previously used was preserved - for better or worse, our Behat tests depend on this when testing various Views
dsnopek’s picture

Oh, one more note! We should probably disable the "defaultcontent" module at the end of the update function. (UPDATE: On second thought, I take that back - the user could be using it for other things and we don't want to disrupt them. We should just add something to the release notes saying its safe to do from the Panopoly end.)

I'm going to try and fix some of the easy ones here and post a new patch in a moment.

dsnopek’s picture

Ok! I've attached a new patch (with interdiff) that fixes all the minor issues I found except for the menu items. I think getting the menu items is important, because otherwise the menu which is shown in the left sidebar of a "Content page" will be empty, and so it doesn't demonstrate what that region is supposed to be for!

Anyway, I'm not sure how to implement that. In my (relatively litttle) experience with Migrate, I've done menu items as a seperate Migration class, which we probably don't want to do. Maybe there's a way we can do it in prepareRow()?

saltednut’s picture

I think you guys are chasing a ghost here. The migrate 2.6 stuff was really just a trial thing for D8 stuff from what I've seen. Migrate 2.5 is the stable release. I haven't heard anything about the 2.6 version actually being released and replacing 2.5 since it contains major API changes.

Regarding #16 - is there a reason why it would be great? Is there a particular Feature in Migrate 2.6 that you guys are looking to use? Just wondering, because we are still using 2.5 in Demo Framework without incident. I did attempt an upgrade last summer when 2.6 was announced and saw some issues (that were later resolved). We've been following Migrate closely for a while now I haven't seen a really good argument for trying to support the dev version this late in the D7 cycle with 2.5 being totally stable.

dsnopek’s picture

@brantwynn: The current version of migrate_d2d requires migrate 2.6 - I just did a simple Drupal 6 -> Panopoly migration using it. Anyway, I'm not working on getting migrate 2.6 support right now, just getting this patch working with 2.5 and committed. :-) I'll look into 2.6 support again sometime in the future.

saltednut’s picture

@dsnopek good to know. Thanks for the clarification.

For #22 - probably not a big deal to include the extra menu class in the same .inc as your content migration class. IMO, this is easier-to-follow DX than a preparerow. When using migrate, I tend to avoid doing the preparerow business unless its absolutely necessary because those statements can be often hard to follow. Having a dependee class for the menu items is easy to follow and people who don't want the menu items can choose to roll back just that class - a nice feature you'd get for free.

dsnopek’s picture

I came up with a way to create menu entries without making a 2nd Migration by adding this function to the PanopolyDemoNode class:

  public function prepare($entity, $row) {
    $entity->menu = array(
      'enabled' => 1,
      'link_title' => $entity->title,
      'menu_name' => 'main-menu',
      'plid' => 0,
    );
  }

However, this isn't quite right - it creates all the menu entries at the top-level but the "Vegetables are Great" item is supposed to be a child of "Content Demo" in order to show the menu widget on the Content Page.

So, I think I agree with @brantwynn that there should be 2nd Migration class to handle the menu items.

mrfelton’s picture

I've added the menu migrations (in a separate class), and updated the update hook so that the old demo content (from defaultcontent) is only deleted if the titles of the nodes were not changed. If someone updated the demo content then we should not delete it because it probably forms some part of their site. The assumption being that if the title wasn't change (from being about 'lovely vegetables' etc) then its safe to assume the content is in its default state. We can't use the features override state because in the updated codebase the defaultcontent features files don't exist anymore.

I also moved some of the stuff for the enable/disable hooks to a helper in panopoly_core as this is needed for all of the panopoly apps demo content modules.

Also I've suppressed the status messages from Migrate since they are not needed in this scope.

http://drupalcode.org/project/panopoly_demo.git/commitdiff/refs/heads/7....

However, the build is failing for some reason https://travis-ci.org/lsolesen/panopoly/builds/15829476. Yet installing works from the browser. Any ideas?

nedjo’s picture

is only deleted if the titles of the nodes were not changed

That'll work in many cases, but sometimes (e.g., an "About" page) it might not. Maybe just test if $node->changed > $node->created ?

dsnopek’s picture

@nedjo: The demo content doesn't contain any pages called "About" - only things you'd probably never user like "Content Demo", "Lovely vegatables", etc..

@mrfelton: I'm not sure why it failed on Travis-CI. :-/ I told it to re-run in case it was just a fluke. In any case, I won't have time to really look into it until next week, but I don't think it should hold back the release if manual testing is all looking good.

nedjo’s picture

The demo content doesn't contain any pages called "About" - only things you'd probably never use

@dsnopek: I know, but since this approach should serve as a model for other default content in Panopoly-based features, we should assume that titles may be retained.

I recently added default content to the Open Outreach distro, see #1858094: Ship Open Outreach with feature-specific sample content. In that case, many of the node titles are designed to be retained.

saltednut’s picture

@nedjo I agree, but if that's the case, maybe this should be extending the classes from the 7.x-1.x branch of 'Import' too. :)

http://drupalcode.org/project/import.git/tree/refs/heads/7.x-1.x

Trying to get people on the same system of using Migrate/CSV for demo content, I think goes beyond the scope of only Panopoly based distros.

Just my 2c.

dsnopek’s picture

@nedjo: This is only about the transition from 'defaultcontent' to 'migrate' and is a one time thing only affecting the content that was specifically created via 'defaultcontent'. While using 'migrate' is a model for default content in Panopoly, this one-time update is not! We won't be using the 'defaultcontent' module anywhere anymore. :-)

dsnopek’s picture

Status: Needs work » Fixed

I've opened a new issue for the Travis-CI problems: #2162481: Travis fails to install with Migrate based panopoly_demo

Closing this issue since the Migrate stuff is in!

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

MacMladen’s picture

Although this issue is closed, I like to ask for a little help: I am working on some demo content for a planned distribution and I was wondering if you would like to help me understand how exactly did you make panopoly_demo a module so I can use same procedure to make something like that for some other content types.

I see that it uses features and migrate. On one hand, it seems to me that you have used features to export structures and dependencies so features seems to have exported everything that was needed for the structure.

Did you then add migrate import with enable / disable functions so that demo content can be turned on and off?

Or is there some procedure in features that can both extract structure and migrate at the same time?

I'd be very grateful for any help!