I have tried migrating my CCK data from a Drupal 6 site to a Drupal 7 site. I tried migrating just one field, though I have 7 fields that have to be migrated for me to consider this upgrade process a success. If you're willing I'd like to get assistance with each of these errors one field at a time as I go through this extremely difficult process. I have researched these matters to the point of feeling like my head is going to explode and cannot find working solutions in what I have read so far. So I am very much in need of guru level guidance from someone that has been through this process before.

I'm currently trying to find a fix for the unlimited length field using a textfield widget error.

---------------------------------------------------------------------------
[this part is in green]

Migrate fields
Status message

Operating in maintenance mode. Go online.
Rolling back field_website_url.

---------------------------------------------------------------------------
[this part is in yellow]

Warning message

Invalid field/widget combination: The field 'field_website_url' in the bundle 'partner' is an unlimited length field using a textfield widget, not allowed in D7. The field length will be set to 255.
Created field field_website_url
Missing formatter: The 'text_hidden' formatter used in 3 view modes for the field_website_url field is not available, these displays will be reset to the default formatter.
Created instance of field_website_url in bundle Partner.

----------------------------------------------------------------------------
[this part is in red]

Error message

Requesting rollback of field "field_website_url" due to failure to convert record:

array ( 'entity_id' => '9', 'revision_id' => '9', 'field_website_url_value' => [value removed, but I believe it is flagged because it is longer than 255 characters], 'field_website_url_format' => '2', 'entity_type' => 'node', 'language' => 'und', 'delta' => '0', 'bundle' => 'partner', )

Cause:

exception 'PDOException' with message 'SQLSTATE[22001]: String data, right truncated: 1406 Data too long for column 'field_website_url_value' at row 1' in [path removed]/includes/database/database.inc:2168 Stack trace: #0 [path removed]/includes/database/database.inc(2168): PDOStatement->execute(Array) #1 [path removed]/includes/database/database.inc(680): DatabaseStatementBase->execute(Array, Array) #2 [path removed]/includes/database/mysql/query.inc(36): DatabaseConnection->query('INSERT INTO {fi...', Array, Array) #3 [path removed]/includes/common.inc(7194): InsertQuery_mysql->execute() #4 [path removed]/sites/all/modules/cck/modules/content_migrate/includes/content_migrate.admin.inc(432): drupal_write_record('field_data_fiel...', Array) #5 [internal function]: _content_migrate_batch_process_migrate_data('field_website_u...', Array) #6 [path removed]/includes/batch.inc(284): call_user_func_array('_content_migrat...', Array) #7 [path removed]/includes/batch.inc(161): _batch_process() #8 [path removed]/includes/batch.inc(80): _batch_do() #9 [path removed]/modules/system/system.admin.inc(2365): _batch_page() #10 [internal function]: system_batch_page() #11 [path removed]/includes/menu.inc(517): call_user_func_array('system_batch_pa...', Array) #12 [path removed]/index.php(21): menu_execute_active_handler() #13 {main}

[There are 6 errors like this, so I'm assuming that means that only 6 of my nodes using this field have data in this field that is more than 255 characters long, but the others are all under 255, so the others did not generate these errors.]
--------------------------------------------------------------------------------------------
[this is what shows in available fields for this field I'm starting with]

field_website_url text

Partner

Invalid field/widget combination: The field 'field_website_url' in the bundle 'partner' is an unlimited length field using a textfield widget, not allowed in D7. The field length will be set to 255.
Missing formatter: The 'text_hidden' formatter used in 3 view modes for the field_website_url field is not available, these displays will be reset to the default formatter.

--------------------------------------------------------------------------------------------

I'd like to solve this textfield widget matter first because that seems to be the one that is stopping this migrate from working. If that matter is resolved, I do not know the 'text_hidden' formatter issue will stop the migration from happening. Note that if the field data gets truncated to 255 and I lose data on these 6 nodes, that is acceptable to me if it means I will get the data successfully imported into all the other nodes of this type. There are nearly 300 nodes using this field on this site, so it will be a major pain to have to recreate all of this manually. If these 6 fields show only the first 255 characters of data after an otherwise successful migration, I can then go into those nodes and just edit these 6 instances of this field to have less characters. That will not be problematic as there are characters in this field on these nodes that can be removed.

If need be, I'd be fine with just editing the MySQL database as needed to change the field length in there, if that is an option, but I do not know how to do this. I would just need instruction for doing that, which would get me a workaround needed if a module fix for this is impossible in a timely manner. I'm on a very tight timeline for getting this site migration from Drupal 6 to Drupal 7 done because there is a limited time for how long our hosting provider is going to maintain anything lower than PHP 5.4 and I have to get 12 different websites upgraded quickly. So any quick workarounds in the MySQL database that will get me past these errors would be fine if you could just provide me with specific instructions on what needs to be done.

Note that I've already upgraded the site from D6 to D7 in all aspects except for these CCK migration problems. I'd much prefer solutions within this D7 version, rather than rolling back to the D6 version and starting the upgrade process over from scratch again, if at all possible. I don't mind rolling back to the D6 version if absolutely necessary though.

Thanks for any assistance you can provide on this issue.

Comments

dman’s picture

I encountered dozens of similar but not identical inconsistencies when doing an upgrade.
Some if these I traced and rewrote as needed ... but occasionally the quickest solution was to :

Identify the problem items from the error messages,
Make a not of the data, and just fix the values in the source.
Replace later if needed. Or just skip it because it was a difficult/wrong value anyway.

That's the most practical today if you have identified that there are only 6 rows to worry about.
Other methods (like extending the default field length on the fly MAY be possible, but I can't say how long it would take you to get it right.

wildlife’s picture

I figured it out just before reading your reply. I found where the data for this field was being stored in the MySQL database. It's in the table content_type_[content type name]. In this case for me it was content_type_partner.

So I went into the six nodes that were throwing the errors and got rid of enough characters in the field to get it under 255. I then reran the migrate module on this field and it worked this time.

So that's a good workaround solution to this problem for people who have only a few nodes going over that 255 character limit where the data can safely be modified to be under 255. Editing six bits of field data is a lot better than editing nearly 300 nodes to rebuild field data.

Now I'm on to the next field that is giving me grief. Should I post it here in this thread, or should I start a new thread for that?

wildlife’s picture

Suggestion for future versions of the CCK migrate module to address this matter.

I think it would be a good idea to add an option for people to choose whether they want the process to continue or not. I understand that this is giving these errors to prevent people from experiencing data loss. But in a situation like mine, it would be much easier to have the results of the initial effort at migrating this field say something like

"You have 6 nodes that will lose data if you proceed with this field migration. The following nodes using this field have data in this field that exceeds 255 characters (list node titles). If you continue with this process all characters after character 255 will be lost. Do you wish to continue? Yes No" Maybe even throw in the mention of editing these nodes in the MySQL database as a possibility as well.

Then code the module to truncate the data in the field before proceeding with the migrate if the user chooses Yes. That way, people in situations like mine that are desperate to get whatever field data they can recognized in Drupal 7 can take what they can get over having to manually re-add everything. Understood if that's not possible, but I thought I'd throw it out there as a suggestion that might save people a lot of work on sites with large numbers of nodes using CCK field data.

wildlife’s picture

I've now succeeded with 3 more fields that were problematic. However with this one and with the others I've succeeded with, I still have this matter:

"Missing formatter: The 'text_hidden' formatter used in 3 view modes for the field_website_url field is not available, these displays will be reset to the default formatter."

This did not prevent the migration from happening, but I am seeing that my Views are exceedingly messed up, and not just with these fields. I know most of that situation will be something to address on the Views issues forum, but I wanted to see if there is anything further that I can do to remedy this specific matter relating to this field migration?

dman’s picture

Not specifically.
I had 20 content types in my migration, so I actually created a stand-alone fake 'hidden' field formatter just to make that warning - and the result - go away. But that's because I was automating re-imports etc.

It's expected that you will have to revisit each of your content type field display modes anyway - that's a big difference between D6 and 7 so needs review.
There you will see that a bunch of previously 'hidden' fields may now be getting displayed. Just drag them into the 'hidden' area again and it should be a lot closer to fine.

It's probably not 'views' related, but may be related to teaser displays - which are often found inside views.

wildlife’s picture

Should I start a new thread for the next matter or keep it in here?

wildlife’s picture

I'll go ahead and post it here

I have another CCK field called thumbnail_image. This image is used on Views listing our Partners. Every partner has a thumbnail image and it is a required field when setting up a new Partner node. I have not tried to migrate this yet. Please advise on the best approach for getting this field to recognize and the images to display. I understand that after migrating this field successfully, I'll have to re-add the field to my Views. I just want to make sure I'll have a successful migration. I've read this thread https://drupal.org/node/1339962 but I'm not at all clear on how I should do this. Again, I'm hoping to be able to do this within Drupal 7 without having to start over again if there is a way to do that. But if I absolutely have to drop back down to a backed up site and database, I can do so.

Here is the warning I'm getting on the initial migrate page:

-----------------------------------------------------------------------------

field_thumbnail_image image

Partner

Changed field type: The 'field_thumbnail_image' field type will be changed from 'filefield' to 'image'.
Missing widget: The 'image' widget is not available for the field_thumbnail_image field, it will be set to the default widget.
Missing formatter: The 'hidden' formatter used in 3 view modes for the field_thumbnail_image field is not available, these displays will be reset to the default formatter.

-----------------------------------------------------------------------------

dman’s picture

Those notices are all fine.

Backup your target site database as far as it is migrated so far, and give it a go.

If you need to reset, it's often helpful to drop and re-create the DB before re-importing (as just re-importing the old DB over a failed conversion causes trouble when remaking new tables) but you should get a routine together that allows you to reset to a last-known-good state pretty quick if you are about to try something you think may fail.

I had a few dozen false starts on my upgrade, so I eventually started taking DB snapshots every few steps, and got used to resetting often.

wildlife’s picture

OK, the importation of the thumbnail image field worked. Good news there.

But now I've noticed that the 6 nodes where I removed characters before migrating those fields are not displaying in Views set up that should have them in the display. If need be, I can just delete these nodes entirely and rebuild them, but I'd like to see if I can understand why this is happening and if there is another remedy first. That way, if I have a situation down the road where I have a site with 1000 nodes and 50 of them need truncating, I could still use that same approach without then having to worry about having to rebuild 50 nodes from scratch afterwards.

And then one more major problem is the migration of a CCK date field for our Events content type. Here is the warning the CCK Migrate module is giving with that:

---------------------------------------------------------------------------------

field_date datetime

Date
Event

Missing widget: The 'date_popup_repeat' widget is not available for the field_date field, it will be set to the default widget.
Missing widget: The 'date_popup_repeat' widget is not available for the field_date field, it will be set to the default widget.

-------------------------------------------------------------------------------------

This field is showing in the CONVERTED FIELDS section. However when I go to edit event nodes, there is no date field present at all, much less a date field present with the correct data for each event. When I go to Structure | Content Types | Event, the date field is not present on the Manage Fields page or the Manage Display page. This is going to be another MAJOR problem if I can't get the dates and times displaying for these Events because we have several hundred events.

If I can get through this problem and then figure out why the 6 nodes are causing fits for Views, then I should be good to go with regard to migrating CCK fields.

dman’s picture

I can't tell what you did with the 6 nodes - I would have just edited them individually using the UI on the original and all would have been well.
If you messed with the DB - I dunno. Or if your view depends on that data - I dunno. Some things you have to figure out based on your own understanding of the site - of which we have none.

ALL the warnings you keep reporting about 'default widget' just mean what they say - the edit widget you will get to use for that field will have been renamed or behave a bit different.
This is all just about the edit UI.
The 'default' is usually a good fallback.
You will want to visit and review each content type field management anyway - to take advantage of some D7 improvements, and also because sometimes it helps to re-save the field UI settings anyway to avoid having undefined settings.

RE other Date issues, I dunno. Do a search as it will be in a different project.

wildlife’s picture

Actually, now I'm seeing that those 6 nodes are present. Something is just wrong with Views. I saw a view that had 255 items when it should have had 261, so I just assumed that the problem was with the 6 edited nodes. On looking at it closer, those 6 nodes I edited are present on the view page and something I haven't determined yet is causing 6 others to not be present. I'm also seeing that two other views are breaking for reasons unknown on the 62nd item in the list. It is not the same node in each view that is breaking it. It's the 62nd node in the list, whatever it may be, that is breaking the View. Very weird and more frustration for me. Upgrading from D6 to D7 should not be this difficult.

I posted about the Date matter in the Date issues area as well in hopes of getting an answer there. Do you think I should try adding a Date field to the Event content type and see if the field data auto-populates in each Event node from what should already be in the database? Or will I risk overwriting all the data if I do that?

wildlife’s picture

Update: The issue with the 6 missing nodes is entirely operator error on my part. The view showing 255 items is actually working properly. It was just coincidence that the discrepancy just happened to be 6 items just after I did what I did with these other 6 nodes.

So that means I'm just down to fixing the Date issue now for the CCK migrate aspect of my site upgrade problems.

rfay’s picture

Priority: Critical » Normal