Hello!
Hope all are well.
I have a live drupal site running. I am working on a new one (which has a different layout and additional CCK fields). I am trying to copy content from the live site to the new one.
I tried downloading many modules like import/export, node import, node export but I could not get it to work. For example, when I use the import/export, I keep running into errors when importing the dbase into the new site. I am running My Sql through Toad software which has an import/export feature. I can manage to select which tables to import, but then nothing gets imported due to errors like "null value" in old field, or the new field cannot be empty although when comparing the columns in both tables (the live and new one), they both are empty!
When I tried literally to copy the whole database and dump it in the same location on the new site, it completely messed up the new site, and showed the current design!
All I need from the live site (as far as I know are the users, node, node revision, comments tables only). But when I move the users table for example, it ruins my existing list of users and I lose my admin user account and password.
When I copy the node table, I get the red box error about a wrong CCK field. (As I mentioned earlier, I have introduced new fields in the new system that I will ask the users to go and update their profiles).
Please please please help!
I think I have tried everything, and it is reaching a point it is no longer fun or interesting, plus I need to have it up and running by this weekend.
Thanks for your support.

Comments

zinasahib’s picture

zinasahib commented
Assigned: zinasahib » Unassigned
Priority: Normal » Critical

Still waiting, please help!

ainigma32’s picture

ainigma32 commented
Status: Active » Postponed (maintainer needs more info)

Some more info please: are the two installations using the same version of Drupal (and what is it) ? And the same contributed modules?
what kind of OS, database and webserver are you using?

- Arie

ainigma32’s picture

ainigma32 commented
Status: Postponed (maintainer needs more info) » Fixed

Looks like zinasahib won't be posting any feedback so I'm setting this to fixed.

Feel free to reopen if you think that is wrong.

- Arie

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

zinasahib’s picture

zinasahib commented
Status: Closed (fixed) » Active

Hi!

I actually gave up on getting an answer from the community.

Thanks Arie for replying (I re-opened the issue) - I am still in the same mess.

Please find below the answers to your questions:
"
Some more info please: are the two installations using the same version of Drupal (and what is it) ? And the same contributed modules?
what kind of OS, database and webserver are you using?
"

1. Yes, both installations use the same version of Drupal (version 5.2) as it is mentioned at the top of my issue.
2. The same contributed modules?
No - in the new drupal site, I have to add more modules. I also had to create new content in users profile (CCK) module which does not exist in the first site.
3. OS: XP
Database: MySql 5
running APache and PHP 5

Any help is really appreciated, this issue is taking way too much time and effort to get solved, it makes me question the choice of going for Drupal in the first place.

ainigma32’s picture

ainigma32 commented
Status: Active » Postponed (maintainer needs more info)

AFAICS there are three ways to go about this:

  • Move the old site as-is to the new location and then build from there (i.e. you will have to recreate the changes you made sofar)
  • Try to move the data fro the old site to the new using plain SQL. That will require knowledge in SQL in general and the structure of the Drupal database
  • Use one of the import/export modules you mentioned earlier. That would mean troubleshooting the ones you tried before.

I would try to go for the last option so I suggest you post the steps you took: what module did you use (incl. version) what steps did you take, what errors did you see exactly... If you can't remember just repeat the process.

And finally, I understand it can be frustrating when your questions seem to go unanswered. But please keep in mind that you're dealing with a community of volunteers and your final remark will likely only discourage people from spending time on your issue.

- Arie

zinasahib’s picture

zinasahib commented

Hi Arie,

I appreciate your attention to my problem.

I shall follow your advise and use the import/export module again. Here is what happened in the past 5 unsuccessful attempts:

1. Module used: Backup_migrate-5.x-1.1.tar
2. Details:

I export the data (leaving the default settings) and move the sql file to the new site.
After installing the module on the new site, I import the file.
I end up with a corrupted site: the new design is completely gone, and I can no longer log into the new site (i.e. my admin account/password are changed or corrupted)

On my 6th attempt, instead of importing the whole database (from the existing site to the new one), I selected only a couple of tables (for example the users table).
When I try to import that table, I run into all sorts of errors like: 0 records copied - field cannot contain null value (although the imported table does not include a null record), however for some reason, in the new table I noticed that it does not start copying from record 0, it skips it and starts from record 1, and although I can see that in the window but there is no way for me to have it start at row 0. I think this is why it is giving me this error.

I am using TOAD for MySQL version 4.0 to access the SQL dbase.

As a last resort, I thought, ok - I know it is horrible to do but I will do it - I ended up literally copying each and every record from the tables into the new site. I had to stop half way through because there seems to be something missing in the data, for example, in my live system, when you access a user's profile, you can track all their posts. Now this does not show in the new one. I was told to look into the ERD diagram of drupal database to figure out which table is related to what - boy is that fun to find!

So I am now back to square one, and what is worse, I should have this new site up and ready to be launched by the end of this week. Yes - I am panicking, thus my frustration.

Any suggestions?
Please let me know if you need any further details.

Thanks,
Zina

P.S> Thank you for attempting to solve my issue.

zinasahib’s picture

zinasahib commented

Hi Arie,

I appreciate your attention to my problem.

I shall follow your advise and use the import/export module again. Here is what happened in the past 5 unsuccessful attempts:

1. Module used: Backup_migrate-5.x-1.1.tar
2. Details:

I export the data (leaving the default settings) and move the sql file to the new site.
After installing the module on the new site, I import the file.
I end up with a corrupted site: the new design is completely gone, and I can no longer log into the new site (i.e. my admin account/password are changed or corrupted)

On my 6th attempt, instead of importing the whole database (from the existing site to the new one), I selected only a couple of tables (for example the users table).
When I try to import that table, I run into all sorts of errors like: 0 records copied - field cannot contain null value (although the imported table does not include a null record), however for some reason, in the new table I noticed that it does not start copying from record 0, it skips it and starts from record 1, and although I can see that in the window but there is no way for me to have it start at row 0. I think this is why it is giving me this error.

I am using TOAD for MySQL version 4.0 to access the SQL dbase.

As a last resort, I thought, ok - I know it is horrible to do but I will do it - I ended up literally copying each and every record from the tables into the new site. I had to stop half way through because there seems to be something missing in the data, for example, in my live system, when you access a user's profile, you can track all their posts. Now this does not show in the new one. I was told to look into the ERD diagram of drupal database to figure out which table is related to what - boy is that fun to find!

So I am now back to square one, and what is worse, I should have this new site up and ready to be launched by the end of this week (I had to delay this project since I couldn't figure this out - but this time there is no escape!). Yes - I am panicking, thus my frustration.

Any suggestions?
Please let me know if you need any further details.

Thanks,
Zina

P.S> Thank you for attempting to solve my issue.

ainigma32’s picture

ainigma32 commented

Well I never had this particular problem at hand but I had a look around. The number of modules to do this using Drupal 5.2 are few and not exactly mature.

I was succesful - up to a point - using the Import / Export API module. Here's what I did:

  • Install two instances of Drupal 5.2 on my testmachine using CVS.
  • Next install the module from here http://drupal.org/project/importexportapi
  • When doing a testrun I ran into this problem #127301: Fatal error: Cannot unset string offsets in .. but that issue contains a patch that fixes the issue.
  • Now create some users and content on site1.
  • Go to /admin/build/export and select User, Page, Story and Comment
  • Select XML as the format and click Next
  • Leave all mappings as they are and click Run Export
  • On the next page select all the xml that is generated and paste it in a text editor
  • Now scroll through the data and remove this part
        <user>
      <uid>1</uid>
      <name>admin</name>
      <pass>1a2bff3f4d5b670bb89c0f1234567d89</pass>
      <mail>d5test@example.com</mail>
      <created>Tue, 03 Mar 2009 20:42:08 +0000</created>
      <access>Thu, 05 Mar 2009 13:55:29 +0000</access>
      <login>Tue, 03 Mar 2009 20:50:30 +0000</login>
      <status>1</status>
      <timezone>0</timezone>
      <mode>0</mode>
      <sort>0</sort>
    </user>

    The identifying part is 1 the name and other stuff depends on what you entered when you set up the site.

  • Save the rest of the file in the directory structure of site2 as /sites/default/modules/importexportapi/testdata/variable_data.xml
  • Go to /admin/build/import on site2
  • Select User, Page, Story and Comment
  • Select XML as the format and click Next
  • Select variable_data.xml as the source
  • Click Run Import (and keep your fingers crossed

If all is well you should see the users and content of site1 appearing in site2.

Please note that content with the same node id/version id will be overwritten. If you want to change that you will have to edit the xml some more.

Please post back how that works out for you.

- Arie

zinasahib’s picture

zinasahib commented

Hi Arie,

I just read your answer, I shall work on it right away and see how it goes. I will write back with updates.

Thanking you so much once again for your support.

Zina

zinasahib’s picture

zinasahib commented

Hi Arie,

Ok - Here is what happened:

* I followed your steps exactly (didn't run into the string error), I exported and imported the fields (User - Story - Comment) only since I don't see any 'Page'.

* After modifying the xml file and exporting it, I imported it on the new site.

* It seems that it copied some of the users/stories. Now here is the situation:

1. Only part of the users are copied (till end of Dec. 2008) to be exact. Any users and stories that were submitted after do not appear.

2. I have not set any restrictions as to what users see (to be exact, what members who register see in terms of stories). So they should all see the same stories. However, when I log in as one of the users, I tend to see less stories.

3. On the live system, when you click on "track" in a user profile, you can see all the stories contributed by this user, however, the "track" in the new system shows nothing.

So I guess your solution does help copy some files but not all. I went over the whole procedure another time, choosing the following fields to export (Content Type - User - Node Access Rule - Story - Comment).

I ended up with the same problems listed above.

I notice that the import feature does not allow importing all fields. I accessed the database (using Toad for MySQL) and compared tables, not all of them show up in the ImportExportAPI interface.

Any suggestions? and thanking you once again for your time and support!

Regards,

Zina

zinasahib’s picture

zinasahib commented

Hi Arie,

Hope you are well..
I have been looking at the data some more, you know what? I don't think it copied anything!
It turns out that the data that exist is what I had moved manually!

When I browse the exported XML file in a browser, I get "invalid character in file...." error...

I tried exporting only Users - same thing!

zinasahib’s picture

zinasahib commented

Hey hey....
An update,
Here is what I did so far:

1. I exported the Users table only using the XML way you explained above. After importing it, it works fine - YAY!

2. I exported the Story part using csv this time, since the XML file for this category has errors. It ended up in 3 files (story revisions, story collections and Taxonomy terms). I imported it but I ended up with this error:
"warning: Invalid argument supplied for foreach() in C:\Program Files\Apache Software Foundation\Apache2.2\htdocs\drupal\modules\cck\fieldgroup.module on line 394."

Well - its only a warning that I can always 'comment out'. However, I do have a (0 reads) next to each story.

This is were I am right now, I'll move on copying the Comments....

zinasahib’s picture

zinasahib commented

Hi again - Arie,

Hope you are well.

Sorry for the many posts, but I tried to report on my progress...

Well, I moved the Comments table using the XML way and it works!!!

My new site now has the old content without affecting the actual design. There are a few tweaks to do here and there, like a couple of duplicated comments showing for 1 story while another story has no comments! The tags of the stories are missing....
So I ended up doing a few modifications to the database itself. (I still have a few more to do).
As far the the "track" pb, it seems to be working fine now after copying the 'comments' part of the database.

Arie: I owe you a drink!

Thanking you once again for helping me out and not giving up on me ;-)

I hope these posts would help others who run into this 'unique' problem.

Regards,
Zina

ainigma32’s picture

ainigma32 commented
Status: Postponed (maintainer needs more info) » Fixed

You're welcome and thank you for posting back how you managed to solve it.

And I will take you up on that drink if (when) we run in to each other at some Drupalcon :-)

BTW you really should upgrade to the latest 5.x version ASAP

- Arie

zinasahib’s picture

zinasahib commented

Hey Arie -
Thanks for posting,

I am running into a weird problem now (I know, all my problems are weird!)

I don't know if I should continue on with this post or start a new one. Here it is (or actually, they are 2 issues)

1. Members can no longer view comments on stories! I went through access priv. / user access / all settings basically and have no idea what causes this problem. This is happening on the new site that I moved the content too.

2. I am running the "Fancy" theme. I added a 'mission statement' and I expect it to show at the top of the content. After setting it up in the block and giving it a value of 0, it still shows below the stories! Looks awful!

Any ideas?

Thanks once again....

Zina

P.S. I am already running Drupal ver. 5, what did you mean by an upgrade?

ainigma32’s picture

ainigma32 commented
Status: Fixed » Postponed (maintainer needs more info)

Re 1: That might be caused by a difference in the node_access table or maybe in user_roles. Could you compare the contents of this table in site1 and site2?

Re2: I would suggest posting an issue on the queue of Fancy here http://drupal.org/project/fancy but it looks like it's not maintained anymore.
I'm not sure what you mean by setting up a block for that. It is usually just printed using page.tpl.php
Look for something like print $site_slogan IIRC that's what prints the mission statement.
If that doesn't do it then yes, open a new issue for that problem.

About upgrading: I meant to say update. You say you are running 5.2 and the latest is 5.16 (!) That holds quite a few security fixes so it is important to stay up-to-date.

- Arie

P.S. You do realise we are pushing the two-drinks boundary... :-D

ainigma32’s picture

ainigma32 commented

Thinking on it some more it's probably $mission in stead of $site_slogan

- Arie

zinasahib’s picture

zinasahib commented

Hey Arie -

Hope you are well...

I am still trying with both problems with no success (and honestly it does not even make sense!)

1. I followed http://drupal.org/node/57136 and all tries are unsuccessful.

2. I compared the node_access and user_roles tables in both sites and they are identical! It is weird! I even compared other tables like access, comments, permissions and there is no difference.

Any suggestions?

I'll keep trying in the meantime and will post any updates...

Zina

P.S. Its (-28 degrees C) outside, may I send some snow and windchill your way ;-)

As far as drinks, keep counting buddy!

zinasahib’s picture

zinasahib commented
Priority: Normal » Critical

Hey hey,

Ok, so now I managed to solve the 'mission statement' problem that was showing at the bottom of the content although the code clearly suggests otherwise!

Here is how:

Go to Administer / Themes (in my case it is the Fancy theme), click configure, and enable "Mission Statement"!

Well, I wonder what I was viewing at the bottom of content then..... ahh.. I guess I'll stop wondering and move on to the next problem... which is about the inability for members to view comments on the new site!

Arie- I still wanna send you some snow and wind chill your way ;-)

Regards,
Zina

stevenpatz’s picture

stevenpatz
he/him
commented
Priority: Critical » Normal
zinasahib’s picture

zinasahib commented
Priority: Critical » Normal

Hi Arie,

Hope you are very well..

Here it goes:

To solve the pb of users (or to be more accurate - members) not being able to view others' comments, go to:

Administrator - User Management - Access Control

Scroll down to Comments Module and check the box corresponding to 'administer comments'. Now I am not in favor of this solution though since it allows members to edit or delete other members' posts.
Also, I definitely do not want to enable this checkbox for anonymous users, so I need another solution.

Any ideas?

Regards,
Zina

zinasahib’s picture

zinasahib commented

Another update....

Let's go back a few steps and remove the checkbox for 'administer comments' to anonymous and registered users.

Playing around with it, I came across a tab I don't frequently use (Administer - By Module) [I usually use the Administer - By Task interface].
Navigate down till you reach "Comment Module" - click on "Comments" then click on the second tab (Approval Queue). To my pleasure, here I found all the comments that are not currently visible to registered users.
Just approve them and voila!

Now both users and anonymous users can view the comments submitted by other users without 'messing up with them' ;-)
Keep in mind of course that in my Access Control permissions, I have only the 'access comments' enabled for both and 'post comments' enabled for registered users only.

Moving on to more issues.....

ainigma32’s picture

ainigma32 commented

If you keep going like this you will have written the "Guide to moving content between Drupal installations" :-D

Seriously though: thanks for sharing your solutions!

- Arie

zinasahib’s picture

zinasahib commented

Hi Arie!

No problem - and thanks a lot for your support and guidance in helping me solve the problem (s).

I have another one for you (I think I should start a new issue, but I am not sure if you will receive it.) Now for this new site, we need to track users behavior in terms of what they click on, what they read, how they reach a link,... etc.

I've gone through all the offered modules and the closest to what I want to do is a module called "Pathtrack". However, when I install and enable it, I run into many errors / problems, and can't even start using its features. Not sure if its a problem with the "Statistics" module that depends on it or its the module itself. Couldn't find answers at the module's link. I couldn't figure it out so I removed the module.

Any suggestions?

Thanks,

Zina

ainigma32’s picture

ainigma32 commented
Status: Postponed (maintainer needs more info) » Fixed

You're welcome.

Please do open another issue with that question, as long as you make it a support request for the Drupal project I will see it.

I will have a look around and post in the new issue.

Setting this one to fixed because it's ... well ... fixed :-)

- Arie

P.S. Don't forget to do the update to 5.16 !

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.