Drupal 7 branch

Hi,

I'll be create a 7.x branch in a couple of days.

Regards

Hi ishanmahajan,

Any help or motivation needed?

We have a Typo3 to Drupal 7 migration coming up with:

• 292 rows in pages table
• 187 in tt_news
• 654 in tt_content
• 62 in customnamedprojects

And are greatly looking forward to the incremental migration capabilities to allow cleanup of the content while the site build continues.

That's our situation. Let us know if you could use any help, if you laugh at our trivial needs, or anything else. Thanks!

_{benjamin, agaric}

Hi mlncn,

I haven't been able to upload the D7 branch yet and its long overdue!.
There is new release of the migrate module(Migrate 2.1) with some api changes and I'm working on upgrading the D6 branch also. I will upload the D7 dev branch as soon as it is in a somewhat stable position and create issues for the tasks left.
You can have a look at the issues once they are there and any help would be greatly appreciated :)

Regards

Committed a 7.x branch. Some testing is required, after which I'll release a stable version.

Regards

I tried the import of a typo3 site with standard content.
I tried it in two ways:

1. created the tables of the content to be imported into a database on the same server with the same username and password
2. created the tables of the content to be imported into the same database

The errors were the same, so I wonder about the Problem.

What irritates me are the ":db_condition_placeholder_1" bits. It is obviously an error with the database access but I have done exactly what is asked for in the README.txt:

$: drush migrate-import Typo3BeUser
WD php: PDOException: SQLSTATE[42000]: Syntax error or access violation: 1142 SELECT command denied to user [error]
'testuser'@'localhost' for table 'be_users': SELECT u.uid AS uid, u.disable AS disable, u.username AS
username, u.admin AS admin, u.usergroup AS usergroup, u.email AS email, u.crdate AS crdate, u.realname AS
realname, u.password AS password, map.destid1 AS migrate_map_destid1, map.needs_update AS needs_update
FROM
{testdatabase.be_users} u
LEFT OUTER JOIN {migrate_map_typo3beuser} map ON uid = map.sourceid1
WHERE (u.deleted = :db_condition_placeholder_0) AND (u.disable IN (:db_condition_placeholder_1)) AND
(map.sourceid1 IS NULL OR map.needs_update = 1)
ORDER BY tstamp desc; Array
(
[:db_condition_placeholder_0] => 0
[:db_condition_placeholder_1] => 0
)
in MigrateSourceSQL->rewind() (line 321 of
/var/www/.../sites/all/modules/migrate/plugins/sources/sql.inc).
PDOException: SQLSTATE[42000]: Syntax error or access violation: 1142 SELECT command denied to user 'testuser'@'localhost' for table 'be_users': SELECT u.uid AS uid, u.disable AS disable, u.username AS username, u.admin AS admin, u.usergroup AS usergroup, u.email AS email, u.crdate AS crdate, u.realname AS realname, u.password AS password, map.destid1 AS migrate_map_destid1, map.needs_update AS needs_update
FROM
{testdatabase.be_users} u
LEFT OUTER JOIN {migrate_map_typo3beuser} map ON uid = map.sourceid1
WHERE (u.deleted = :db_condition_placeholder_0) AND (u.disable IN (:db_condition_placeholder_1)) AND (map.sourceid1 IS NULL OR map.needs_update = 1)
ORDER BY tstamp desc; Array
(
[:db_condition_placeholder_0] => 0
[:db_condition_placeholder_1] => 0
)
in MigrateSourceSQL->rewind() (line 321 of /var/www/.../sites/all/modules/migrate/plugins/sources/sql.inc).
Drush command terminated abnormally due to an unrecoverable error.

I am using TYPO3_migrate 7.1.2 and as was suggested the migrate module version 7.2.1
with these things (using a clean drupal 7.8)

When using the frontend migrate I basically get the same (see attached screenshot.png)

Hi,
Seems like the user "testuser" doesn't have the necessary MySQL permissions.
Can you do one thing?
Log into MySQL with user "testuser" and run a simple select command on the be_users table. See if it runs properly.
OR
if you have root access to the database then try changing your settings.php to run as root.

Regards
Ishan

thanks Ishan for the fast reply,

I have tried of course, I have run the command as "testuser" and I have every access to the database as well the drupal tables as on the typo3 tables.

I wonder if this is a problem of the versions.
the original typo3 install is version TYPO3 Ver. 4.5.3
drupal version 7.8

I wonder about the syntax of the sql statement, I tried to run the command in mysql itself (via phpmyadmin) it just gave me a syntax error for this query, no permission error.

My second step was to not put the typo3 tables into a different database but I put it into the same database as the drupal installation itself. I can run any command here as this username/password combo.

Now I have tried to run the query again and modified it:

SELECT u.uid AS uid, u.disable AS disable, u.username AS username, u.admin AS admin, u.usergroup AS usergroup, u.email AS email, u.crdate AS crdate, u.realname AS realname, u.password AS
PASSWORD , map.destid1 AS migrate_map_destid1, map.needs_update AS needs_update
FROM {testdatabase.be_users} u
LEFT OUTER JOIN {migrate_map_typo3beuser}map ON uid = map.sourceid1
WHERE (
u.deleted = : db_condition_placeholder_0
)
AND (
u.disable
IN (
: db_condition_placeholder_1
)
)
AND (
map.sourceid1 IS NULL
OR map.needs_update =1
)
ORDER BY tstamp DESC ;

ended in Syntax errors again.
I just cut of the part after the error message:

SELECT u.uid AS uid, u.disable AS disable, u.username AS username, u.admin AS admin, u.usergroup AS usergroup, u.email AS email, u.crdate AS crdate, u.realname AS realname, u.password AS
PASSWORD , map.destid1 AS migrate_map_destid1, map.needs_update AS needs_update
FROM {testdatabase.be_users} 

This query told me that there was no field u.uid, in my tables the fields were called uid, disable...
there was no map.destid1 and no map.needs_update field either nor a corresponding field, executing the query agan as:

SELECT  uid,  disable,  username,  admin,  usergroup,  email,  crdate,  realname, PASSWORD 
FROM {testdatabase.be_users} 

Worked fine and showed me the results. but unfortunately the LEFT OUTER JOIN still seems to be a problem...

Hi,
This could be related to the version of TYPO3. I've tested this module with TYPO3 version 4.1
Can you post here the structure of your "be_users" table. I think there might be some changes in the table structure. Do you have the "deleted" field in your be_users table?

Regards,
Ishan

I just tested the module with a fresh installation of TYPO3 4.5.3. I was successfully able to migrate be users. Attaching the screenshot of the source query.

Maybe there are some hidden requirements I just tested this with a minimal setup, trying now with the standard setup / activating all core modules.
I have now tried with a completely fresh install.
Drupal 7.8
Migrate 7.x-2.1
Migrate UI 7.x-2.1
Typo3 Migrate 7.x-1.2
Typo3 News Migrate 7.x-1.2
Typo3 Pages Migrate 7.x-1.2
Typo3 Users Migrate 7.x-1.2

That typo3 Database stems from a 4.5.3 System the structure of "be_users" is as follows:
uid pid tstamp username password admin usergroup disable starttime endtime lang email db_mountpoints options crdate cruser_id realName userMods allowed_languages uc file_mountpoints fileoper_perms workspace_perms lockToDomain disableIPlock deleted TSconfig lastlogin createdByAction usergroup_cached_list workspace_id workspace_preview tt_news_categorymounts tx_dam_mountpoints tx_simulatebe_feuserusername

I have no other modules installed, and tested only activate the module Typo3 Users Migrate (and the migrate modules)

I came across some minor documentation errors in README.txt of the 7.x.2.1 Version of typo3migrate.
line 26: IMPORTANT: Set the source typo3 database name at admin/content/typo3
under admin/content/typo3 I don't find the settings for the database, these settings reside under:
admin/settings/typo3

My steps:

1. install drupal and modules (versions above) on a debian LAMP stack with Apache/2.2.16 (Debian), MySQL client version: 5.1.49 server version: 5.1.49-3-log, PHP 5.3.3-7+squeeze3 with Suhosin-Patch
2. use a database table-prefix for drupal: alpha_
3. copy the typo3 tables into the same database without table-prefixes, eg. as be_users
4. visit admin/settings/typo3 set my typo3 database to the same as in drupal settings
5. visit admin/content/migrate
6. get the error above

What am I doing wrong?
I have tried to import the examples from the migrate module, - no problem with this, and I was able to rollback completely.
Am I missing to set up some fieldMapping? Or some additional setup in base.inc? I don't actually see any problems here.

I have made headway...

1. Installed drupal 7.9 without table prefix for the mysql tables
2. installed migrate and TYPO3migrate in the versions above (I left them in place)
3. migrated news and users

This proves to me that there are assumptions in the code regarding the table-prefixes (there must be no table prefix in drupal tables)

I have now tried the same again, over and over and it works mostly. There are some issues, though:

1. content of typo3pages ist cut off after some time, it is not exactly clear where this cut happens, I could not discover a pattern yet
2. although standard text format for all text fields is Full html, text in news items and the beginning bits from the typo3pages are inserted as "plain text"

still investigating... should I post my findings to another place?

Hi triple5,

I have made headway...

Installed drupal 7.9 without table prefix for the mysql tables
installed migrate and TYPO3migrate in the versions above (I left them in place)
migrated news and users
This proves to me that there are assumptions in the code regarding the table-prefixes (there must be no table prefix in drupal tables)

You are absolutely right. I guess this a bug which should be resolved in the next release. Right now the TYPO3_migrate module will not work when table prefix is added to the drupal database. Thank you for pointing this out.

content of typo3pages ist cut off after some time, it is not exactly clear where this cut happens, I could not discover a pattern yet

My guess is that this could be due to the value of "group_concat_max_len" MySQL variable. Set this variable to a high value(try 100024) and see.

although standard text format for all text fields is Full html, text in news items and the beginning bits from the typo3pages are inserted as "plain text"

I'll have to test this one out and see what is causing the problem.

still investigating... should I post my findings to another place?

I would request you to create separate issues for the bugs that you find.

Regards,
Ishan

closed due to no activity.