Support for Drupal 7 is ending on 5 January 2025—it’s time to migrate to Drupal 10! Learn about the many benefits of Drupal 10 and find migration tools in our resource center.
Problem/Motivation
If you import from CSV, XML, JSON (non-Drupal sources), it is fairly common to have dates formatted in lots of different ways. Let's provide a mechanism to format them to the required formats provided by Drupal.
Proposed resolution
Code. Do it.
Write tests.
Solution:
- Ability to migrate a source in various date formats to a date field
- Ability to migrate a source in various date formats to a date range field
- Documentation on how to do these migrations on https://www.drupal.org/docs/8/api/migrate-api/migrate-process/migrate-pr...
Remaining tasks
Reviews
User interface changes
API changes
Data model changes
Comment | File | Size | Author |
---|---|---|---|
#108 | migrate_date.png | 56.14 KB | sylus |
#105 | interdiff-2820490-101-105.txt | 928 bytes | shabana.navas |
#104 | backport_for_8.3-2820490-104.patch | 7.91 KB | shabana.navas |
#97 | 2820490-97.patch | 8.11 KB | shabana.navas |
#97 | interdiff-92-97.txt | 1.1 KB | shabana.navas |
Comments
Comment #2
heddnComment #3
heddnBumping this down to v2 for now to get a test pass.
Comment #4
heddnComment #5
Grayside CreditAttribution: Grayside at Phase2 for Norwegian Cruise Line commentedShouldn't this use the DateFormatter service so we can accept predefined formats in the Drupal site as well?
Copypasta
Comment #6
heddnre #5
1) I disagree. There's only a few pre-set formats that the DB will accept the data for a date field. And the chances that those formats are also going to be used to present the data to a public audience are slim. It is a question of audience. Date formats are for people, while the format here is for the DB. The audience for date formatters in the GUI is a reusable date format for non-technical people who aren't writing any code. The audience of a process plugin is a person with some knowledge of software development, who can lookup valid date formats on php.net's website.
2) Fixed.
Comment #7
StevenWill CreditAttribution: StevenWill commentedI tried to test with a d6 date field using the yml format provided in the comments. I received an error that the value is an array.
////
#0 /var/www/html/modules/migrate_plus/src/Plugin/migrate/process/FormatDate.php(56):
Drupal\Component\Datetime\DateTimePlus::createFromFormat('Y-m-d\\TH:i:s', Array, NULL, Array)
This is the yml entry...
field_release_date:
plugin: format_date
from_format: 'Y-m-d\TH:i:s'
to_format: 'Y-m-d'
source: field_release_date
Comment #8
heddnre #7, if you read the IS, this is for dates that come from non-Drupal source. For drupal sources, just migrate the value verbatim into the D8 destination.
Comment #9
RKopacz CreditAttribution: RKopacz commentedHas this patch been committed? Assuming not, so we must apply to migrate plus if we wish to use, correct?
Comment #10
heddnre #9, there is no commit on this yet. It is waiting for someone to review it and mark as tested by the community (RTBC).
Comment #11
RKopacz CreditAttribution: RKopacz commentedGot it, @heddn, thank you.
Comment #12
Ada Hernandez CreditAttribution: Ada Hernandez at MTech, LLC commentedI have tested this patch and works good for my migration.
Comment #13
mstrelan CreditAttribution: mstrelan commentedUnfortunately for me DateTimePlus::createFromFormat() throws an error if my from format is d/m/Y but some of my data does not have leading zeroes. That's probably an issue for Core, but it's worth noting that native PHP's DateTime::createFromFormat() works for d/m/Y or j/m/Y or j/n/Y with or without leading zeroes.
EDIT: Never mind, this can be worked around by disabling format validation.
Comment #14
Albert Volkman CreditAttribution: Albert Volkman commentedRTBC +1
Comment #15
Albert Volkman CreditAttribution: Albert Volkman commentedActually, I take that back. For some reason the value coming in for the first argument of
FormatDate::transform
is an array. I have to alter this patch to pass$value['value']
toDateTimePlus::createFromFormat
. Of course, it's entirely possible that my configuration is incorrect-Comment #16
damondt CreditAttribution: damondt as a volunteer commentedShouldn't you test if $value is null in FormatDate::transform (and in that case just return null) so that it doesn't throw an exception for empty values?
Comment #17
heddnGood idea.
Comment #18
mikeryanPer https://www.drupal.org/node/2566779#comment-11784111, moving to the core issue queue.
Needs to be rerolled for core namespaces.
Comment #19
Manuel Garcia CreditAttribution: Manuel Garcia as a volunteer and at Appnovation commentedPutting the files in the right place and addressing the namespaces.
Comment #20
heddnWrong group/namespace.
And can we get an interdiff? It would make the review easier.
Comment #22
Manuel Garcia CreditAttribution: Manuel Garcia as a volunteer and at Appnovation commentedComment #24
Manuel Garcia CreditAttribution: Manuel Garcia as a volunteer and at Appnovation commentedOops, here's the good patch, interdiff on #22 is good.
Comment #26
heddnSimple re-roll from migrate_plus => migrate module. It has tests and was already RTBC in the other namespace so I think this can be RTBCed here too. I'm simply RTBCing the re-roll.
I've also added a draft change record.
Comment #27
heddnChanging to a task since this is necessary for #2566779: Migration D6 > D8 of CCK date fields
Comment #28
tstoecklerShould we not catch a possible exception thrown here and re-throw a MigrateException? Possibly a SkipRowException or something like that?
I.e. if somehow an invalid value ends up here.
Comment #29
heddnThere isn't a chance of an exception getting caught, because ::format() already catches Exception and adds to an errors array. So it is impossible to catch such exception.
Comment #30
mikeryanYou can follow the format_date plugin with a skip_on_empty to cause a skip - and when #2655154: Optionally log messages for skip_on_empty and skip_row_if_not_set lands, you'll be able to generate a message as well.
Comment #31
heddn@tstoeckler back to RTBC?
Comment #32
tstoecklerBut
DateTimePlus::createFromFormat()
can throw an exception already. See https://api.drupal.org/api/drupal/core%21lib%21Drupal%21Component%21Date...So I think that one should be caught?
Comment #33
heddnGood catch. I didn't notice the createFromFormat; only the format in the calling code. Can we add a test to see if this only happens on a malformed format or is it also possible with a malformed time string? It isn't really clear what causes a FALSE to be returned from createFromFormat.
If it is a malformed format, I'd really like for a loud failure to occur. But if that is lumped in with a malformed data string, well then, I guess we don't have a lot of options. We need to support malformed time strings.
The complexities here are that createFromFormat is throwing an Exception. Which is really nasty. I really hate to catch Exception, it is so generic.
Comment #34
Albert Volkman CreditAttribution: Albert Volkman commentedSo... did anyone see my comment? https://www.drupal.org/node/2820490#comment-11778439
Comment #35
heddnre #34 it is almost certainly due to your configuration.
Comment #36
mpdonadioDateTimePlus::createFromFormat() will throw an \Exception when the $value can't be validated against the $fromFormat and turned into a proper \DateTime object. Think of it as a parsing-regex (eg, preg w/ matches). It is mostly unforgiving, but with some of the format placeholders there can be ambiguity. Since characters that aren't placeholders get interpreted as literals (like for separators), I am unsure is is possible to have an invalid format, just a value that won't match against a particular format.
I'm making an issue to change this to \InvalidArgumentException. It will have to be against 8.3.x, but with the chance to apply to 8.2.x b/c it is a small disruption and also would help this (discussed in IRC w/ @xjm).
Comment #37
Albert Volkman CreditAttribution: Albert Volkman commented@heddn I posted my configuration. Do you see anything wrong with it?
Comment #38
heddnre: #37, please open a new support request for it. But I think the source data needs some preprocessing first. You can use the extract process plugin to retrieve the 'value'.
re: #36, that is great info. So can we ignore throwing the exception then by adding the following snippet to the migration? Or do we still need to try/catch the thing?
Comment #39
mpdonadioYou still need to check for the exception. One can be thrown before that setting is used. See the demo.
I would do something like this. Also adjusted a few small things while I was in there. This comes up green locally.
Comment #40
mpdonadioTwo things about these lines. I think the docblock should be updated to show these as an example, at a minimum the one for timezone.
Also, neither of these lines have test coverage. I don't think settings is really worth it, especially since I think you should almost always be calling that with validate_format==TRUE. Given the number of bugs that have been uncovered and probably still exist with timezones in Drupal 8, I think the timezone setting should have explicit test coverage. It may also be needed once one of the TZ related issues gets ironed out (I referenced a few).
Comment #42
mikeryanComment #43
mikeryanThe plugin doesn't *return* formats, it *applies* formats to the source data.
Before the examples, we should explicitly describe the configuration options - from_format, to_format, timezone, and (as mpdonadio suggests) settings. With a link to php.net documentation on date/time format strings. Please do the same in the change record as well.
Coding standard: catch on separate line.
Comment #44
Manuel Garcia CreditAttribution: Manuel Garcia as a volunteer and at Appnovation commentedComment #45
Manuel Garcia CreditAttribution: Manuel Garcia as a volunteer and at Appnovation commentedAddressing #43
Comment #46
Manuel Garcia CreditAttribution: Manuel Garcia as a volunteer and at Appnovation commentedComment #47
mikeryanThanks! But there's also mpdonadio's request in #40, I agree:
Comment #48
Manuel Garcia CreditAttribution: Manuel Garcia as a volunteer and at Appnovation commentedComment #49
RKopacz CreditAttribution: RKopacz commentedUsing the patch in #45, not working for me. Have a CSV file, the yml works correctly without the date field. But when I add
As per the doxygen comments in FormatDate.php which was installed after applying the patch. 'date' is the column label on the CSV column for the date field.
The import fails. The field in the CSV is configured as follows:
03/01/2006 18:00:00
For March 1, 2006 at 6pm, which seems to be correct. I applied the patch to core 8.2.3
Can anyone spot what I am doing wrong?
Comment #50
mikeryan@RKopacz: The configuration should be indented 2 spaces rather than 4, but that shouldn't make a difference in practice.
Exactly what does "the import fails" mean? If there were error messages, please post them here (
drush mmsg [migration-id]
). If the field doesn't appear to be properly populated on the site, can you look at the SQL table node__field_date and see whether the field was populated at all, and if so how it's formatted?Thanks.
Comment #51
RKopacz CreditAttribution: RKopacz commented@mikeryan thank you for your quick response. I didnt' notice that the pasted text had four spaces. The code in fact has two spaces.
When I run the migration, I get this:
Processed 124 items (0 created, 0 updated, 124 failed, 0 ignored) - done with 'meetings'
That drush command was very helpful, thank you for that! It returned a set of repetitive messages, which all look like this.
87f84924539b4548a3fe8a041f6f8e949efe2b29fa3b429674d82f9f125435a6 1 Format date plugin could not transform "12/03/2014 18:00:00" using the format "m/d/Y".
I thought it might have to do with the destination format, so that seems to confirm it. But is that is not the destination format, what is?
Comment #52
mikeryanActually, it seems like something to do with from_format - in your .yml you have it as 'm/d/Y H:i:s', but the error message reports trying to use "m/d/Y". Is it possible that you edited your .yml from an original value of 'm/d/Y' but have not reloaded (via a config import or config_devel module) your changes?
Comment #53
RKopacz CreditAttribution: RKopacz commented@mikeryan yes, I just saw that before I received your message. I am now rolling back the DB to start fresh, and retrying. I will let you know how it works out.
Comment #54
RKopacz CreditAttribution: RKopacz commentedHa @mikeryan, when I rolled back the DB and reinstalled the migration module for the yml, it worked! Thank you for your quick feedback, those drush commands are like gold. Very helpful.
My last Everest on this is to now pull in a speaker content type using the entity_lookup plugin. Will be experimenting shortly with that. It seems as though entity_lookup docs are focused on taxonomies, not nodes. If you have any insights, they are welcome, although I realize they are not appropriate for this thread. I will be doing them in a group where the dated node will be dependent on the speaker node.
Thank you again!
Comment #55
heddnAssigning to work on the test coverage.
Comment #56
heddnAdded tests and doxygen entry in response to #40. I think if this comes back green, all points are now addressed.
Comment #57
heddnBad interdiff. Better now.
Comment #58
marysmech CreditAttribution: marysmech commentedI tried patch from #56 for date field migration from D6 to D8. When I use format from plugin example:
I get error:
DateTime::createFromFormat() expects parameter 2 to be string, array given DateTimePlus.php:219
I have been debugging the problem and second parameter (
$value
) is array ofvalue
anddelta
e.g.Am I doing something wrong? How should I pass field value? I have been trying
source: 'field_valid/value'
but then is$value
empty.Comment #59
adriancidWith patch #56 and #45 I always have this error:
[error] Drupal\Component\Plugin\Exception\PluginNotFoundException: The "format_date" plugin does not exist.
Comment #60
marysmech CreditAttribution: marysmech commentedI had same problem. For me problem was that I have Drupal installed via composer and after patch apply
format_date
plugin was added to wrong directory (toweb/core/core/modules/...
instead ofweb/core/modules/migrate/...
).Comment #61
adriancidI found that if you apply the patch #56 and reinstall the migrate module the plugin works
Comment #62
mikeryanComment #63
phenaproximaLet's pass $e to the MigrateException so that the calling code has access to the previous exception (and can examine its backtrace or other relevant data).
Comment #64
mpdonadio#2830079: Change DateTimePlus to throw more specific exceptions landed, so we may want to adjust the try/catch to use the more specific exception.
Comment #65
Manuel Garcia CreditAttribution: Manuel Garcia as a volunteer and at Appnovation commentedLike @mpdonadio mentioned, #2830079: Change DateTimePlus to throw more specific exceptions has landed.
Comment #66
mpdonadioUpdated for #63 and #64/#65.
Don't think it is worth having different message strings since we display $e->getMessage(), which will further explain the problem.
Comment #68
mpdonadioAh. I did that against 8.3.x, and #2830079: Change DateTimePlus to throw more specific exceptions was only applied to that and not 8.2.x :/
Here is a proper version against 8.2.x that keeps the @todo and the single catch, and addresses #63. FormatDateTest passes locally.
Attached two interdiffs to make it easier to see the change.
Comment #70
mpdonadioAppears to be a random failure. Opened #2843024: Random fail in LanguageUILanguageNegotiationTest.
Comment #71
josephdpurcell CreditAttribution: josephdpurcell at Digital Bridge Solutions commentedComment #72
josephdpurcell CreditAttribution: josephdpurcell at Digital Bridge Solutions commentedUpdating the description to reflect the needs of the duplicated tickets 2830296 and 2813539 as I understand them--please revise if needed.
Comment #73
quietone CreditAttribution: quietone as a volunteer commentedOnly comments about coding standards, didn't review the code.
DateTimePlus?
And an additional explanatory paragraph would be useful.
extra space before asterisk.
Extra line.
Not sure if this matters, but this is at the end of the docbloc in the current documented process plugins. See ArrayBuild.php
This needs some text. The @see information will be moved to an @see section and no test will appear here.
The indentation of the examples should be 2 spaces.
Needs to show a real world example following each code example.
Doc bloc missing argument documentation
s/Attempt to transformed/Attempts to transform/
There are two lines at the end of the file.
Is there supposed to be a blank line between these?
Comment #74
mpdonadio#73-1, no it should reference \DateTime::createFromFormat directly. DateTimePlus::createFromFormat() doesn't define the formats, and is a fancy wrapper around \DateTime::createFromFormat(). I am not sure what an explanatory paragraph could potentially say, but a @see http://php.net/manual/en/datetime.createfromformat.php would probably be beneficial here.
Comment #75
jofitz CreditAttribution: jofitz at ComputerMinds commentedComment #76
Manuel Garcia CreditAttribution: Manuel Garcia as a volunteer and at Appnovation commentedChanges on #75 look good to me.
Setting to needs work for the outstanding issues on #73.1, 5 and 7.
Comment #77
jofitz CreditAttribution: jofitz at ComputerMinds commented#73.1 Corrected
/DateTime
to/DateTimePlus
and expanded upon the explanatory sentence.#73.5 Added text to docbloc.
#73.7 Added a real world example to each example.
Comment #78
Manuel Garcia CreditAttribution: Manuel Garcia as a volunteer and at Appnovation commentedThanks @Jo! Changes on #77 look good.
Assuming #77 comes back green, I'm taking the leap and setting it again to RTBC since we now have tests, we're taking advantage of #2830079: Change DateTimePlus to throw more specific exceptions being in, and code styling issues have been addressed.
Comment #79
mpdonadioThe docs here for the formats should be referencing \DateTime::createFromFormat with a @see http://php.net/manual/datetime.createfromformat.php. There are some subtle differences between the two that come into play when you are parsing versus displaying.
Still not sure if I agree with this comment here. \DateTimePlus::createFromFormat() is what is being called but the format strings aren't defined or documented there (they have a @see to \DateTime::createFromFormat). When we fix the first point above, this may become moot.
Comment #80
josephdpurcell CreditAttribution: josephdpurcell at Digital Bridge Solutions commentedThe #2813539: Drupal 8 Date Range Migration ticket was moved into this one. Looking at the #77 code I assume this patch isn't for supporting a date time range migration, just date time migrations.
How would a migration to a date time range field work with #77?
Perhaps #77 does in fact support date time ranges by using subfields like @jcisio suggested? For example:
If that is the case, a date time range example somewheres in codes or a test would be useful, or also at https://www.drupal.org/docs/8/api/migrate-api/migrate-process/migrate-pr....
Comment #81
heddnTwo super small nits. And let's just move this to 8.3 an handle the exception properly.
Let's use @link doxygen.
https://www.drupal.org/docs/develop/coding-standards/api-documentation-a...
$e->getCode() is not used and probably isn't necessary.
Comment #82
heddnHmm, date_range is going to be different depending on if the destination or source fields has a cardinality of >1. If it is >1, then it would follow a pattern that uses the iterator plugin:
https://gist.github.com/heddn/00f80f1ac388cd579bb8d142a4fe560f
If it is cardinality = 1, then you'd just do normal process plugin mapping that utilizes sub fields.
https://www.mtech-llc.com/blog/charlotte-leon/migration-csv-data-paragraphs is a blog post that demonstrates the differences in functionality between these types of cardinalities.
Comment #83
mpdonadio#81-1, good idea, but still think this should refer to http://php.net/manual/datetime.createfromformat.php.
#81-2, since the type signature of MigrateException::__construct is
not sure how you can not pass the code and still have the previous exception get used?
#81-3, if anyone wants to tackle this, you can refer to https://www.drupal.org/files/issues/interdiff-56-66.txt
Comment #85
mikeryanThis is a blocker to considering the Drupal-to-Drupal migration path stable.
Comment #86
osopolarAs it took me some time to get it right, here my migration configuration for date fields in a drupal 6 to drupal 8 migration:
Comment #87
mpdonadioThink this addresses all outstanding feedback except #81-2, which I don't think is possible. Test passes locally.
Comment #88
heddnAll feedback addressed. Since this was already rtbc, and I haven't contributed in the 3 months since then I could probably flip the switch. Doing that now.
Comment #89
alexpottThere's no test coverage of setting settings in the migration. To be fair validate_format is not tested anywhere in the code base as far as I can see. Can we can an issue created to add test coverage for it.
I'm undecided as to whether we should add test coverage here. We are allowing migration builders to use it... what do you think @mpdonadio?
Comment #90
quietone CreditAttribution: quietone as a volunteer commentedSo good to see this happening.
I am only reviewing the documentation not any of the code. The doc content is fine but the structure/layout should be the same as the other process plugins. They are currently being documented and this one looks and read differently than the others. Let's make this one similar. See #2776179: [meta] Add process plugin documentation to the codebase.
Summaries are supposed to be one line and begin with a verb. How about 'Converts date/datetime format.
Adding an explanatory paragraph may be useful. Not sure.
Can we made this simpler, it took two reads to process this. 'The destination format.'
All of these should be moved left two spaces and capitalize the first character of the explanation.
Precede all the examples with a single Example line: 'Example:'
Move this to the end of the example and make it simpler. 'If the source value was '01/05/1955 10:43:22' the transformed value would be 1955-01-05T10:43:22'. And the same for the other examples.
Add an @see for the interface. "* @see \Drupal\migrate\Plugin\MigrateProcessInterface"
Comment #91
mpdonadioAssigning to myself to ponder #89; anyone feel free to fix the stuff in #90.
The setting we are talking about handles this:
We have coverage for it in core in DateTimePlusTest::testInvalidDates; the setting defaults to TRUE and the test checks for the exception. We don't have coverage when it is FALSE.
But, this setting on DateTimePlus is really one that I really think shouldn't be there, and that code needs to be refactored anyway. I don't see what you would pass in false. I'm half-tempted to say that we should remove the settings. That is the only setting on DTP.
Comment #92
shabana.navas CreditAttribution: shabana.navas at Acro Commerce commentedAdded new patch with changes applied from recommendations in #90.
Comment #94
Manuel Garcia CreditAttribution: Manuel Garcia as a volunteer and at Appnovation commented@shabana.navas thanks for working on this! If I may suggest a couple of things to help your patches get reviewed:
Comment #95
Manuel Garcia CreditAttribution: Manuel Garcia as a volunteer and at Appnovation commentedOops! did the interdiff the wrong way around. Here is the good one.
Comment #96
heddnthere's probably space to say from what, to what in the 80 characters. I think more is better.
I intentionally mention these formats because when I was migrating into the destination, I found it very useful to know what the required destination format was in D8. Let's explicitly mention what the required destination formats are in the description if we remove the constants. I'm all for #90.5, but let's still slip in what the destination format should be.
DATETIME_DATE_STORAGE_FORMAT
DATETIME_DATETIME_STORAGE_FORMAT
The first is a small nit. The second feels more important to me.
Comment #97
shabana.navas CreditAttribution: shabana.navas at Acro Commerce commentedAdded changes mentioned in #96 by Lucas. The summary now has just a little more detail and added a little bit of explanation for the examples.
Comment #98
heddnLooking good to me now. Thanks for the interdiff on that last patch.
I discussed #91/#89 with @mpdonadio in IRC and we agreed to open a follow-up to handle the testing and changes for validate_format = FALSE.
I've opened #2865829: DateTimePlus test of validate_format = FALSE as a follow-up.
With that, I think this is good to go again.
Comment #99
alexpottUpdating issue credit. I'm crediting people whose feedback had material effect on the patch - I hope I have not missing anything. Thanks to everyone else who tried the patch out on their migrations.
Comment #100
alexpottCommitted 2c2a7a8 and pushed to 8.4.x. Thanks!
As this is an experimental module and an addition setting to patch to ported and will add to 8.3.x once 8.3.0 is released.
Fixed the following on commit.
This should follow the indentation rules. Also @link can be longer than 80 chars.
We should have one @see for this and all the @see's should be together and the end of the doc block. We should just have
See DateTimePlus::__construct()
in the text here.Here's the diff:
Comment #102
heddnTo backport, take the patch that was committed in #101 and replace the exception catching logic to only look for Exception. The patch in #101 should apply against 8.3. But DateTimePlus switches its signature for throwing exceptions.
Comment #103
heddnTagging novice for the backport. See #103 for the steps necessary.
Comment #104
shabana.navas CreditAttribution: shabana.navas at Acro Commerce commentedBackport patch for 8.3.x with change as mentioned in #102.
Comment #105
shabana.navas CreditAttribution: shabana.navas at Acro Commerce commentedForgot interdiff.
Comment #106
heddnAnd the backport is now RTBC. The exception is swapped out.
Comment #107
Manuel Garcia CreditAttribution: Manuel Garcia as a volunteer and at Appnovation commentedExcellent, thanks @shabana.navas
+1 to the RTBC
Comment #108
sylus CreditAttribution: sylus commentedThis was also mentioned in comment #60 but this patch simply doesn't work with composer + cweagans/composer-patches. Has anyone else tried this? All of my other core patches apply perfectly and this one says it does to but ends up with a structure similar to below:
Comment #109
heddnThat's due to https://github.com/cweagans/composer-patches/issues/43
Comment #110
Gábor Hojtsy#2830079: Change DateTimePlus to throw more specific exceptions was in 8.3.x already so why would we need to catch the more generic exception in 8.3.x?
Comment #111
heddnIf that's the case then I don't see why #101 cannot get directly committed to 8.3.x.
Looks like I misread
in #100. This can now be directly committed to 8.3 then.
Comment #113
Gábor HojtsyVerified once again that the right exception is thrown in 8.3.x. And indeed. Cherry-picked.
Comment #115
ozinIn current implementation the source format and destination format use the same timezone which comes from the settings, but we could have case when we need different timezone for destination format, for instance we have date in America/Denver timezone but we need to convert it to UTC. I think make sense to add timezone setting for the destination date:
Does it make sense, if so then I will provide patch?
Comment #116
Manuel Garcia CreditAttribution: Manuel Garcia as a volunteer and at Appnovation commented@ozin I could see that use case.
I say go for it - though perhaps create a follow up issue to this one, since technically it would be a feature request I believe.
Also, you'll have to provide test coverage for it on
core/modules/migrate/tests/src/Unit/process/FormatDateTest.php
.Comment #117
josephdpurcell CreditAttribution: josephdpurcell at Digital Bridge Solutions commentedOff hand, I don't think it is possible to use the "from_format" and "to_format" to specify the timezone conversion for you. As such, the proposal from #115 I think sounds great!
Two ideas on that:
1. Perhaps have the keys be "from_timezone" and "to_timezone" to be consistent.
2. Allow the string values for these keys to be anything that could pass into a DateTimeZone constructor http://php.net/manual/en/datetimezone.construct.php
Comment #118
mikeryanThe format_date process plugin was committed to core and this issue closed back in April. Any proposed extensions to it should be opened as a new "Feature request" issue.
Comment #119
ozinI've creae new issue https://www.drupal.org/node/2883892. This weekend we have Drupal Camp in Kyiv(http://camp17.drupal.ua) I hope on codesprints we will implement this feature.
Comment #120
samirjusic CreditAttribution: samirjusic as a volunteer commentedAs a continuation of comment #86 and for anyone looking for a way to migrate date ranges from D7 to D8, the following snippet works on 8.3.4:
Some notes:
The issue I had was with the date format. It seems that D8 requires Y-m-d\TH:i:s format. D7 date field I had kept the date in a slightly different format (Y-m-d H:i:s). When transferred as-is to D8, the end result is that the D8 date field would never show in views or template files (since, I presume, it was wrongly formatted). The above snippet makes migrating D8 date ranges (or simple dates) from D7 to D8 work fine.
Comment #121
Isaiah Nixon CreditAttribution: Isaiah Nixon as a volunteer commentedFor anyone who is having trouble migrating D7 dates that don't use the date range ability, simply use the first half of samirjusic's code above. The extract function is still necessary. Also, even if the date field in D7 is displayed as mm/dd/yyyy you will still need to use the
'Y-m-d H:i:s'
for the import because that's how D7 always stores it in the databases.Comment #122
taggartj CreditAttribution: taggartj commentedHello all I am creating a module https://www.drupal.org/project/migration_mapper
which when done will make it easy for devs to create custom migrations easily through a ui,
basicly you choose what entity type you want to migrate to upload a csv or paste raw json
and map your data to your entity then export a migration plus config to then run,
and am just now (at the time of this comment) trying to handle datetime fields
see http://cgit.drupalcode.org/migration_mapper/commit/?id=dfdda33
line "// THIS IS SCARRY !!! as every one has a different format."
if anyone can share some knowledge on the best way to handle that array aka:
would it be fare to just put ?
from_format: 'WHATEVER YOUR PHP DATE FORMAT IS IN '
to_format: 'Y-m-d\TH:i:s'
ps Sorry to hijack this closed issue.
Comment #123
docans CreditAttribution: docans commentedI have tried all of the code above but i cant seem to be able to migrate my date field from drupal 7 to drupal 8. Any idea what i did wrong.
here is my code
Can someone please tell me why the body field is able to migrate but the "field_student_article_date" is not able to migrate
Comment #124
docans CreditAttribution: docans commentedComment #125
heddnPlease open a new support request. This issue is already closed for some time.
Comment #126
TMWagner CreditAttribution: TMWagner commentedSeems it would be better to re-open this and keep the thread together. The issue was obviously never resolved.
It is still an issue with D8.6.2
Very generally, the repro for me was migrating from a Drupal 7 site to a brand new install of Drupal 7 (literally brand new install via Composer).
The "template" code generated by running "migrate-upgrade --configure-only" resulted in the following
That fails-
Replacing that with:
field_date: field_news_date
Fails -
The code that works is:
Comment #127
joshua.boltz CreditAttribution: joshua.boltz commentedI've found the same results, where the format_date process plugin is not working.
When i try to use it like documented in past comments in this issue, i am presented with these warnings:
[warning] DateTime::createFromFormat() expects parameter 2 to be string, array given DateTimePlus.php:246
And the migration fails.
The code that works in what's described in #126
Comment #128
Wim LeersFYI: For automatically migrating D7
date
fields with thetodate
setting enabled: they result in data loss when usingmigrate_drupal
today. #3095237: Migrate Drupal 7 date field "todate" value fixes that.Comment #129
joey-santiago CreditAttribution: joey-santiago at Trimble Solutions Corporation commentedI'm migrating a source like this
<field_release_date>2014-09-09 00:00:00</field_release_date>
with this:
Given my settings on the d8 side for field_release_date don't use time for the date.
Comment #130
quietone CreditAttribution: quietone at PreviousNext commentedPublish change record