CCK custom content type fields and scheduling using mailhandler

yes i have the problem and how about integration with token

Yes, is there a way to post content to specific fields

I would be very interested in hearing how to do this as well.

this would be very useful.

I am also very interested in doing this.

i'd love to know how to do this too - just to join the chorus :-)

I am specifically trying to get this working with the calendar module, so that our events coordinator can send in events via email.

I haven't found any ready made solutions but there should be a nice clue in the http://drupal.org/project/mailsave project. Imagefield handles the saving bit for images into CCK fields which is a far more complex trick than putting text into a field...

So if you are just looking for textfields I'm sure you can find out how to do this from this code.

Email sucks for sending structured information like this. I don't see this getting solved anytime soon.

still, how would you go around implementing a simple url field insertion for the userlink content type?
this is very crucial to us, since content fields are at the core of our site and, with or without relevance, taxonomy parsing keeps failing - even after removing the free tag vocabulary.

besides, what do you mean by "Email sucks for sending structured information like this"?
it is are already (supposedly) supporting taxonomies, which are hierarchical, so what's the problem supporting an extra field for enabling sending of interesting content types such as userlink - something e-mail would be ideal for?

I'm actually doing this in the 6.0 version ... I'm using the mail handler hook and parsing through parts of the message and then populating the cck values that I want. I agree that email isn't great for doing this, but sometimes it is very convenient. One of the other things that I'm doing that might be of some interest is that I'm using plain text as the node body, but then capturing the html formatted message in a seperate cck field ... that way, I've got something that is easy to read and don't have to worry much about bad html ... but also get the context of the original fully formatted message if I need to reference it.

The difference between taxonomies and CCK fields, is that taxonomy terms are numbers, while a cck text field may contain line breaks, html code, symbols, and basically - anything.

I see Moshe's point in saying that email is not suitable for more-than-the-plain-text type of input, and although we do pass arrays, they are still one liners.

However, since CCK is indeed the de-facto way of constructing a node (and it's doing it's way into core), we should find a way to relate to it's fields. I don't know how.

@matt_paz: please share your processing of the fields - maybe we can learn something from it.

Here's a simple example ...

function mailhandler_mod_mailhandler(&$node, $stream, $msg_number, &$header, $mailbox) {

            $node->field_user_stakeholders[0]['value'] = $header->fromaddress;

}

@matt_paz: Thanks. Can you show an example mail also - I was more curious about the mail side of the processing.

@z.stolar -> Not sure what you're looking for there ... can you elaborate? Thanks, Matt

Yes,
Let's say you have a cck field of Address. It has the value of:

field_address: 134 76th st.
Apt. 4E
Forest Hills. Queens
New York 12345 NY

If you stick it in the mail's commands section, by default, the new lines will break your field.
I am interested to know how you instruct your users to structure their mails, in order for that field to be captured correctly (e.g., do you use special headers, or strings, to delimit the different fields?)

@matt_paz - is your code for D6 or D5? This issue is similar to what I'm seeing in D5 when I try to set a Date field and a Text field (that's configured for a dropdown widget on form entry) in hook_mailhandler().

I wouldn't be surprised if this "just works" in D6, though.

There are some pretty good use cases for wanting to stick email information into CCK fields.

Having CCK fields support in mailhandler is a growing demand, and as CCK moves slowly into core, and as in many cases there is very little use of the classic title-body structure of a node, I think CCK support will be the next major addition to mailhandler.

I still don't know how, though.

I believe that was a D5 snippet ... looking forward to porting to D6 tho'

with the latest releases it sort of works, at least it grabs the first character.

I've tested so far with text fields, floats, and integers, but every time cck grabs the first character only and any data sent has to be int eh format field_whatever_field: [value].

I was getting "cant unset string offsets" up until the latest release of cck just came out, now it seems to go through but only grabs the first char.

So... How much will it take?? This has been a major feature request for years. Who wants to be famous as the savior of mailhandler?? How much do we need to raise to get someone to fix this??

Anyone?? I wish i had a clue but i'm just not good enough yet.. I figure if we can pass taxonomy why not cck?? I enter a value in a field on the form and it goes through, why not sending that value in the email?? I guess i just don't grasp why this is such an impossiblility.

@regmanabq: see comment#12 and comment#8 for few reasons why this is difficult.

However - it should be possible. I'll try to come up with a simple solution that can be used as a basis for modifications.

subscription

Bounty for Integration Work
What bounty would it take to allow Mailhandler to talk to CCK?
Perhaps a capable someone (I'm not, but am willing to test) can make this feature happen for text, numbers, dates and node reference fields (lookup by ID and by title cross reference), and nice to have but not critical: imagefields and filefields.
If one or more of those critical someones could provide some indication of how much that bounty should be, I'll contribute as much I can (money and testing time) and hopefully interested others will match / contribute.

Preliminary Scoping
If detailed function scoping is a challenge, here's a starting point of what I could imagine working - nothing in stone, but sometimes change is easier than create. Please excuse any noob oversights or lack of undertanding vis a vis not understanding the core module code and how it operates.

----- Sample Email Excerpt Start -----

CCK(Name)[Fred Flintstone]
CCK(Picture)[*.jpg][Dino.jpg]

----- Sample Email Excerpt End -----

When sent to the polled address, if the above excerpt was in the email, the processing would adhere to the other Mailhandler commands present in the email submission, but the two lines would be processed as follows:

The first line would try to populate a CCK field named “Name” to be populated with the value “Fred Flintstone”. If the “Name” field was a text field, it would simply populate the field – in the event the number of characters in the field was smaller than the number of characters in the submitted email, then the posting would simply be truncated. If the “Name” field was not present, a number field, or date field, then the module would register an error per the Mailhandler error handling routing which allows the transmission of a rejection email to the submitting address indicating “Field Not Present” (if the field wasn’t present in the target node) or “Wrong Field Type” (if the field type would not allow the contents to populate the field). If the “Name” field was a node reference field, then the module would test if the value “Fred Flintstone” exactly matched a node title in the list of nodes eligible in the “cross referenceable” fields – and if no match was found, it would test if the Value “Fred Flintstone” exactly matched a node ID in the list of nodes eligible in the “cross referenceable” fields.

In the event imagefield or filefield integration was accomplished, the second line would attempt to attach the first attachment with a “jpg” extensions into the imagefield or filefield named “Picture” renamed to “Dino.jpg”. In the event there were no “jpg” files attached no insertion would be made. If more than one was present then only the first in the sequence would be attached.

With the CCK fields applied as required, the logic for confirming successful node creation would be tested (e.g. all required CCK fields were present, non-duplicate field node titles were not effected, etc.) In the event any node creation logic failed, then the module would trigger the “node failed” process within mail handler and transmit notification of that failure back to the submitted, with some description of the failed logic to allow correct submission. Similarly, all other mailhandler and CCK logic should be respected.

Markdown syntax recognizes a <br /> when we end a line with two or more spaces. Maybe it could be an interesting way for Mailhandler? For example, say a space is represented by •:

field_address: 134 76th st.
Apt. 4E
Forest Hills. Queens
New York 12345 NY••

So the field_address' value would be all the lines until the one ending with two or more spaces.

If we accept the basic idea of using field names (which are not always intuitive), and some sort of markup in email body, then what do you think of the following format:

usual processing

status:1
type:story

text field with the name of field_description

[field_description]
here comes the content of the description field, with <strong>HTML capabilities, when applied</strong>
[/field_description]

This field will be validated according to the same rules that apply to content submitted directly on the site.

This approach adds complexity to the mail composition, but hey, we knew it wasn't going to be fun :)
It should also be rather easy to implement, as it can be implemented as an independent module, which defines the above input format.

Yes, why not. Anyway, with an email template, it's not so complex.

The email markup sounds good.

I can appreciate your concern about using field description - its a usability tradeoff, but I think on balance, it makes sense to use field description.

Another thought around this feature, that might make the discussion moot is, what about integrating MailHandler and the Rules engine? I ask because the rules engine already populates CCK values (and also exposes other hooks like publishing status, commenting, users, etc.). Might a more powerful solution design be giving MailHandler a simple syntax to use to identify fields and subsequently expose them to the rules engine?

Might there be some code to borrow from the WordFilter project?

My preliminary review suggests:

Advantages of this design would be:
a) Rules wouldn't have to work on all content creations of a certain type
b) MailHandler could drive additional events (approvals, etc.)
c) Tied with Rules email action - complex workflow validations would be possible

Drawbacks of this design would be:
a) Increased overhead for processing
b) A larger break from the original programming design of MailHandler
c) Most feeds for Rules are a firm object in a Drupal environment, and the thinking seems to be towards consolidating the exposed objects uniformly to core functions (like Views). Is it too ambitious to contemplate something as odd an email event in this context?

I’m not sure if this is relevant, but could looking into to Node Import (http://drupal.org/project/node_import) be helpful? Maybe you could get node import to accept a spread sheet via email.

What would be really awesome is if someone could figure out a way developers could create a form on there Drupal site that would allow users to submit multiple nodes of the same type from one form.

For example: I go to my site where I have set up a create multiple content page. I fill out the required cck fields in table style format – when I click submit it sends all the completed rows to mail handler in an email (each completed row = 1 node – then mail handler sends it to node import , and the fields are processed and nodes are created. This way I have the option of either creating my nodes from a csv file or just going to the site and filling out the form.

If some one knows of a way that this can already be done using druapl 5 please let me know.

Ssubscribing

@troym559: This is a bit off-topic.
Also, if you already create your content on your site, you don't need mailhandler.
I suggest propose it at the node_import project issue queue.

@mkmckinn: I'm not thoroughly familiar with rules. Can you expand your idea?

However, take into consideration that our solution should be simple. I'm not against mailhandler using modern APIs, but it's functionality shouldn't get more complex than it already is.

@z.stolar,
Sorry to take so long to respond.
My thoughts around using the rules engine is actually to leverage other modules work - similar to your templates.
I'm thinking that the rules API can provide hooks to next workflow steps - perhaps a way to extract form processing from email creation. This would let mailhandler focus on creating the node, and then passing post processing on to other modules. Those other modules might include ones that parse newly created forms and adjust form field values. If we wanted to get tighter, one might also note that the rule engine itself can populate CCK fields and as a result, if we could have mailhandler fire the rules engine based on parsed entries, that set of rules could be used to populate the fields themselves.
For example, imagine a tweak to MailHandler lets it find all entries in the body that look like ("FieldName")_:("Content"), it then fires rules that for that node, try to populate the CCK field named "FieldName" with "Content". We might be limited to text based parsing to start, but it might take us part of the way there.

Cheers,
--Malcolm.

I've just read through this issue. Messages could be processed several different ways. What if we make mailhandler processing pluggable? This would allow different processors to be written that handle things differently as needed. There could be the existing command-based processing as a plugin, but then also a feedapi-based parser and mapper that could work with feedapi and the feedapi element mapper (http://drupal.org/project/feedapi) and http://drupal.org/project/feedapi_mapper). Feedapi is somewhat a misnomer because of the great flexibility of that module. It lets you write your own "feed" parsers and processors; for example, there's a feedapi CSV parser that will take CSV data which you can map to nodes and fields.

This will help keep the main mailhandler module more of a reliable library, and make it easier to write plugin modules. Thoughts?

Ian

What about the existing mailhandler hook?
This hook passes just about everything about the messages, to modules implementing it, and can be used for further processing.

Does the processing you talk about need to be invoked in a different manner?

The way it works now, every message runs through mailhandler_process_message() which does the command parsing, including building the taxonomy parameter, etc. Only after this is hook_mailhandler invoked. Granted, hook_mailhandler lets you do pretty much everything you need in order to further process and alter the node object. But, what I'm thinking is that mailhandler_process_message() could be the point at which to plug in alternative processors for a mailbox, and make the current commands processing optional in order to allow some other processor to completely take over. I imagine though the current command processing would remain in mailhandler module and serve as the default processor for a mailbox. I am wondering whether there is some baseline functionality in mailhandler_process_message() that all processor plugins would want to benefit from, or whether nothing should be taken for granted. I don't want to take this thread too off-topic. My main thoughts are that:

1. The CCK handling should be done in a separate module
2. Any reason why it cannot be done in a separate module might be a good sign to make small tweaks to the mailhandler module to make it possible and keep the mailhandler interface clean and nicely pluggable.

If #1 is already possible, then I can open a separate issue that's focused on making the processor selection more pluggable in order not to divert this thread.

Ian

CCK support could and should be processed in an external module / .inc file.
Also, tweaks to the processing process can be introduced, to allow complete override of the default processing. That's a nice idea that will allow whole new usage of mailhandler's basic functionality.

However, I don't think the two have to be tied together (e.g., I don't think they're dependent).

Subscribing

Looking for any existing code to help get this posting in D6.

Actually i would not need this, but default values for CCK fields seem to be ignored for content created with Mailhandler which might be something to address in Mailhandler directly.

sub

subscribing, greetings, Martijn

Would this help anybody else ? - it's a patch on the D6 revision retrieving revision 1.1.2.24
MOST cck fields use 'value' - but not all.
You mileage will vary.

field_MyCCKField: something
will work,
arrays wont
neither will cck fields that done use value.

cvs diff
cvs diff: Diffing .
Index: mailhandler.retrieve.inc
===================================================================
RCS file: /cvs/drupal-contrib/contributions/modules/mailhandler/Attic/mailhandler.retrieve.inc,v
retrieving revision 1.1.2.24
diff -r1.1.2.24 mailhandler.retrieve.inc
276c276
<
---
>
314a315,316
> } else if (substr($data[0],0,6) == 'field_') {
> $data[1]=array(array("value"=>$data[1]));
315a318
>

Would this help anybody else ? - it's a patch on the D6 revision retrieving revision 1.1.2.24
MOST cck fields use 'value' - but not all.
You mileage will vary.

field_MyCCKField: something
will work,
arrays wont
neither will cck fields that done use value.

cvs diff
cvs diff: Diffing .
Index: mailhandler.retrieve.inc
===================================================================
RCS file: /cvs/drupal-contrib/contributions/modules/mailhandler/Attic/mailhandler.retrieve.inc,v
retrieving revision 1.1.2.24
diff -r1.1.2.24 mailhandler.retrieve.inc
276c276
<
---
>
314a315,316
> } else if (substr($data[0],0,6) == 'field_') {
> $data[1]=array(array("value"=>$data[1]));
315a318
>

Maybe we should get a list of all fields in the submitted node type, and then look for occurrences in the mail's command lines, rather than guessing the name of a field, and it's structure.
In case we find a valid field name, the way to process it's values will be based on the field's configuration.

subscribe

Subscribing. Being able to fill in a single node reference field would solve a huge problem for me.

Subscribing.

I'm working on feeds integration for mailhandler for a 2.x version. No code committed yet but this should work well using feeds module's mapping API. I should have a prototype soon with a limited feature set.

Subscribing

+1 Subscribing

+1 Subscribing

i think moshe's point about trying to add structure to email is spot on. however, you may be able to get around some of this by adding an email template export...

perhaps a tab added to the create content form for each content type for which mailhandler is enabled...

click on a tab that says "Email Template"...

the resulting page displays an example email template for copy/paste and a button that says "email me a copy" or some such....

thinking out loud, but this feature request seems to rely on a bit of training for the end user, which means it needs to be taken into account or it will not be adopted/used to it's full extent.

been monitoring the oil spill since april and the tools that are proliferating to try and help citizens post their sightings etc. seems like an oil report content type with email template would go a long way to allowing folks to use any platform that supports email.

e.g...

[location]
add a street address here. closest city and state minimum please
[/location]

[oil_type]
sheen, tar balls, etc.
[/oil_type]

...you get the point

i guess it could be abstracted to any channel that messaging supports but that's OT for this queue.

just my .02

r,
c

+1 Subscribing

+1 Subscribing

Subscribing - I'd love to see multigroup support too! haha ;)

If anyone's interested in taking this development on, I think Mailhandler 2.x (http://github.com/ianshward/Mailhandler-2.0) is nearing stability enough for someone to start working on a cck field command plugin which will let you use some kind of markup in your email and map to cck fields on the node. Get in touch with me if you'd like to discuss how to get started.

I have opened a new issue related to this feature: #921774: Support CCK creation using Feeds and Mailhandler

I'm interested, but not capable. perhaps there is an interested developer interested in a bounty / sponsor?

Adding CCK fields mailhandler command would be very Great! Does anybody here as an idea on how to do it?

Right now I confess I'm doing a fantabulous kludge. Using Mailhandler to import nodes of one document type, which in turn then become a feed for the Feeds module to import.

It seems like someone who knows more coding than I should be able to do this integration. I'd at least be willing to contribute to a bounty.

I am new to Mailhandler.

However, doesn't Mailhandler already handle taxonomy?

And Rules handles taxonomy and CCK, I think, again, not too experienced.

Couldn't Rules convert taxonomy based on Mailhandler into CCK fields?

I see some mention of Rules above but not this specific idea.

Found this thread on Rules and Taxonomy. http://groups.drupal.org/node/101159

I am not yet successful converting Taxonomy to cck field but it seems possible with some php code.

Here is how to convert one taxonomy term to cck using Rules.

I use it by setting up a taxonomy term in a mailbox and then using Rules to convert it to a cck field after saving.

Rule-
Event: After Saving New Content
Conditions: (Textual Comparison [node:term] + term) AND (content type)
Action: Populate CCK field

Rules [node:term] apparently only looks for the lowest value taxonomy term so this only works with one term per node.

There is also a PHP code which could be used as the condition and could handle multiple terms but I have not figured that out yet.

Rules module populates cck fields.

For me, I created Event -> Node Creation and Action -> Populate CCK Fields.

Very easy. Other options can be added.

Thus As I see it, a text needs to be added to the documentation of current mailhandler along the lines of

Howto populate CCK Fields via mailhandler
{... the full description with example has to be added here...}

And then this issue can be closed?

It would be great if someone who has this working could fill in the example and some texts.

Absolutely... I agree, if someone can do that, then we're in a position to close this issue. ;)

This is working well in Rules, if anyone has an interest I will be happy to write it up.

Yes, please itserich, if you could write up that text, that would be great!

I'm confident that if you notify the maintainer (http://drupal.org/user/4736/contact) and cut'n paste him the text he will be happy to include it. You could also pull the GIT repo and make a patch, but maybe this is to much to ask for if you're not into these things ;)

Using Rules to populate a CCK Field is relatively simple.

Rules is installed like any module. At the Admin menu go to Rules -> Triggered Rules -> Add a new rule

There are three steps in creating a Rule: The event, the condition, and the action.

Event - Content is going to be saved
Condition - Node: Content has Type (choose whatever content type is created via Mailhandler)
Action - CCK: Populate a Field - Choose the Field to be Populated and the CCK Field should pop up and can be populated with text or a token.

Attached is a screen shot.

This is the basic way to populate a CCK Field.

There are ways to customize it. For instance, I have the same content type created on the web site, and for those items I do not want CCK fields to be populated, so I create a unique content type when it is created via Mailhandler, populate the fields, and then set up another Rule to convert the node to the same type as is used when the node is created on the web site.

I have not perfected the conversion yet, currently it happens upon viewing the emailed content which is not efficient.

+1 Subscribing

+1 Subscribing

I haven't read the whole issue, but setting fields via commands can easily be accomplished in 6.x-2.x and 7.x-2.x, I would recommend upgrading to those versions.

If there are still issues remaining to be solved here, please re-open.