Contact form emails not arriving.

In my case, this turned out to be rather more complicated than I ever expected and was all to do with SPF records.

I came to learn that SPF records are similar to DNS records and tell a recipient's email service which domains are allowed to deliver email on behalf of the sender. This is to guard against email address spoofing, where a message has apparently been sent by a genuine sender but in fact has originated elsewhere. Moreover, this issue is nothing to do with Drupal and to confuse matters further, email address spoofing has a number of valid applications, as well as also being widely used by spammers.

To enable a quick reply, email from a Drupal contact form appears to be from the individual who completed the form, but actually this is not the case as the email has actually been sent using php mail by your Drupal website at www.yourdomain.com or perhaps via an smtp server such as mail.yourdomain.com. Alternatively it may have been sent through an unrelated IP Address or domain, perhaps that of an external mail server.

If using SMTP an SPF record may be included with the inbound email to verify that it has been sent via approved channels and third party email services such as Gmail and Outlook use this information to help determine whether an inbound message is likely to be genuine or not.

Confusion can arise because some services may not even classify suspect inbound messages as spam and may bounce them before they ever arrive at their destination. In our case, this was the reason why messages were not being found in the client's Outlook spam folder.

In the case of this client's shared hosting, the SPF records were automatically taken care of by the hosting company. Using cPanel we were able to view the records in the Advanced DNS Zone editor. From there we could see that SPF records were fairly strict and allowed email from related MX records plus a specified IP address. Initially, this seemed logical but we came to realise that this rule did not allow for Outlook also being used to send email for this client, quite independently of the clients's website and their hosting account. There had to be something else (perhaps another SPF record somewhere) and it turned out that my client's Outlook service was configured for the highest levels of spam protection by the system administrators at their head office.

To get around this I first tried installing the SMTP module. This is used to send website email via a specified smtp server instead of php mail. I can see how this would be a fitting solution in many cases, but this didn't help with my client's problem.

They are now making use of the Mandrill module to help ensure that messages are being delivered. The Mandrill service provides a number of other methods for authenticating email and once I enabled, contact form emails started immediately coming through to my client but they were being classed as spam. This was actually a big improvement, as it was now possible to configure some email spam rules to filter them out. This wasn't the end of our troubles, but things were getting better.

By the way. There is more to using Mandril than just installing the Drupal module and I advise researching the subject well. This video, although pretty long, proved to be invaluable in helping me to understand the issues.

As mentioned above, I eventually narrowed the problem down to an Enterprise scale installation of Office 365 Outlook which didn't allow for email being sent through other channels, such as a client's website, when it was sharing exactly the same domain. In fairness, it would not be practical for this client's service provider have knowledge of all other channels either.

I had to provide SPF records to acknowledge Outlook's primary role in the sending of this organisation's email whilst also advising it of others. I have since found similar issues for a different client using who is using Google Apps (Gmail). I guess that Outlook believed it owned the canonical MX or SPF record and had to be properly convinced that email was allowed to go via alternative channels.

Summed up; SPF records may need to acknowledge email being sent by third party services such as Outlook or Gmail as well by the website's domain or IP address for any email generated by site.

If there is anything I've missed or if I've got anything wrong, please chip in.

----
The official SPF documentation site is http://www.openspf.org/

The examples given below may not be right for you but should help with the general principles

Allow Server IP Address and Outlook - Office 365 customers (change the IP address for your own website)
v=spf1 +a +mx +ip4:95.142.159.2 include:spf.protection.outlook.com ~all

Allow Mandrill and Gmail - Google Apps customers
v=spf1 include:_spf.google.com include:spf.mandrillapp.com ~all

Allow Outlook, Mandrill and Gmail - For an organisation using both Outlook and Google Apps
v=spf1 include:spf.protection.outlook.com include:spf.mandrillapp.com include:_spf.google.com -all

This last example might sound unlikely, but consider a school using Office 365 for staff email, but Google Apps for Education and Google Classroom for its pupils.