The following is a bit technical, but bear with me. It is useful to understand how some of the mail protection policies work and the impact they may have on your mail in some circumstances.
I have a large number of email accounts, I don’t tend to check them all, but consolidate them by setting up forwarding. So any mail coming into say myname@somedomain.co.uk is forwarded to myname@someotherdomain.co.uk. Then I only need to check one email account at “someotherdomain.co.uk“.
Up until a few weeks ago, I had this working ok within a single hosting company. I made some changes when a second hosting company came on line, and moved one of my accounts to hosting company #2. Then I started to see problems.
I am using a forwarder in one of my hosting accounts. This may be relevant to you if you are using forwarders in your hosting. I generally set up one to handle messages posted to your site. These are generally handled through a mail account called info@yourdomain.org.uk which may then be passed to your normal administrative email account which could be xxxx@btconnect.com or another service provider email.
In my case I have two mail accounts, one forwards incoming mails to a second account. Let’s call them account A and B. If account A forwards all incoming mail to account B, then account B will contain A + B messages. Said in another way, account B will contain all of the messages normally addressed to account B and all of the messages forwarded from account A.
I spotted some issues where some messages were not getting through, and worse still; some messages did not get any visible rejection notice to alert me to the fact that the messages had been rejected.
A common thread
I had just booked something on Expedia, but the message never made it to account B. It did make it to account A. So given that account A forwards all of the messages to account B, what went wrong?
On close inspection there were some domains where the messages did not get forwarded, in my case they were twitter.com, expedia.com and facebook.com. I also had some from local government sites that were not forwarded too.
I raised a case with technical support which bounced back and forth several times, they could not help, and they failed to understand the problem.
SPF is enabled
There is a feature called SPF with email set up. It stands for Sender Policy Framework. It is a feature which is designed to detect and prevent spoofing. This is where it is possible to send a message and claim to be obama@whitehouse.com, however a receiving system checks the server IP address of the senders domain name (in this case whitehouse.com) and finds out that it is not the same as the real senders IP address. The assumption is that because the message did not originate from the whitehouse.com server the message must be spoofed (fake) and therefore it gets rejected.
Local Government (UK) tend to use this feature to reduce spam and spoofed messages as do many large companies. In my case I found that Expedia, Twitter and Facebook also use this feature too. It is also worth pointing out that not everyone uses it.
Tracing the path
Having seen messages in account A, but not in account B, I then checked the mail in account A to see if any error messages were present. There were none. Therefore it was reasonable to assume that all messages were forwarded without any errors.
At the receiving end; through cPanel it is possible to check the mail path. I entered the receiving account email address and this created a log of messages that had been processed. The majority were received ok (99%), occassionally there was an error message.
I compared a message received ok to a message that had been rejected and found the following.
| Event: | rejected. This message was rejected at SMTP time by an RBL, filter, or other configuration. |
| User: | -remote- |
| Domain: | |
| Sender: | giffgaff@info.giffgaff.com |
| Sent Time: | May 14, 2015 3:29:15 PM |
| Sender Host: | res151.servconfig.com |
| Sender IP: | 173.247.240.261 |
| Authentication: | unauthorized |
| Spam Score: | |
| Recipient: | yyyyy@xxxxxxxxxxxxx.com |
| Delivery User: | wnptrm15 |
| Delivery Domain: | xxxxxxxxxxxxx.com |
| Delivered To: | |
| Router: | reject |
| Transport: | **rejected** |
| Out Time: | May 14, 2015 3:29:15 PM |
| ID: | 1Ysu96-0008pa-Zq |
| Delivery Host: | res151.servconfig.com |
| Delivery IP: | 173.247.240.261 |
| Size: | 0 bytes |
| Result: | SPF: 173.247.240.261 is not allowed to send mail from info.giffgaff.com |
If you are checking your mail go to cpanel for your account, and look for the mail trace icon under email and run a trace. This will tell you if everything that came into the account was accepted or not. Check anything that was rejected.
What does it mean?
The message above was rejected because it was sent to account A and then forwarded to account B. It was successfully sent to account A and then forwarded to account B but failed at that point. The reason it failed was because it was sent from giffgaff@info.giffgaff.com. At the point it was received by account B a check was made (SPF) to establish if the senders IP address matched the entry for the sender which was giffgaff.com. It was rejected because the IP address for the forwarding mail server did not match the IP address of the giffgaff.com mail server. This is not surprising because it is the forwarding mail box (account A) that sent the mail. This is what the the following entry is saying: SPF: 173.247.240.261 is not allowed to send mail from info.giffgaff.com
So why did it work before?
Before I made changes to the hosting, Both account A and B were in the same hosting company. Mail is checked at the boundary or edge of the data centre, in other words at the point it enters the hosting system. This is where it is tested against the SPF settings. If both accounts are within the same datacentre, then the mail passes through the edge of the datacentre on the way in, then is locally transferred to the forwarded account. That second transaction is not checked again against the SPF because the message is moved within the hosting provider. It is not tested a second time.
The change I made was to move the second account to another service provider. Now the forwarded mail is presented to the edge of the 2nd data centre because it had to be transferred across the internet to the second hosting company. At the edge the SPF is checked a second time, and is rejected.
So what does this mean to me?
If you suspect that mail is being dropped, perhaps because you are within a local government network (I know some of you are), then check the email trace facility in cPanel. If it is failing because the mail is being forwarded and then checked, consider cancelling the forwarding function and access the mail account directly.
Also bear in mind that SPF is there to protect you and the service provider. So you will find rejection notices for “real” spoofed email and spam. So check what has been rejected if you see any rejection notices.