I had an email from one of my clients informing me that they had 4000 emails in their inbox which had suddenly appeared. There were more being added every minute. Typically this charity received 3 or 4 a week.
What happened?
It appeared that someone was directing spam messages through the website at a rate of 4 per minute. Analysis of the emails coming in was interesting. They all came in through a contact form which had several anti spam measures included which had been defeated. Perhaps this was their goal to write an automated program which defeated the forms anti spam measures and fill the inbox. Which is a sort of denial of service attack.
Every email was different. They allegedly came from different email addresses, and the payload in the message was also different. So spam filtering could not work on a sender address, or easily by looking for some keywords or domain names.
The message encouraged the viewer to click on an obfuscated link (the destination of the link is coded so you cannot tell where it is going to). The coding for each link was also different.
Meanwhile while looking at this the volume of emails had now hit 6000. I had seen this once before, but I guess it happens regularly, because much spam traffic is automatically generated through forms. The question was what to do?
Start by looking at the emails
Each email had a couple of common elements in it. There was a brief message followed by a varying number of: >>>>>>> then a web address (always different) and then a varying number of: <<<<<<<.
If you look in your hosting, or through your webmail account you can set up filters for incoming mail. The filters work at a very detailed level per email account. I had not used them before.
I logged into the email account via webmail and chose Settings and then Filters and set up a couple of filters.
In the example above you can see I have set two filters up. Lets look at them to see how they work, and what limitations they may present.
Filter 1 Looking for this sequence “>>>>>>>>>>”
All of the email had two sequences of greater and less than symbols to draw the viewers attention to a web address they wanted you to click on.
I set up a rule which says check all incoming email to this account and look in the body of the email. If you find “>>>>>>>>>>>>” as a sequence in this email move the message to the Spam folder. Here are the settings.
You can see from this layout that you could build a very complex filter and have lots of conditions that need to be met before deciding what to do with the email. I could have refined it further, but I was under pressure at the time to regain control of the account. So I created a second filter on the Web address in each message.
Filter 2 Catching a web address
In this case all of the messages had a different email address. They all started as https://www.google.com/url?….. followed by a series of random characters. This is a method of hiding the destination address. But because all of the messages contained this common sequence I could use it. Here is the second filter.
What happens now?
With these two filters set up any messages coming into the inbox that contained either “>>>>>>>>>>” or more in a sequence OR a message carrying “https://www.google.com/url?” were now automatically sent to the spam folder. While this might look like success it was not the whole story.
Messages like the one below were now going into the spam folder, all of them were being trapped. However the computer was still hitting the website 4 x a minute. While email was now usable, the problem had not gone away.
The hosting company keeps a log of activity on the website. It is not an endless log, but it will show you maybe the last 3-4 hrs of activity. I decided to take a look there. This would tell me if it was a network of computers from around the world (which has happened before too), or a single computer running some sort of automated program.
Here is an example log with an access to the contact form highlighted.
The access log shows that something(s) are constantly accessing the contact us form on this website. The time stamps of the log show as high as 5 time per minute. These accesses are coming from two sources: 138.199.27.215 and 138.199.18.148. When I did this the first time through I only saw one IP address.
In the hosting you can block accesses to the website by country, by IP address or by IP address range. The country block is not fool proof because people can appear through IP addresses which are not listed, or pop up through a VPN somewhere.
Where is the spammer located?
Can I block by country? I went to this website and entered the IP address to see what it came back with. https://www.iplocation.net/ It came back with the following information:
This tells me several things. There are actually several entries from various databases, I have shown 3 here. It tells me the computer is located in France, in Paris at Ile-de-France. It also tells me that the service provider is a UK based company called Datacamp Limited.
Block France and then the whole world!
I next went back into the hosting and looked at the filtering options. This client only has clients local to their area, and certainly in the UK. So there is probably no need to have this website visible in other countries. At least for a short period. So I first blocked France, then blocked everywhere apart from the UK.
This action did not immediately work. Possibly because the block is not instant. The logs showed continuous accesses to the site. There is another reason, just speculation; the account was being served through a CDN (Content Delivery Network) and it might take some time for the content to age and be replaced. I tried clearing the server side cache but it had no effect. So what now?
Block by IP address
As I knew the IP address of the computer sending the messages into the website, it is possible to block based on the IP address which I did.
That stopped some of the accesses but then another IP address popped up. I have been here before, and traced a network of compromised servers around the world that were sending spam several years ago. The new address was 138.199.18.148. So there were either two computers, or more likely one computer with two ports attached to the internet.
The second IP address is similar to the first (first two number are 138.199) and are also with the hosting company Datacamp Ltd.
There is another type of block where you can block a series of IP addresses on the basis that hosting companies will acquire blocks of IP addresses which are in sequence. I set up a filter in the hosting which blocks 65,535 addresses all of which start with 138.199.
This blocks any computer working on this range of addresses: 138.199.0.0 to 138.199.255.255.
Here are the blocks in the hosting control panel.
In this case the entry 138.199.* overrides the other two entries which were specific to two locations.
This prevents these computers from seeing the contact form. So if they cannot see it they cannot post data into it. The website is not visible to them.
There is a second log in the hosting which records errors. You can see that each time the automated computer attempts to access the form it receives a 403 response from the website which means “Forbidden”. Their access is denied.
At this point the automated spamming computer cannot reach the website and cannot send any messages through the contact form. But it is still trying. It is still trying to do this 3 days later. Someone has left it on to keep attempting to hit this website. So what else can you do?
Contact the Hosting Company
People connect to the internet through a service provider or hosting company. In this case I had strong evidence that the person doing this was accessing the internet through a connection provided by Datacamp Ltd. So I tried to reach out to Datacamp through their website: https://datacamp.co.uk/ The website has a single email address visible and a telephone number. I sent an email to datacamp@datacamp.co.uk to advise them of the problem and request that they shut down the connection. Nothing happened, no response. So then I phoned 020 3808 5949 and left my number. No reply. I then did some searches on Datacamp Ltd and found that it is a parent company for CDN77. I went on that site and got on a chat with a representative of that company. They gave me another email address: abuse@datacamp.co.uk which finally did get a response.
At the time of writing the offending computer in France is still trying to hit the website contact form. Tomorrow I will escalate it yet again.
Whose fault is it?
It is unlikely that we will ever know who was behind this. There was probably no malice or directed attack at this particular charity other than they were testing something. A similar case a few years ago involved around 20 computers that were around the world doing something similar. They were less sophisticated than this attack, because in this case it defeated some tests in a form. I also changed the tests by replacing some different questions and it was still able to defeat it. I will look for some alternative methods to keep automated machines out.
It may well be the case that the computer this process is running on is a highjacked server someone has taken over, or has infiltrated. The owner knows nothing about it, and the service provider has no relationship at all with the people behind the attack. We will never know.
Sadly dealing with this type of thing is a fact of life. It goes on all of the time. But if it happens to you, we have some tools available in the hosting to minimise any loss of service.
If anything else happens here, I will update this.
Update 15/04/21
We are now a week later after this was first reported. The senders IP address has changed twice over the past week, but the messages continue. If it had not been detected, this account would now have 50,000 junk emails in it!
Over the course of the past 5 days I have contacted the service provider’s security team, and they have passed the message on to their customer. Their customer is a VPN provider. People use VPNs to hide their identity or the source of the message. The computer that is sending the messages is an unmanaged system in a data centre which is subcontracted out to their client. It is their client’s customers, or even their customer’s customers causing the problem. Nothing can be traced. Frustrating isn’t it?
I have contacted Action Fraud, and it seems the only thing I can do short of taking a private legal action against the company under the Computer Misuse Act 1990 for denial of service. There is nothing anyone can do, even though this is obviously wrong. It is also outside of the area that OfCom is responsible for. I have been advised to raise a case with Action Fraud which I will probably with the amount of evidence I now have.
