For many charities they may independently buy hosting and build a website, or someone else builds it for them and they don’t take any more action. There is a flaw in this, not all website hosting works as smoothly as you may think it does, and as you will see in this article, sometimes things happen which will substantially break a site, ….. and you are unaware of it.
Over the past two weeks I have raised around 10 support cases with the hosting provider we are using. They have been brilliant over the past year, I don’t have any regrets moving to them, the sites are generally trouble free. Since we moved to them last year, I have raised 178 support cases on your behalf, you never knew that did you?
Having worked in Engineering in the past, I am a firm believer in carrying out maintenance and checking things are working, and automating as much as possible. The managed service I provide (incorporated in the £4/ month I charge) includes me checking each site every three months and making sure there are no problems. While this is a tedious task, it usually does not take too long. I update anything that is outstanding and make sure the site is running ok. the last step is I take a backup of the site and store it in on a local server. It would normally take around 10 minutes a site.
Until March 2018
In March the time per site has often been 30 minutes or more. I was due to back everything up again in March, I started going around websites and began to notice a pattern. Hence the 10 cases I have raised. Firstly don’t panic, your sites are all working ok from the outside, these are problems on the inside, and as of today, I have been around everything once, and am going around again to make sure the problem is resolved.
Updraftplus is a plugin we use on all of the sites, it is premium version with additional functions from the free version. It usually just works. What I found to varying degrees on sites was the following.
- Rather than between 4 and 6 current backups, I had 23 or more, in one case 100.
- In the worse case the backups were occupying 9GB of memory on the server for one account
- A message about wp-cron not working
- Some updates had not occurred
- In a small number of cases the backup mechanism was not working at all
- In 85% of cases the settings for Updraftplus were corrupted (hence the erratic behaviour)
How did this happen?
We have been struggling to figure out what has happened here to cause this. Updraftplus tell me that there are no cases like this, and I know that 15% of the sites were ok. So not likely it is a bug. Much of this points to something called wp-cron. I cannot be absolutely sure about this, but I think it is related.
What is wp-cron
Wp-cron is a script built into WordPress that executes on the website whenever a visitor arrives on your website. What it does is executes a list of outstanding tasks that need to be run at certain intervals. For example, every 10 minutes, every 30 minutes, every hour, once per day etc. It is a core part of WordPress.
I first started noticing a few problems with the wp-cron process around August or September of last year. It turned out that the hosting company was finding that the volume of requests to run wp-cron was overloading the processing time on the server. I can see how this might happen on a very high volume site, but all of the sites I host are small regional sites. What they did was to make a change to how wp-cron was handled, it would only execute the first task in the list. It would ignore the others. Next time someone visited a site it would execute one more. What this means is that for an infrequently visited site it may end up with a long queue of outstanding tasks. In some cases I have seen more than 20.
Technically this means I have to use another method to make sure that any housekeeping tasks (this is what wp-cron generally handles) get executed on the site. In many cases where I have seen this problem I have set up a task in the control panel to execute every 10-15 minutes and force wp-cron to clear it’s backlog.
The most concerning thing for me here, is that this change fundamentally affected many sites, and a work around has to be incorporated. I have not done this for all sites, only the ones exhibiting the problem. You as a user would not be aware of the problem. But it meant that automatic updates would not occur, and your site would progressively get larger (not a problem for us, but in the former hosting it would eventually cause the site to stop working).
Form is not working
Then I came across something else. Someone reported a form was not sending emails, I checked it. It looked as though it was, no error messages were returned, but I could see the destination email box was empty.
I raised another ticket. I was told there that malware was detected on the website in question and the email sending function (associated with the form) had been disabled. Well, if any malware is detected I want to know about it straight away.
The hosting company is in the process of introducing a scanning system that checks the servers for the presence of files containing a sequence of characters that may indicate a virus or malware. This scanning system was operating in the hosting and was unilaterally disabling the email function associated with forms. But nobody was telling me. I have received apologies, and they realise it may not have been a smart thing to do. In the first case while I was assured that it was a real threat, it turned out to be a signature for an intrusion attempt held in the database for the site. This is normal, this is what WordFence does. Technically this is referred to as a “False Positive” meaning that something has been detected as a problem, and actually it isn’t. If it had been detected in a file in the hosting that would be different.
Further dialogue and further tests revealed six other cases. All of these have been cleaned and are working now. In the latter six cases the problems were detected overnight and I took action that night, the following morning it was all sorted out.
While I think it is great that the hosting company is taking an initiative like this, it demonstrates a certain lack of regard for those using the hosting. We all assume it just works. When they start to play around with things, often things get broken. The last two hosting companies were the same. It is only when you are so deeply plugged into these systems do you realise something has changed. Our current hosting company unlike the last one, are much more open about it, and have diligently worked with me to resolve any problems.
How things are now?
At the time of writing I have worked through all accounts, all sites are working ok, and up to date. I am now doing a second pass and setting up Updraftplus backups to a new DropBox account. The full resolution to the problems I have found are not complete. An investigation is currently proceeding with Updraftplus, and another one is underway at our hosting company. Once that concludes I will provide an update.