Status (by robm at Sun Mar 28 06:46 UTC)
We’ve been experiencing a few problems with the truedomain milter that are causing it to delay some email. Because of these problems, we’ve temporarily disabled it until we can resolve the issues. This means that the headers the milter normally adds that protect email sent from domains such as facebook and linked in will not currently be appearing on emails.
Status (by robm at Mon Mar 15 01:33 UTC)
There are currently problems delivering SMS messages to US phone numbers. This is due to problems at our SMS gateway provider.
http://support.clickatell.com/us_notifications.php
Many US carriers allow you to send an SMS to a user just by sending an email to their email-to-SMS gateway address, so you can try using that instead.
http://en.wikipedia.org/wiki/List_of_carriers_providing_SMS_transit
Status (by robm at Sat Mar 13 02:34 UTC)
One imap server is down, affecting email access for a small number of users. We’re investigating
Update (by robm at Sat Mar 13 03:20 UTC)
We failed everything over to replicas while we get the server restored, so everyone should have access to email again
Status (by robm at Thu Mar 11 00:47 UTC)
One imap server is down, affecting email access for some users. We’re looking into it.
Update (by robm at Thu Mar 11 01:20 UTC)
All services have been restored
Status (by robm at Mon Mar 8 05:25 UTC)
The old web interface is currently maintained on 3 separate servers. Over the weekend, we re-installed one of those servers due to some data corruption that occured on it when it crashed. Unfortunately the re-install wasn’t 100% complete, and this caused some internal DNS failures. The result of this is that if you use the old web interface, and happened to get this problem machine as your web server (1 in 3 chance based on your IP), any sent email will have double bounced and disappeared. A copy will have been placed in your Sent Items folder, but the email was not actually sent.
We’ll work out which users were affected by this, and email them to let them know, and how they can resend their email.
This will only have affected some users (1 in 3 randomly based on your IP), only users that use the old web interface, and only users that have sent email in the last 21 hours.
Status (by robm at Sun Mar 7 03:07 UTC)
Users using the fastmail.es domain will have been seeing delayed and/or bounced email for the last 24 hours or so. More details about what happened are below.
On Friday and Saturday, we were going through all our domains updating the contact details for them to try and make them consistent. Since we have quite a few domains at many different registrars, this involved logging into lots of different registrars websites.
fastmail.es is registered at eurodns.com, it’s the only domain we have registered there.
Some background about their website: they have a concept of DNS profiles. This lets you set a standard set of nameserver and/or DNS records that you can apply to multiple domains. They have a "Eurodns default", and we’ve setup a "Messaging engine" one. Really annoyingly, you can’t delete the Eurodns default profile, even if you never want to use it.
For some reason their website is particularly stupid. When you edit the contact details for a domain, for some reason there’s also a menu to select and change the "DNS profile" to use for the domain at the same time. Nothing to do with contact information, so why this is there when you edit the contact info, I have no idea.
When I edited the contact details, for some damn reason, it reset the profile to the "Eurodns default" one, not the "Messaging engine" one that it’s been set to for years
I didn’t notice that till after I submitted the contact details change and saw that it was going to change the DNS profile as well.
Now when you change contact details, it’s not instant, instead it submits it into a "pending changes" queue. While there’s pending outstanding changes waiting for your domain, it won’t let you edit anything else. You’re stuck waiting for the change to go through even if you want to cancel it.
So I immediately submitted a support ticket with them to make sure they didn’t change the DNS profile, only the contact information.
It seems they ignored the support ticket, and went ahead and made the contact and DNS profile change about 24 hours later anyway. In fact they still haven’t responded to the support ticket, despite it being submitted on Friday 7:50AM CET, and the confirmation email I got about the support ticket said "Our office opening hours are Monday to Friday from 8AM to 12AM and 1PM to 5PM CET".
We’ve now submitted another change request, to set the DNS profile back to the "Messaging engine" one, but again, we have to wait for that pending request to go through their internal process.
In the meantime, I’ve updated the "Eurodns default" DNS profile (eg. the DNS their nameservers are generating), to try and set it closer to what we generate, namely that http://fastmail.es will go to our login screen, and email sent to *@fastmail.es and *@*.fastmail.es goes to our MX servers. I’m waiting for that DNS to propagate, which should happen in the next hour or two. That should mean that email will start flowing again properly.
I’m incredibly annoyed with eurodns, the interface is clearly silly, and the fact their support team hasn’t responded and didn’t do as asked even after 24 hours is unacceptable. Unfortunately the .es TLD isn’t supported by our main registrar (enom), and on the list of .es supported registrars (https://www.nic.es/listado-agentes/agenteRegistrador/1447) we have no existing relationship with any of them, so we’d be transferring one domain from one poor registrar to another unknown registrar.
Looking at the user list, this problem will have affected about 50 paid users, and 250 Guest accounts. For those affected, we’re really sorry that this happened. We’ve never had something like this happen before, and we’re really sorry that it did.
In 10 years, this is the first time anything like this has happened, and I’m really, really annoyed about it. I’m also really, really annoyed with eurodns.
For affected paid subscription accounts in the @fastmail.es domain, we’ll give you a 2 month extension on your subscription if you contact support as a way to say sorry for this happened. We know that reliable email is very important, and we always strive to make the most reliable service possible.
Status (by robm at Thu Mar 4 01:29 UTC)
The FastMail file storage system was unavailable for the last hour. This would have resulted in an error when trying to upload files, or when trying to add attachments to emails via the web interface. This has now been fixed.
