FastMail.FM Status

Status (by robm at Tue Aug 12 14:10 UTC)
One of our IMAP servers is down. We’re investigating

Update (by robm at Tue Aug 12 14:28 UTC)
We’ve found the problem, and we’re about to reboot the machine.

As the problem is isolated to one machine, and we should be able to fix it shortly, we’re not going to fail services over to the replica machine.

We should have the server back up in about 10-15 minutes

Update (by robm at Tue Aug 12 14:46 UTC)
All services have been restored

Update (by robm at Tue Aug 12 14:53 UTC)
After restarting all the services, one of the web servers had a bad software revision on it, causing lots of errors like:

Fatal: GenerateHTML error: IMAP fetch of uids a,b,c failed – Unexpected response line

This has been fixed

Status (by robm at Fri Aug 8 00:32 UTC)
The problem that caused the network outage yesterday has been diagnosed, and we’re working with NYI to create a redundant uplink so it doesn’t happen again in the future. More details on our main blog.

http://blog.fastmail.fm/2008/08/08/network-outage-and-details/

See these forum threads here:

http://www.emaildiscussions.com/showthread.php?t=53569

http://www.emaildiscussions.com/showthread.php?t=53570

There’s a problem at NYI, and we don’t know exactly what’s wrong yet because we can’t contact anyone or get in to the machines.  Updates as soon as I know anything.

Bron.

(argh, sorry – I managed to “unpublished” this first time through.  I’ll publish it now!)

UPDATE: Melbourne 10:38pm.  I’m on the phone to NYI now.  They’re just checking our cabinet, and their network guy is on the case.  I’ll know more soon.

UPDATE 2: Melbourne 10:49.  We’ve been back up about 3 minutes now.  I’m running around testing our machines, all looks good so far.  It was a network issue at NYI.

UPDATE 3: Melbourne 11:00. And we dropped off the map again.  NYI knows.  Hopefully this time it will be a real fix.

UPDATE 4: Just go read the forum threads already, I’ll keep posting updates there.  We’re bouncing up and down like a yoyo. (yeah, the grumpy is starting to kick in now)

Status (by robm at Wed Aug 6 23:14 UTC)
We recently updated our SSL certificate as described on our main blog.

http://blog.fastmail.fm/2008/08/05/new-mailmessagingenginecom-ssl-certificate/

There’s been a few issues for a few users (mostly Eudora users). There’s now directions on how Eudora users can fix the certificate warning issue on the main blog post.

Status (by robm at Tue Aug 5 11:41 UTC)
Due to an error when installing some new machines, the system that allows the web interface to report the number of messages learnt in a users bayes database on the Options -> Spam/Virus protection screen was not working, and report 0 messages for users.

This has now been fixed.

During this time, the bayes databases were still being used for spam scanning, they just weren’t reporting the correct numbers in the web interface

Update (by robm at Wed Aug 6 00:40 UTC)
Ooops, spoke too soon. The fix that was rolled out wasn’t quite right, and still caused some "0 messages" count problems.

Should really be fixed now