Wednesday, December 19, 2012

UPDATED: Cox e-mail system suffers massive failure

Cox outage almost resolved..."about 80% of e-mails have been delivered"...no cause given for outage...if you want a refund, ask for one
By Will Collette

UPDATE: for most local Cox customers, e-mail service was restored sometime Monday. Mine came back during late morning.

However, I am still getting e-mails that were sent during the weekend and don't know how long it will be before all the e-mails are sent. Cox's latest bulletin says 80% of backed up e-mails have now been delivered.

Cox has given no explanation for the massive, multi-state outage, other than their early statement saying it was NOT cyber-terrorism.

That leaves Cox's failure. You will NOT get a rebate for the lost service...unless you ask for it. I did, and Cox notified me that they will take $5 off my bill. If you want a rebate, send an e-mail to customer service (click here).


I know that many Progressive Charlestown readers use Cox as their internet service provider and have their e-mail addresses on cox.net. So do I.

It seems Cox's e-mail service over a multiple state region covering the East Coast and Midwest collapsed sometime after Friday midnight and Cox can't get it back up and running.

Cox residential e-mail users have the bizarre experience where they can send e-mails, but can't receive them. If you send someone on Cox (e.g. me) an e-mail, that message is stuck somewhere in the ether. You will not be told that your message did not go through.  Cox say that when it fixes this SNAFU, it will send all those e-mails on. Oh boy, I am so looking forward to several hundred messages arriving all at once, especially all those intriguing business opportunities from Nigeria and the Middle East.

Here is the latest pack of excuses from Cox, though there is no mention of how much of a refund, if any, they plan to give subscribers when they fix this issue. I certainly plan to ask for one. Here's their "explanation:"



Alert

Summary
Email Outage Update as of 3:30 pm, December 16th

Detail

First we want to acknowledge the impact of the residential email service outage on our customers. We know this experience is not consistent with our promise to deliver reliable products and services, and sincerely apologize for the frustration this has caused.

Here is what we know:

1.             We have isolated the cause to an issue within the email storage platform that supports Cox Residential Email in service areas in our Midwest and East Coast regions. This outage does not affect Cox Business customers.

2.             Cox technical teams are working around the clock with email platform vendors to restore service as quickly and safely as possible. Unfortunately, we do not expect services to be restored today.

3.             This outage is not the result of a cyber-attack or any form of cyber terrorism.

4.             Some customers have asked about a redundant backup system. Our enterprise infrastructure exceeds industry standards and includes multiple levels of support designed to ensure reliability. Unfortunately, this outage affected the primary and secondary failover (back-up) systems.

5.             Our email service is supported by multiple servers. As we successfully bring each server back online, customers on that server will be able to access their email.

6.             Currently, all messages that have been received since Friday are queued in our system and we anticipate delivering messages to customers on a rolling basis as we bring the platform back online.

7.             We cannot provide a definitive restoration time, but are expecting multiple phases, including when customers will be able to send and receive new messages as well as the delivery of currently queued emails.

8.             If you must send or receive an email during this outage, you may want to create a temporary new email account from a service such as Yahoo! or Hotmail.  This will not allow you to gain access to your cox.net account; however, you will be able to send and receive email while your cox.net email is inaccessible.

We want to reiterate our commitment to restore services to our customers as soon as possible. As we stated earlier today, the multiple components and processes that make up our email system require care, and some time, to bring them back online.

Impact by Residential Customer Scenario

·                     Customers who have and actively use a cox.net email account (username@cox.net).

o                  Customers attempting to access their cox.net email account through webmail (Classic and/or Enhanced Webmail) will not be able to access their messages and cannot send or receive email.

o                  Customers attempting to access their cox.net email account through an email client such as Outlook, Outlook Express, Thunderbird, iPhone and Android smartphones, or an iPad may be able to access messages received prior to Friday evening, however no new messages will be downloaded; they will receive an error message when attempting to retrieve new messages.  When cox.net email service is restored, any queued messages will be downloaded at that time.  They should be able to successfully send messages from their cox.net account.

·                     Customers who have and DO NOT actively use a cox.net email account (username@cox.net).

While their cox.net email account is impacted as described, they will not see any direct impact, as they do not use their account.

·                     Customers who have an email account from a service such as Yahoo!, or Hotmail.

These email accounts are not impacted by this issue.  Only cox.net email accounts (username@cox.net) are impacted by this outage.

·                     Customers who have created a new temporary email account from a service such as Yahoo!, or Hotmail since Friday evening as a result of this outage.

These email accounts are not impacted by this issue, however do not allow customers to gain access to their cox.net email account.  Only cox.net email accounts (username@cox.net) are impacted by this outage.

Product FAQs 

Q:  Are all Cox High Speed Internet Customers Impacted?

A:  No, only Residential Cox High Speed Internet Customers in the East and Midwest who have and use a cox.net email address (username@cox.net) are impacted.  Customers who use other email services such as Gmail or Yahoo! email will not see any impact to those services.

Q:  If someone is trying to send an email to my cox.net account, will they get an error message?  

A:  No, people who send email to your cox.net email account will not receive an error message; they will not know you are unable to receive emails.

Q:  What will happen to the emails that are being sent to my cox.net email address during this outage period?
A:  Currently, all incoming messages are queued in our system and we expect to deliver messages to customers on a rolling basis as we bring our system back online.  Full message delivery may take several days.

Q:  If I must send an email during this outage, what options do I have?  

A:  If you must send or receive an email during this outage, you may want to create a temporary new email account from a service such as Yahoo!, or Hotmail.  This will not allow you to gain access to your cox.net account, however you will be able to send and receive email while your cox.net email is inaccessible.