[BB] Possible Bb Service interruptions SATURDAY 8am-11pm

Bb Users,

 

Blackboard is performing emergency maintenance to replace a network switch in response to the outage we experienced on July 13^th. That outage affected all of their hosted clients at their “Northern Virginia Data Center 2” facility.

 

The maintenance will begin at 8:00am and is expected to finish by 11:00pm. Blackboard will remain available, but periodic service interruptions may occur during that time.

 

They will be sending me updates, and if they provide anything beyond “We are still working…things are going well…” I will pass that information along.

 

I apologize for any inconvenience this may cause and the short notice.  If you would like to read Blackboard’s announcement for more details regarding what they are doing, please see below.

 

_____________________________________

JENNIFER CLISTER Blackboard System Manager University of Miami IAC 305-284-3949

 

 

 

 

 

 

 

------------------------------------------------------------------------------------------------------------

Tue 7/28/2009 5:39 PM

General Announcement:     Dear Valued Customer,

 

Subject: Emergency maintenance notification for Saturday, August 1st 2009 for Northern Virginia datacenter 2 (VA2)

 

Date:                                     Saturday, August 1st, 2009

Start time:                           08:00 US Eastern

Estimated end time:       22:00 US Eastern

 

Services:                              All clients at Northern Virginia Data Center 2 (VA2)

Type of work:                    Network Switch Replacement

Purpose of work:             After action to July 13th network event.

Impact of work:      

 

On Monday, July 13th, 2009, we experienced a data center-wide outage as a result of a failing network switch.  Through a coordinated effort with our vendor, we determined that a full switch replacement is the best course of action to increase reliability and stability at our Northern Virginia Data Center 2 (VA2). As this device is a part of our core network infrastructure, we require an extended emergency maintenance window to perform the hardware replacement.

 

This emergency maintenance is planned for Saturday, August 1st 2009, and the work will start at 08:00 hrs US Eastern and is expected to continue until 22:00 hrs US Eastern.  As a precaution, we advise that you inform your faculty and students to expect intermittent service interruptions during this maintenance window.  We will take every precaution to keep the work seamless and maintain system availability.

 

We understand that this is short notice and unfortunately we must perform this maintenance. 

[BB] Last Week's Bb Outage-explained

Bb Users,

 

For those who are interested in what happened to cause Blackboard to go down unexpectedly last Monday afternoon (7/13), here is a brief description:

 

Blackboard noticed some network slowdown, and while troubleshooting they realized that the core back end switches were not functioning properly. They brought out the vendor and their own team of engineers to stabilize the network – which they did. They are still uncertain what the root cause of the problem was, but as a precaution they are planning on replacing the switches during one of our scheduled maintenance windows (between 1am and 7am on some Saturday mornings).  They are still working on a specific action plan, and if you are interested in hearing the specifics when they send it along, please let me know.

 

There were MANY schools affected by this outage, and probably around another 30 or so (including us) that had to have their databases rebooted even after service was restored before we could access our courses.

 

Thank you again for your patience.

 

_____________________________________

JENNIFER CLISTER Blackboard System Manager University of Miami IAC 305-284-3949

 

Re: [BB] Status: UP

Blackboard Users,

Our Blackboard system was restored within the last hour or so, but access to individual courses was just restored. There were approximately 30 schools who were still experiencing database errors when attempting to access individual courses requiring those databases to be restarted one-by-one.

If you are still experiencing problems, please try closing and restarting your browser.

If you continue to experience problems, please contact the help desk.

If you have anything you would really like me to say to the people at Blackboard while I am at the Blackboard World conference this week, please send them to me, and I'll do my best to find someone to listen.

As always, thank you for your patience.

-Jennifer

__________________
JENNIFER CLISTER
Blackboard System Manager
University of Miami IAC
305-284-3949
________________________________________
From: UM Blackboard Announcements [BB@LISTSERV.MIAMI.EDU] On Behalf Of Vilberg, William R. [bill.vilberg@MIAMI.EDU]
Sent: Monday, July 13, 2009 4:01 PM
To: BB@LISTSERV.MIAMI.EDU
Subject: [BB] Status: UP

It appears that Blackboard is up again, after a 2 hour 30 minute
outage for us and 200 other schools. The BbWorld09 annual conference
starts tommorow, so Jennifer should come back with some interesting
stories because of this outage.

Bill Vilberg
[See http://three.sentenc.es]

[BB] Status: UP

It appears that Blackboard is up again, after a 2 hour 30 minute
outage for us and 200 other schools. The BbWorld09 annual conference
starts tommorow, so Jennifer should come back with some interesting
stories because of this outage.

Bill Vilberg
[See http://three.sentenc.es]

[BB] Bb status

When I just checked, our Blackboard system was still down. Other
Blackboard systems are coming up, one-by-one. The problem was,
apparently, caused by a lack of network connectivity at the site that
hosts our equipment. Hopefully we will be back up soon.

Bill Vilberg

[BB] Blackboard is down

Our Blackboard server went down today at approximately 1:30 PM.
We are currently working with Blackboard to restore service as soon as possible.

You will receive an email notification when service is restored.

Please note that this is a high priority issue as this is also affecting other universities' servers.

We apologize for any inconvenience,

Blackboard Help Desk
(305)284-3949<tel:(305)284-3949>