3 – 7 July is a short week because ReedSmith observes the 4th of July holiday. The NOC does continue to work though because they monitor all 28 office locations worldwide for 24/7 operations. I only participated though from 5-7 July though I did make up 20 hours’ worth of work. I did learn a lot very fast though.
The NOC does a constant monitoring of systems, but also tasks and follow-ups on problems. The NOC also deals with numerous outside vendors for information storage, local site maintenance worldwide, and network traffic monitoring. They coordinate this by doing three or more teleconferencing calls and going through a list of different changes, updates, or even patches that happen across the network utilities. They reference who will be conducting the Environmental Change, who will oversee the change and what resources are needed for the job.
The NOC monitors most of these processes through email, so the traffic usually flows through Outlook Exchange. They have monitors reporting set up through Powershell commands to take the tolerance levels from Solarwinds Orion network monitor, BlackBerry management, Mimecast Network spam filtering, VMWare monitoring, and NETBotz sensors. These systems tell the NOC that tolerance levels are too high, too low or not reporting at all times. The NOC watches these levels for every system and location to ensure constant redundancy, traffic flow, and backups. I would average that an email comes in once every 30 seconds reporting about something on an easy day. On a day where there are troubles, there are numerous emails that come in for monitoring each 30 seconds, along with calls and sometimes face to face analyst conversations.