It was a simple presentation given in early March 2020, basic but fundamentally to the point. How would the State manage business continuity in the time of COVID-19?
The answer was painfully obvious - we, the Office of Technology were not in a place to have thousands of State employees head home in a hurry, so never let a good crisis go to waste !!!
This is the type of situation where someone had to step up with a single vision, voice and drive the process quickly and across the entire agency. I knew that the agencies would be counting on the Office of Technology to deliver core capabilities so that they could focus on the mission requirements.
The "War-Room" process was effective and collaborative, and in 3 days we enabled multiple scenarios. Within 6 days from start new hardware was in place, and within 8 the teething issues were resolved.
This is the paper that Jared Linder (CIO at FSSA) and I wrote about the process in under 1200 words. However we did not get to really dig into the details or lessons learned.
Priority - from the CIO down these activities were considered to be and communicated as a top priority.
The most critical enabling factor was to have all the players in the room at once. I mean this horizontally and vertically. Horizontally the SME's from all the departments that had the tech know-how and vertically the Exec's that could make decisions.
There was also an element of luck, we had purchased a new set of firewalls for an alternate purpose that could be deployed as an upgrade to the VPN VERY quickly.
Recognition that we had to have a single source of communication to our constituents and customers to reduce confusion and speed resolution. We set up a single location for ALL help documents, FAQ's and other information. This was something that had been loosely talked about for a long time, but had never reached the point where it was a priority.
An interesting aside was that this process actually allowed us to clean up some technical debt by making decisions on lingering issues.
Some issues hadn't been resolved because the right folks (horizontal and vertical) were not in the room at the same time. (Specifically we cleaned up some lingering issues with MDM / VPN and remote access for external collaborators)
There were other technical debt issues resolved because NOW there was a priority for our customers to drive the resolution with their employees. Multi Factor Authentication (MFA) was required for ANY remote access this forced the registration of all remaining employees into Self Service Password reset & MFA.
Engaging the front line Help Desk folks in all sessions REALLY helped in a number of ways.
It put the end user perspective front and center and consequently drove the detail and types of documents.
We were also able to update Help Desk processes in real time with the appropriate documentation so that as users called in the HelpDesk could give them the appropriate guidance.
As new tickets came in for users to request remote services, we were able to pivot the battle plans and modify our priorities to support the most requested tools.
Customer communication - this was an interesting situation. One recurring message from our customers had been a consistent lack of communication and clarity from IOT. In this rapidly evolving situation it was clear that our customers needed to know that we were "on it" so that they could be thinking about their missions, not VPN accounts. Our customers communicated up their chain that we were working on supporting remote work.
At this point, no decisions at a State level had been made about remote work and I believe that this was the issue that had us told to "stop communicating with our customers".
So we ceased formal communication through official channels (Like MS Teams) and relied on personal networks, text and after hours meetings to keep key customers informed and solicit feedback. This was effective, and at the point where we were allowed to communicate "Formally" (a few days later) we already received feedback and refined our communications to be more effective.
Once we had the go ahead to communicate, we set up a recurring daily teams call, and were VERY proactive in publishing everything into the Teams Channel that our customers had access to. We took their feedback and were able to resolve issues (people/process or tech related) VERY quickly also.
Lessons learned ?
In a crisis, establish the priority, engage the right folks horizontally & vertically. Communicate, communicate oh and communicate. You'll be surprised what you can accomplish.
Photo by Austin Distel on Unsplash