Business Review

Once your organization is consistently conducting On-Call Reviews and Service Reviews, that data can be used to conduct Business Reviews at the senior leadership level. For an organization to navigate a digital transformation initiative successfully, it’s necessary to examine if current investments are paying their desired dividends.

The purpose of this review is to understand the connection between real-time operations and business outcomes. The meeting seeks alignment across technical and business leadership on goals and expected outcomes. Initiatives and investments can then be adjusted to achieve those goals.

Cadence#

Even the most nimble organization with highly autonomous teams needs sufficient time to show measurable results when implementing change. Business Reviews should occur on a quarterly basis. Most organizations already do this in the form of Quarterly Business Reviews. This review should be one component of that occasion.

The best time to run this review is approximately 2-4 weeks after the end of the quarter. That allows enough time to assemble the data after quarter-end, but is also soon enough that the information is still fresh and relevant.

Scope#

While the On-Call and Service Reviews had specific scope to certain teams, services, and departments, the Business Review has a simple scope: the entire digital business. This means every team, every technical service, and every business service. This review is an opportunity to create full alignment around a company’s business goals.

This review should cover all operations since the last Business Review, ideally the last 3 months. It requires a significant investment of time and focus. For many organizations, a half-day may be required. For the largest organizations, a full day or more may be required while smaller organizations may find that a few hours will suffice.

Meeting Owner and Attendees#

An effective Business Review focuses on impact to the business and investment into better future business outcomes, from a technical perspective. We recommend that this review be sponsored by the CIO, CTO, or the most senior technology leader within the organization. The meeting should be owned and run by this person, but they will also need the organizational and administrative support of many others, including department heads, program managers, analysts, and executive assistants, to ensure the meeting is effective.

The review meeting can be quite large. It’s not uncommon for this meeting to have one to two dozen participants at many companies. All department or divisional leaders across Engineering, IT, and Operations (such as most Directors and Vice Presidents) should be attending. Additional attendees often include the CISO, the CEO, and Product Management leadership. For companies with technology products at their core or for smaller organizations, it is common for non-technical executives like the CEO, COO, and CFO to attend. In larger organizations and/or those that are less technologically centered, attendance from non–technical executives is less common.

Metrics#

The metrics that drive this review are focused on impact to the business, which is defined as the impact to your people (on-call responders), your business costs in time and dollars, and the impact to your customers and top-line success measures. The recommended metrics for this review include:

Product availability against SLA (total and by business service)
Time without major incident (total, by business service, and by technical service)
Major incident count (total, by business service, and by technical service)
Major incident duration (total, by business service, and by technical service)
Responder cost in hours (total and by team)
Responder cost in dollars (total and by team)
Unique count or % of customers impacted by incidents (total and for each major incident)
Dollars lost or at risk from incidents (total and for each major incident)
List of all or most-impacting major incidents, with details on: duration, total response hours, total response cost, business service(s) impacted, and customers impacted
The organization's business success metrics, such as performance in terms of revenue, new sales, usage, costs, and any other metric the organization uses to gauge overall business success

Conducting the Review#

Although lengthy, the Business Review meeting is a highly focused activity, looking at business impact from real-time operations. Generally speaking, the meeting should follow this type of agenda:

Distribute Business Review metric reports to all attendees ahead of time (this provides an opportunity to pre-read and collect any supplemental information that may be helpful).
Start meeting by reviewing business success metrics for the organization. Create a shared understanding of how the business did with its key performance indicators for the quarter.
Review uptime and availability for the entire product portfolio, as well as each service.
Go deeper into uptime and availability by reviewing the data on how many customers were impacted, how sales suffered, and/or the revenue put at risk (for subscription and service businesses).
Understand the cost of response: how many total person hours went into responding to a major incident, the cost of that time, and the impact on the business' ability to execute on their planned work and innovation targets.
Identify the technical services that had the greatest contribution to underperforming customer availability or cost of response metrics.
Starting with the major incidents that had the greatest customer and business impacts, review all or many major incidents from the quarter. Focus on the underlying technical service and its historical stability, contributing factors to the incident, incident duration, resolution workflow, postmortem learnings, and status of postmortem follow-up items.
Finally, use the last part of the meeting to review action items suggested throughout the meeting, create new action items, and ensure owners are assigned to each action item.

A successful Business Review meeting will allow you to answer most or all of these questions:

Was it a good or bad quarter for our digital business?
What was the performance against stated SLAs and SLOs? What was the time without major incidents across the organization?
How many major incidents did the business experience? Is this figure trending up or down? Which business services suffered the most from incidents?
How many customers were impacted by major incidents (unique count or percent)?
How much revenue was lost or put at risk?
Was customer impact equal across business services or are there specific business services that suffer more than others?
How much does it typically cost the company to resolve a major incident?
What percentage of company time is spent responding to high-urgency incident alerts?
What should be changed across digital operations to ensure that customer and business impact metrics meet company goals for the next quarter?
How should investments and allocations of people, process, and technology be adjusted to meet company business goals?

Taking Action#

As with all operational reviews, the appropriate action to take will be highly dependent on what was learned in the Business Review, organizational culture, current focus, and current goals. However, there are a few general tips and common areas of focus for taking action after a Business Review.

Two of the most valuable things that the participants of the Business Review can do are to validate the importance of existing initiatives and redirect investments to initiatives that can improve business outcomes.

Investment can be made many different ways, but they tend to apply to either people, process, or technology. Examples of investments in people might mean changing their focus or hiring more. Investments in the process might include training or program management. Investments in technology could mean bringing in new tools or finding ways to leverage existing tools for automation.

Whatever actions are identified during your Business Reviews, it is important that you align them each with one of the three business impacts: people, business costs, and customer impact. Finally, set goals and define how you’ll measure achievement of those goals.

Final Notes and Considerations#

The Business Review is different from the other operational reviews in a notable way: It relies heavily on the concept of business services and impact metrics that directly measure the results of incidents and operations on customers and top-line revenues.

It may not be possible for your teams to have this data readily available when you first start running Business Reviews. It may also be initially difficult to have impact metrics that are well aligned with business services (your revenue generating or key customer experiences). In practice, only the highest-maturity digital organizations achieve that level of alignment between sales/customer impact and service operations. It may take your organization a bit of time to get to that level. But it is possible and you can do it!

Again, if your organization is conducting On-Call Reviews and Service Reviews, there should be few to no surprises in this meeting. One of the goals of running operational reviews at different levels of the organization is to create alignment across all leaders on the impact of operations on business outcomes. Many of the attendees of this review will be leaders of their own monthly service reviews, of which some attendees will be sponsors of their own on-call reviews.