Credit: rawpixel.com @ Pexels
It is sometimes said that success breeds success. For a digital business well into a digital transformation, one question will start popping up: “Is the way we're doing things paying off?” The Business Review exists to answer that question.
The purpose of this review is to understand the connection between real-time digital operations and business outcomes. You’ll seek alignment across Engineering, IT, and Operations leadership on goals and expected outcomes, and then adjust your initiatives and investments to achieve those goals.
Even the most nimble organization with highly autonomous teams will need some time to show impact and change. That said, it is recommended that Business Reviews be conducted on a quarterly basis and aligned with the company’s fiscal calendar. The best time to run this review is between 2 and 4 weeks after quarter-end, which allows enough time to assemble the data after quarter-end, but also soon enough that the information is still fresh and relevant.
While the On-Call and Service Reviews had specific scope to certain teams, services, and departments, the Business Review has a simple scope: the entire digital business. This means every team, every technical service, and every digital product. This review is an opportunity for full alignment around a company’s business goals.
This review should cover all operations since the last Business Review, ideally the last 3 months.
This review requires a significant investment of time and focus, and for many organizations, a half-day will be required. For the largest organizations, you’ll need a full day, and for smaller organizations, 2-3 hours may be sufficient.
Meeting Owner & Attendees#
For an effective Business Review that focuses on impact to the business and investment into better business outcomes in the future, it is recommended that this review be sponsored by the CIO, CTO, or the most senior technology leader within the organization. While the meeting should be owned and run by this person, they will still need organizational and administrative support of many others, including department heads, program managers, analysts, and executive assistants, to ensure meeting time is used effectively.
This review meeting is typically larger, with one to two dozen participants at many companies. All department or divisional leaders across Engineering, IT, and Operations (such as most Directors and Vice Presidents) should be attending. Additional attendees often include the CISO, the CEO, and Product Management leadership. For organizations whose primary product is digital, or those born digital and/or smaller organizations, it is common for the CEO, COO, and CFO to attend. For larger organizations and/or those with a mix of digital and non-digital products, attendance from the CEO, COO, and CFO is less common.
The metrics that drive this review are focused on impact to the business, which is defined as the impact to your people (on-call responders), your business costs in time and dollars, and the impact to your customers and top-line success measures. The suggested metrics for this review include:
- Product availability against SLA (total and by business service)
- Time without major incident (total, by business service, and by technical service)
- Major incident count (total, by business service, and by technical service)
- Major incident duration (total, by business service, and by technical service)
- Responder cost in hours (total and by team)
- Responder cost in dollars (total and by team)
- Unique count or % of customers impacted by incidents (total and for each major incident)
- Dollars lost or at risk from incidents (total and for each major incident)
- List of all or most-impacting major incidents, with details on: duration, total response hours, total response cost, business service(s) impacted, and customers impacted
- The organization's business success metrics, such as performance in terms of revenue, new sales, usage, costs, and any other metric the organization uses to gauge overall business success
Conducting the Review#
The Business Review meeting is a highly focused activity, looking at business impact from digital operations. Generally, the meeting flow will go as follows:
- Distribute Business Review metric reports to all attendees ahead of time.
- This provides an opportunity for a pre-read and to collect any supplemental information that may be helpful
- Start meeting by reviewing business success metrics for the organization. Create a shared understanding of how the business did with its key performance indicators for the quarter.
- Review uptime and availability for the entire product portfolio, as well as each product.
- Go deeper into uptime and availability by reviewing the data on how many customers were impacted, how sales suffered, and/or the revenue put at risk (for subscription and service businesses).
- Understand the cost of response. How many total person hours went into major incident response, the cost of that time, and the impact on the business' ability to execute on their planned work and innovation targets.
- Identify the technical services that had the greatest contribution to underperforming customer availability or cost of response metrics.
- Starting with the major incidents that had the greatest customer and business impacts, review all or many major incidents from the quarter. Focus on the offending technical service and the historical stability, incident root cause, incident duration and resolution workflow, postmortem learnings, and status of postmortem follow-up items.
- Finally, use the last part of the meeting to review action items suggested throughout the meeting, create new action items, and ensure owners are assigned to each action item.
A successful Business Review meeting will allow you to answer most or all of these questions:
- Was it a good or bad quarter for our digital business?
- What was the performance against stated SLAs? What was the time without major incident across the organization?
- How many major incidents did the business experience? Is this figure trending up or down? Which business services suffered the most from incidents?
- How many customers were impacted by major incidents (unique count or percent)?
- How much revenue was lost or put at risk?
- Was customer impact equal across business services or are there specific business services that suffer more than others?
- How much does it typically cost us to resolve a major incident?
- What percentage of our time is spent responding to high-urgency incident alerts?
- What should we change across our digital operations to ensure that our customer and business impact metrics meet our goals for the next quarter?
- How should we adjust our investment and allocation of people, process, and technology to meet our business goals?
As with all Operational Reviews, the appropriate action to take will be highly dependent on what was learned in the review, how the organization runs, and current focus or goals. There are however a few general tips and common areas of focus for taking action after a business review.
Two of the most valuable things that the participants of the Business Review can do are to validate the importance of initiatives and redirect investments to initiatives that can improve business outcomes. Investment can come in many forms, but tend to be investments of people, process, or technology. Investments in people may mean changing their focus or hiring more. Investments in process may include training or program management. Investments in technology could mean bringing in new tools or finding ways to leverage existing tools for automation.
Whatever actions come from your Business Reviews, it is important that you align them each with one of the three business impacts: people, business costs, and customer impact. Finally, set goals and define how you’ll measure achievement of those goals.
Final Notes & Considerations#
The business review is different than the other reviews in a notable way: it relies heavily on the concept of business services and impact metrics that directly measure the results of incidents and operations on customers and top-line revenues. It may not be realistic to have this data readily available when you first start running these reviews, or to have impact metrics well aligned with business services....aka revenue generating or key customer experiences. Typically, only the highest maturity digital organizations achieve this level of alignment between sales/customer impact and service opeations. It may take a bit to get there, but it is possible and you can do it!
In closing, if your organization is conducting on-call and service reviews, there should be no surprises in this meeting. One of the goals of running operational reviews at different levels of the organization is to create alignment across all leaders on the impact of operations on business outcomes. Many of the attendees of this review will be leaders of their own monthly service reviews, of which some attendees will be sponsors of their own on-call reviews.