3 Steps to Ensuring Your AI Initiative Does Not Fail

Exhibit A: Gartner's Hype Cycle

Technically speaking, AI has been around since the 1950s. Today's business challenges warrant the commercialization of this technology and indeed, billions of dollars are being invested to capitalize on these opportunities. While the application of AI technologies to the practice of law is in its infancy, expectations are high and in some cases, it is a deep descent into a trough of disillusionment.

Much has been written about the various technologies and the hypothesized disruption on the horizon. The temptation is to become caught up in the thrill of innovation — the fascination with the technology — without clearly defining the business problem that it intends to solve.

We all want to be a hero. Saturday morning cartoons left us with the inherent desire to be the one to solve the unsolvable problem. The new era of AI-enabled technologies has the lure of being the magic wand to save the day. However, many initiatives involving the evaluation of emerging technology fail by design, or rather, lack of design. The technology can come across as a fantasy solution to an ill-defined problem. Or, an AI-initiative may be unable to secure funding because the value is not quantified in tangible business terms that matter to the decision makers.

By properly defining the business need, developing a compelling business case, and conducting an effective proof of concept for an AI related initiative, the slope to realistic application of the technology becomes more gradual.

Step 1: Define the Business Need(s)

HBR Consulting (HBR) co-hosted an event in collaboration with The Cowen Group in December 2016 focusing on exploring how AI is being applied in the legal industry. More than 100 professionals from corporate law departments, law firms and service providers attended. During the event, a survey was conducted to gauge the barriers to investing in AI. The greatest barriers identified were: a lack of defined business need and an unclear understanding of the benefits to the business.

At its essence, defining the business need is a simple equation: we need X to achieve Y as measured by Z.

What is the 'Y?'

AI-enabled technology has been cast to purportedly address a variety of applications, underscoring the need to clearly define the business problem to be solved. A few of the represented uses include: contract due diligence, legal research, outcome prediction, expertise automation, intellectual property, virtual legal assistants, contract review, practice automation and the ever-popular e-discovery document review. The wide range of AI use cases provides even more reason to clearly define and articulate the specific “Y” of a proposed initiative.

To define the business need addressed by the proposed investment, first identify the beneficiaries: clients, attorneys, staff or the overall organization, and the anticipated benefits received.

• Will clients benefit? If so, how? E.g., increased value, faster responsiveness, greater efficiency and cost savings.
• Will attorneys and staff benefit? If so, how? E.g., increased job satisfaction, attract talent in recruiting, ability to focus on higher value activities.
• Will the organization benefit? If so, how? E.g., improved profitability (increased market share through differentiation, opportunities to establish additional lines of business revenue, cost containment by accomplishing more with less resources (law department) or improved law firm profit margins).

As we know, law firms are in the business of providing legal services to clients with the objective of providing highly valued services while also generating profit. Separately, law departments were established to solve legal issues for businesses in a more cost-effective manner. To garner support for an investment of time, money and resources, the proposed outcome needs to align with the respective business objective.

When proposing an initiative to evaluate the feasibility of investing in an AI-enabled technology, it is critical to prove the anticipated value of the technology. Once the need is clearly defined, a business case needs to be developed to justify the investment.

Step 2: Create the Business Case

The purpose of a business case is to justify the investment. To do so, address the identified business need(s) via the proposed solution, quantify the anticipated benefits and define the initiative's parameters to secure approval.

Problem Statement

The preface should clearly state the problem that will be solved in a problem statement. A problem statement is a concise description of the business need(s) to be addressed by the initiative.

Here are some examples:

• We propose investing in (X) AI-enabled technology in the areas of litigation and M&A to improve realization, increase profit margins and deliver results to clients faster (Y).
• We propose evaluating the application of (X) AI-enabled technology to the process of generating routine contracts to reduce turnaround time to the business units and focus the department's attorneys on higher-value activities (Y).
• We propose conducting a pilot project of (X) AI-enabled technology with the employment practice to deliver faster results to clients, improve profitability and increase attorney job satisfaction by automating the process to provide guidance for routine employment related inquiries (Y).

The problem statement should be supported by background information to provide context (e.g., metrics from prior cases that demonstrate the need).

Measuring the Z: Quantify the Problem and the Benefit of the Solution

The benefit of the investment is best demonstrated by showing how the outcome will result in value to the organization in ways that align with the overall strategic goals. In other words, is the justification to spend the buck, to make a buck, save a buck or mitigate risk?

Opportunities to quantify the value should be a natural consequence of defining the business need(s) and crafting the problem statement. The approach to measure the benefit(s) needs to be defined and articulated early in the process to justify the value and to inform the scope of the initiative. For example, if the contemplated scope of the proposed initiative is to conduct a proof of concept applied to one case, is it accurate to extrapolate the results to all matters? Probably not. In this example, the scope should be adjusted to afford the opportunity to gather metrics from enough representative matters to draw an accurate estimation of the value to be provided.

When quantifying the benefits, state them in measurable outcomes. It is also useful to align the anticipated benefits to the organization's strategic goals, firm culture and client expectations.

Here is an example:

We propose conducting a pilot of (X) AI-enabled technology with the employment practice to deliver faster results to clients, improve profitability and increase attorney job satisfaction by automating the process to provide guidance for routine employment related inquiries. It is anticipated that the cycle time for responding to routine inquiries will be reduced by 25%, realization will be improved by 5%, resulting in an additional recovery of ~$1M in fees annually, and client and attorney satisfaction will be improved.

To validate the anticipated benefits, we will: 1) Measure the turnaround time in the complete lifecycle of similar matters to measure how much and if the application of technology resulted in faster delivery of results; 2) Examine the reduction in activities that typically are subject to write-off or write-downs, contrasted against other similar matters; 3) Evaluate the cost of delivering the services in matters applying the technology, against similar matters addressed without the technology to estimate value to the firm from the efficiency gains and measure the profit margin; and 4) Elicit feedback from the attorneys and their clients to gauge the level of satisfaction with the new process(es).

Ultimately, the purpose of quantifying the benefit is to make the case for investing the proposed time, resources or money. However, it is just as important to measure the cost of not undertaking the initiative, which should be stated as well. This statement should be illustrated by quantifying the effect of maintaining the status quo.

For example:

The volume of routine employment-related inquiries from the business units has increased 4x over the past 36 months. If this trend continues, by 2019 we will need to add two additional employment attorneys to the team to handle the work resulting in at least $600k/year of additional overhead. Those monies could be allocated to provide higher-value legal guidance with the efficiency gains anticipated by applying AI-enabled technology to routine transactions.

Assuming the business case was effective in garnering support to pursue investigation into an AI-enabled technology solution, the best way to break a fall into the “trough of disillusionment” is to conduct a proof of concept.

Step 3: Conduct an Effective Proof of Concept

A proof of concept (POC) is a contained test conducted to decide the feasibility of applying an approach to technology as a way to solve a business need. The POC should serve to validate or disprove hypotheses related to the value expected as compared to the representations made during a sales process or when crafting the business case. Additionally, a critical purpose of the POC is to gauge the level of effort, and the requirements to effectively implement and support the technology long term.

POCs sound like a fun idea on paper but well-intended initiatives often never finish or do not provide the right evidence to inform a buying decision. Oftentimes, participants lose steam and go back to their “day jobs” with a sense of discontent over the POC process. However, a well-structured POC plan with clear objectives, roles and responsibilities, testing scenarios, and pre-defined success criteria can mitigate project fatigue.

A few critical success factors to consider when developing a POC plan:

• Draft a POC plan with a pen, on paper (seriously);
• State the objective and business problems to be solved;
• Identify a contained set of scenarios specifically aligned to the business problems to test;
• Clearly define the roles and responsibilities between the vendor, law firm / law department and specific project team members;
• Limit the involvement of the most time-constrained participants to testing specific scenarios (or parts of specific scenarios);
• Develop the criteria and processes to measure the results in advance (e.g., time spent on specific activities, qualitative results and quantitative evaluation aligned to the business case);
• Articulate both the success criteria in advance and the fatal criteria warranting the end of the POC;
• Document the results of the testing as they are performed and log issues / interactions with the provider;
• Evaluate and document the requirements to support the proposed solution long term (e.g., resources, expertise, IT infrastructure and support);
• Most importantly, define the finish line (e.g., the POC is complete when these four scenarios have been tested) within a contained time. Do not spend more than the suggested 45-60 days; and
• Lastly, document the outcome in a POC Report (on paper).

Armed with the knowledge gained during the POC, hypotheses can be validated, adjusted or disproven. The results should guide an eyes-wide-open decision on whether to move forward with the product, test another alternative or find another approach to address the business solution.

Keys to Success

Despite the articles postulating that legal professionals are afraid of being replaced by robots, we found the opposite in our survey. Our results showed that 100% of participants from law firms and in-house law departments are inspired, optimistic or encouraged by AI's impact on their careers. Alas, the opportunity exists to be a hero; to be the change agent that leads the organization to great gains.

However, the surest way to be perceived as the opposite of a hero to your organization's leadership is to waste precious time, money or political capital on technology experiments.

By properly identifying the business opportunity that provides meaningful value for an AI-related initiative, developing a compelling business case, and conducting an effective proof of concept, you may just get that cape after all.

***** Bobbi Basile is a managing director at HBR Consulting. She advises law departments on how to support, protect and further corporate business objectives through the alignment of teams, processes and tools in the area of law department operations, information and records management, and electronic discovery.