Duel of the e-Discovery Dollars: Cloud vs. Appliance

We constructed a study of a hypothetical, but typical, e-discovery client ' a large law firm with a mix of large and small cases ' and analyzed the total costs over a three-year span, using either a cloud or an in-house e-discovery platform. The outcome was dramatic: Using our most conservative figures, the cloud produced cost savings of 36% ' $2.3 million ' over in-house.

Methodology

For our hypothetical client, we assumed that the law firm is managing 200 small cases of 25 GB each and 25 large cases of 200 GB each over a span of three years. That is a total of 10 TB of data, but since it is rare for all the data in a case to arrive at once, we spread that over the three years ' 3,333 GB per year.

We also assumed that the total data would be culled at a rate of 67% ' the average rate reported by a recent industry survey ' bringing the annual quantity of data to 1,100 GB after culling. We further assumed a maximum of 500 users on the system (attorneys, reviewers, project managers, etc.).

To establish the expenses to build the TCO model, we did the following:

• Selected popular in-house processing and hosting platforms that are widely available on the market today to compare against a typical cloud-based platform.
• Obtained actual quotations from hardware and software suppliers.
• Calculated annual hardware and software maintenance fees at 20% of the up-front capital expenditures.
• Accounted for technology refresh by giving hardware a three-year useful life, which is the typical schedule in the IT industry.
• Excluded full redundancy for the in-house platform.
• Excluded business impact due to downtime because it varies too greatly from company to company.

This last item is significant. Server downtime is a real risk businesses face. If downtime costs are included, then the cost-effectiveness of an on-demand, cloud-based service is even more dramatic.

Cost Factors

To perform our cost comparison, we examined the following expenses:

• Up-front costs, which include the initial procurement costs such as hardware equipment and software licenses.
• Other one-time fees, which consisted primarily of processing by the cloud provider to ingest and cull ESI.
• Recurring fees, which include monthly hosting charges, annual software subscription fees and annual hardware maintenance fees for upgrades and technical support.
• Ongoing operational expenses, which include the costs of data center co-location, point-to-point connectivity, office real estate and staff to support an in-house appliance.

Let's look at these one by one.

Up-front Costs

One recognized advantage of the cloud is the absence of start-up costs. Because the cloud provider hosts and maintains the application on its own servers, no up-front investment is required for hardware and installation. Table 1 below illustrates this.

[IMGCAP(1)]

For our analysis, servers and storage were configured to meet the specification requirements of the selected processing and hosting platforms. Servers were configured to fulfill Web/application, processing, search, analytics and database roles.

Other One-Time Fees

Other one-time fees consist primarily of site setup, processing and productions. Table 2 below shows our estimates of typical fees.

[IMGCAP(2)]

For the cloud platform, the site setup fee includes site consultation, instructor-led Web training and setting up standard fields, review forms, dynamic folders and user accounts.

The processing fee includes ingestion ' the extraction of metadata, text and natives files ' and culling ' filtering the data via de-NISTing, deduplication, filetype filtering and date filtering.

There would be no processing fees for the in-house platform because the equipment costs and software licensing are accounted for in other expense categories.

Recurring Fees

Although both cloud and in-house applications involve recurring fees, they differ widely in nature, as Figure 3 below shows.

[IMGCAP(3)]

The in-house appliance would incur annual recurring fees relating to hardware maintenance and software subscriptions associated with the processing and hosting platforms. Again, for the purpose of this case study, we have selected popular processing and hosting platforms that are widely available on the market today. We have selected platforms that are capable of processing and hosting 10 TB of data over a three-year period. To keep it simple, we have calculated the hardware maintenance fees, entitling the buyer to upgrades and technical support, at 20% of the up-front capital expenditures.

For the cloud-based application, there are no maintenance or licensing fees. There would be a recurring monthly hosting fee, charged by the GB. Assuming that the cull rate is 67%, then the data being hosted is 1,100 GB the first year, 2,200 GB the second year and 3,300 GB the third year.

Ongoing Operating Expenses

Just as the cloud platform required no up-front costs, it also requires no ongoing operational expenses. The same cannot be said for the in-house platform, as Figure 4 below illustrates.

[IMGCAP(4)]

The ongoing operational expenses required to support the in-house platform include:

• Data center co-location to in-house hardware equipment and provide redundancies in power, cooling and 24x7x365 manned security versus an on-premise server room.
• Point-to-point connectivity between the data center co-location and office. Due to very high traffic volumes with processing and hosting ESI, we have factored in a dedicated GigE link offering speeds up to 1000Mbps.
• Real estate cost for staff office space. We have estimated the real estate space to be 2,000 square feet at $30 per square foot annually to accommodate a staff of seven.
• IT staff includes one network administrator, one help desk analyst and one database administrator to manage and maintain the infrastructure. We have also included one programmer to assist with customization projects.
• e-Discovery staff includes one e-discovery manager and three e-discovery analysts to support the in-house appliance. We have budgeted for three project managers in the first year, five in the second year and eight in the third.

(To set the salaries for e-discovery staff, we used the average salaries identified by The Cowen Group in its 2011 salary survey of law firm litigation support staff. For IT salaries, we used data from CBSalary.com.)

Summing Up the Total Cost

When the total costs over the three years are added up, the final cost for the cloud platform is $4 million versus $6.3 million for the in-house platform. That represents a savings of 36% with the cloud platform. Figure 5 below summarizes the numbers.

[IMGCAP(5)]

Thirty-six percent cost savings using the cloud over an in-house appliance is clearly dramatic ' and probably far greater than many would have expected. A further advantage of the cloud that these numbers do not show is that, not only does it require no up-front capital investment, but it also provides the flexibility to quickly ramp-up when activity increases and terminate costs when the project is finished. With an in-house platform, operating expenses continue, regardless of the level of activity, and there is constant worry about the investment becoming an idle money pit.

Intangible Cost Considerations

Not every cost associated with an e-discovery platform is capable of precise calculation. There are intangible considerations that range from the reputation and stability of the vendor to the security of the infrastructure to the defensibility of the process. Our TCO analysis could not factor in these intangibles, but they should be considered whenever selecting a platform.

You cannot put a price tag on experience, for example. But the track record of the vendor should be a key consideration. Further, there is value in selecting a vendor that has an intimate knowledge of a proprietary application versus one that claims to be a jack-of-all-trades.

There is also value in the consulting services the vendor provides and in the depth of experience of its consultants. Ideally, a vendor should provide project and technical consultants who have measurable project-management and technical expertise. The vendor should also provide consultants with expertise in predictive coding, analytics and computer-assisted search.

Another intangible to consider in a vendor is research and development. The ideal vendor is one that is constantly engaged in pushing its technology to the next level. A key advantage of cloud providers is that improvements to the software are rolled out in a manner that is seamless to the user. No upgrades or reinstallations are ever required.

Two key factors to take into account when comparing cloud applications versus locally installed appliances are infrastructure and security. Reputable cloud providers invest heavily in infrastructure
and security.

The Slippery Issue of Review Efficiency

One of the most significant variables in the overall cost of e-discovery is review ' that process of reviewing documents and tagging them as responsive or privileged. Our TCO analysis omits review because it is difficult to measure with precision. However, there is evidence that the right platform can improve the speed and accuracy of review and therefore lower its cost.

The problem with measuring and comparing review rates is that so many variables affect the calculation. In addition to variations in document formats and languages, other factors that affect review
rates include:

• The experience of the reviewers.
• The reviewers' familiarity with the material.
• The technologies used to search and organize the material prior to eyes-on review.
• The supervision, quality control, and feedback mechanisms being used.

That said, industry metrics do exist. Among the most commonly cited are the findings of the Text Retrieval Conference (TREC) Legal Track. Its studies in 2006 and 2007 found that the average rate for relevance review was 20-25 documents per hour. (The TREC studies can be found at http://1.usa.gov/TjwfVm and http://1.usa.gov/ToWmdO, respectively.) A 2011 law review article concluded that the industry average was 50 documents per hour. See, “Accounting for the Costs of Electronic Discovery,” by David Degnan, Minnesota Journal of Law, Science & Technology, Vol. 12, Issue 1, http://bit.ly/UqiAPH.

By contrast, an independent analysis of review rates using Catalyst's platform concluded that reviewers achieved average rates of 142 documents per hour for responsiveness review and 76 documents per hour for privilege review. Needless to say, this could add up to significant cost savings in a large-scale review.

Defensibility and Legal Ethics

Two other intangible but essential considerations for legal professionals are defensibility and professional responsibility.

Defensibility is not an easy concept to define, let alone measure. In general terms, defensibility involves any aspect of e-discovery that a lawyer may have to defend in front of a judge. A failure of defensibility can have severe consequences, from waiver of attorney-client privilege to steep monetary or procedural sanctions.

Your choice of platform can bolster or weaken the defensibility of your process. A platform should provide robust tracking and reporting capabilities, so that you can oversee and document the steps you take. It should provide for collaboration, redundancy and quality control, so that the possibility of a misstep is minimized.

With regard to professional responsibility, a lawyer has an ethical duty to ensure the confidentiality and security of client information. Ethical opinions make clear that this duty extends to technology. Before using any platform for e-discovery, cloud or local, the lawyer should take steps to ensure that the platform and the vendor meet the high standards that ethics rules demand.

Conclusion

As a long-time trial lawyer, I've learned to take evidence over opinions any day. While we often hear it said that bringing e-discovery in-house saves money, the facts show otherwise. When you look at the total cost of ownership of cloud versus appliance platforms ' when you factor in all the required and ancillary costs ' the cloud wins hands down.

To many e-discovery professionals, the debate over cloud versus appliance is akin to those over Mac versus PC or Coke versus Pepsi. Each side has its diehard advocates, whose loyalties are often grounded more on habit than on facts.

Among those who favor appliance-based e-discovery applications, one of the most common arguments is, “By bringing e-discovery in-house, we will reduce our costs.” As often as I've heard that assertion, I've never seen hard evidence to back it up. So I decided to gather the facts for myself, with help from several e-discovery experts and analysts.

We examined the total cost of ownership (TCO) of cloud-based and appliance-based e-discovery platforms to see how they compared. While TCO is a well-studied discipline within IT generally, it is seldom applied to e-discovery. This is surprising, given that one cannot accurately evaluate the cost of an application without laying out all of the expenses required to support it, including infrastructure, technology, staff and ongoing operational expenses.

We constructed a study of a hypothetical, but typical, e-discovery client ' a large law firm with a mix of large and small cases ' and analyzed the total costs over a three-year span, using either a cloud or an in-house e-discovery platform. The outcome was dramatic: Using our most conservative figures, the cloud produced cost savings of 36% ' $2.3 million ' over in-house.

Methodology

For our hypothetical client, we assumed that the law firm is managing 200 small cases of 25 GB each and 25 large cases of 200 GB each over a span of three years. That is a total of 10 TB of data, but since it is rare for all the data in a case to arrive at once, we spread that over the three years ' 3,333 GB per year.

We also assumed that the total data would be culled at a rate of 67% ' the average rate reported by a recent industry survey ' bringing the annual quantity of data to 1,100 GB after culling. We further assumed a maximum of 500 users on the system (attorneys, reviewers, project managers, etc.).

To establish the expenses to build the TCO model, we did the following:

• Selected popular in-house processing and hosting platforms that are widely available on the market today to compare against a typical cloud-based platform.
• Obtained actual quotations from hardware and software suppliers.
• Calculated annual hardware and software maintenance fees at 20% of the up-front capital expenditures.
• Accounted for technology refresh by giving hardware a three-year useful life, which is the typical schedule in the IT industry.
• Excluded full redundancy for the in-house platform.
• Excluded business impact due to downtime because it varies too greatly from company to company.

This last item is significant. Server downtime is a real risk businesses face. If downtime costs are included, then the cost-effectiveness of an on-demand, cloud-based service is even more dramatic.

Cost Factors

To perform our cost comparison, we examined the following expenses:

• Up-front costs, which include the initial procurement costs such as hardware equipment and software licenses.
• Other one-time fees, which consisted primarily of processing by the cloud provider to ingest and cull ESI.
• Recurring fees, which include monthly hosting charges, annual software subscription fees and annual hardware maintenance fees for upgrades and technical support.
• Ongoing operational expenses, which include the costs of data center co-location, point-to-point connectivity, office real estate and staff to support an in-house appliance.

Let's look at these one by one.

Up-front Costs

One recognized advantage of the cloud is the absence of start-up costs. Because the cloud provider hosts and maintains the application on its own servers, no up-front investment is required for hardware and installation. Table 1 below illustrates this.

[IMGCAP(1)]

For our analysis, servers and storage were configured to meet the specification requirements of the selected processing and hosting platforms. Servers were configured to fulfill Web/application, processing, search, analytics and database roles.

Other One-Time Fees

Other one-time fees consist primarily of site setup, processing and productions. Table 2 below shows our estimates of typical fees.

[IMGCAP(2)]

For the cloud platform, the site setup fee includes site consultation, instructor-led Web training and setting up standard fields, review forms, dynamic folders and user accounts.

The processing fee includes ingestion ' the extraction of metadata, text and natives files ' and culling ' filtering the data via de-NISTing, deduplication, filetype filtering and date filtering.

There would be no processing fees for the in-house platform because the equipment costs and software licensing are accounted for in other expense categories.

Recurring Fees

Although both cloud and in-house applications involve recurring fees, they differ widely in nature, as Figure 3 below shows.

[IMGCAP(3)]

The in-house appliance would incur annual recurring fees relating to hardware maintenance and software subscriptions associated with the processing and hosting platforms. Again, for the purpose of this case study, we have selected popular processing and hosting platforms that are widely available on the market today. We have selected platforms that are capable of processing and hosting 10 TB of data over a three-year period. To keep it simple, we have calculated the hardware maintenance fees, entitling the buyer to upgrades and technical support, at 20% of the up-front capital expenditures.

For the cloud-based application, there are no maintenance or licensing fees. There would be a recurring monthly hosting fee, charged by the GB. Assuming that the cull rate is 67%, then the data being hosted is 1,100 GB the first year, 2,200 GB the second year and 3,300 GB the third year.

Ongoing Operating Expenses

Just as the cloud platform required no up-front costs, it also requires no ongoing operational expenses. The same cannot be said for the in-house platform, as Figure 4 below illustrates.

[IMGCAP(4)]

The ongoing operational expenses required to support the in-house platform include:

• Data center co-location to in-house hardware equipment and provide redundancies in power, cooling and 24x7x365 manned security versus an on-premise server room.
• Point-to-point connectivity between the data center co-location and office. Due to very high traffic volumes with processing and hosting ESI, we have factored in a dedicated GigE link offering speeds up to 1000Mbps.
• Real estate cost for staff office space. We have estimated the real estate space to be 2,000 square feet at $30 per square foot annually to accommodate a staff of seven.
• IT staff includes one network administrator, one help desk analyst and one database administrator to manage and maintain the infrastructure. We have also included one programmer to assist with customization projects.
• e-Discovery staff includes one e-discovery manager and three e-discovery analysts to support the in-house appliance. We have budgeted for three project managers in the first year, five in the second year and eight in the third.

(To set the salaries for e-discovery staff, we used the average salaries identified by The Cowen Group in its 2011 salary survey of law firm litigation support staff. For IT salaries, we used data from CBSalary.com.)

Summing Up the Total Cost

When the total costs over the three years are added up, the final cost for the cloud platform is $4 million versus $6.3 million for the in-house platform. That represents a savings of 36% with the cloud platform. Figure 5 below summarizes the numbers.

[IMGCAP(5)]

Thirty-six percent cost savings using the cloud over an in-house appliance is clearly dramatic ' and probably far greater than many would have expected. A further advantage of the cloud that these numbers do not show is that, not only does it require no up-front capital investment, but it also provides the flexibility to quickly ramp-up when activity increases and terminate costs when the project is finished. With an in-house platform, operating expenses continue, regardless of the level of activity, and there is constant worry about the investment becoming an idle money pit.

Intangible Cost Considerations

Not every cost associated with an e-discovery platform is capable of precise calculation. There are intangible considerations that range from the reputation and stability of the vendor to the security of the infrastructure to the defensibility of the process. Our TCO analysis could not factor in these intangibles, but they should be considered whenever selecting a platform.

You cannot put a price tag on experience, for example. But the track record of the vendor should be a key consideration. Further, there is value in selecting a vendor that has an intimate knowledge of a proprietary application versus one that claims to be a jack-of-all-trades.

There is also value in the consulting services the vendor provides and in the depth of experience of its consultants. Ideally, a vendor should provide project and technical consultants who have measurable project-management and technical expertise. The vendor should also provide consultants with expertise in predictive coding, analytics and computer-assisted search.

Another intangible to consider in a vendor is research and development. The ideal vendor is one that is constantly engaged in pushing its technology to the next level. A key advantage of cloud providers is that improvements to the software are rolled out in a manner that is seamless to the user. No upgrades or reinstallations are ever required.

Two key factors to take into account when comparing cloud applications versus locally installed appliances are infrastructure and security. Reputable cloud providers invest heavily in infrastructure
and security.

The Slippery Issue of Review Efficiency

One of the most significant variables in the overall cost of e-discovery is review ' that process of reviewing documents and tagging them as responsive or privileged. Our TCO analysis omits review because it is difficult to measure with precision. However, there is evidence that the right platform can improve the speed and accuracy of review and therefore lower its cost.

The problem with measuring and comparing review rates is that so many variables affect the calculation. In addition to variations in document formats and languages, other factors that affect review
rates include:

• The experience of the reviewers.
• The reviewers' familiarity with the material.
• The technologies used to search and organize the material prior to eyes-on review.
• The supervision, quality control, and feedback mechanisms being used.

That said, industry metrics do exist. Among the most commonly cited are the findings of the Text Retrieval Conference (TREC) Legal Track. Its studies in 2006 and 2007 found that the average rate for relevance review was 20-25 documents per hour. (The TREC studies can be found at http://1.usa.gov/TjwfVm and http://1.usa.gov/ToWmdO, respectively.) A 2011 law review article concluded that the industry average was 50 documents per hour. See, “Accounting for the Costs of Electronic Discovery,” by David Degnan, Minnesota Journal of Law, Science & Technology, Vol. 12, Issue 1, http://bit.ly/UqiAPH.

By contrast, an independent analysis of review rates using Catalyst's platform concluded that reviewers achieved average rates of 142 documents per hour for responsiveness review and 76 documents per hour for privilege review. Needless to say, this could add up to significant cost savings in a large-scale review.

Defensibility and Legal Ethics

Two other intangible but essential considerations for legal professionals are defensibility and professional responsibility.

Defensibility is not an easy concept to define, let alone measure. In general terms, defensibility involves any aspect of e-discovery that a lawyer may have to defend in front of a judge. A failure of defensibility can have severe consequences, from waiver of attorney-client privilege to steep monetary or procedural sanctions.

Your choice of platform can bolster or weaken the defensibility of your process. A platform should provide robust tracking and reporting capabilities, so that you can oversee and document the steps you take. It should provide for collaboration, redundancy and quality control, so that the possibility of a misstep is minimized.

With regard to professional responsibility, a lawyer has an ethical duty to ensure the confidentiality and security of client information. Ethical opinions make clear that this duty extends to technology. Before using any platform for e-discovery, cloud or local, the lawyer should take steps to ensure that the platform and the vendor meet the high standards that ethics rules demand.

Conclusion

As a long-time trial lawyer, I've learned to take evidence over opinions any day. While we often hear it said that bringing e-discovery in-house saves money, the facts show otherwise. When you look at the total cost of ownership of cloud versus appliance platforms ' when you factor in all the required and ancillary costs ' the cloud wins hands down.