Everything But the Kitchen Sink

You've just pulled a steaming pepperoni pizza from the oven and open a kitchen drawer to look for the right tool. Will any tool do? They're all kitchen gadgets. Do you reach for your universal kitchen aid because it claims to do everything ' it slices, it dices? Or, do you reach for the pizza cutter?

There are many tools that are potentially useful ' but finding the right gadget for the job can make the difference between eating pizza now and going hungry. This same quandary can confront attorneys searching their firms' knowledge management (KM) systems for useful intellectual capital using integrated search engine technologies in tandem with specialized KM solutions.

Recently, many firms have taken the innovative step of integrating search engine technologies, such as Google, into their KM systems as a universal or enterprise search utility. This approach allows attorneys to search for materials in addition to documents, such as e-mails, spreadsheets and business support databases. By integrating search engines into their respective intranets, firms enable users to search the entire intranet, document repository or document management system (DMS) for files meeting the search criteria.

On the Web, Google has become the benchmark against which all other search systems are measured, both in terms of ease of use and accuracy.

But does it necessarily apply to legal research? In practice, lawyers need search systems in many situations ranging from well-defined and frequently performed actions to less-predictable and occasional requests. Lawyers may, for example, have need for a search tool for purposes of precedent retrieval, legal research, expert or resource location, electronic discovery, marketing, conflicts checking, etc. In each circumstance, their needs will likely differ. In some cases, they will want a high precision tool; in others, lawyers will need a tool that can cast its net broadly.

Recall v. Precision

The results returned by search engines certainly provide more file and document options, but it is important to understand the difference between “knowledge management” results returned by search engines and results returned from searches on specialized knowledge management tools.

An essential argument for implementing a knowledge management system for many firms is to cut down on time spent looking for past documents that might be useful. For legal research and precedent retrieval systems, it is important that KM systems do not return every single document matching a particular search request, but rather the best material or gold-standard documents.

Another important part of the value proposition of KM systems is their transparency. They should require little, if any, work or input from users or administrators to sort and retrieve useful documents. Automated document vetting should identify the valuable documents containing useful content, and eliminate unwanted or confidential materials. “Trainable” cataloging and search capabilities, approaching artificial intelligence, should “learn” which documents best match a search request. When users run searches for briefs dealing with tax law, the KM system understands the nature of the search, and locates the best examples of documents that are most closely related to the search query. Finally, a KM system should provide more than just assistance finding documents; it should help attorneys review the results by offering smart summaries, and offer features to help attorneys reuse precedent in the drafting process.

Discussing this matter with numerous lawyers confirms that they are satisfied with Google searches for hobbies or vacation plans, but they would not be pleased if a work product retrieval system returned large volumes of materials. In the business context, time is money. Moreover, the value of legal materials may not be clearly discernable from their titles (a search request for a Loan Agreement may return hundreds of similarly titled documents) and, unlike Web collections, legal documents are generally much longer and more complex than typical Web content (each of those Loan Agreements may be 100 pages long) and may take substantial time to read to find the right one.

Breadth v. Depth

The sources from which search engines and knowledge management systems pull information separates the two types of tools. Search engines offer breadth and provide the benefit of returning all types of files, from spreadsheets to e-mails. Search engines can crawl in and out of multiple file locations, including servers, folders, DMS systems and the firm's intranet.

Knowledge management tools, on the other hand, return depth by providing numerous indexing methods to precisely locate valuable and reusable materials. Sophisticated knowledge management systems will have tools to automatically identify valuable materials and filter out unwanted documents. KM systems also have the capability to automatically catalog collections ' they may offer a specified taxonomy, such as the West Key Number System; or firm information administrators may set up a customized framework. Advanced KM systems may also have the capacity to automatically extract key profile information from the documents, such as titles, parties, effective dates and jurisdiction. Searches, then, are run on this catalog and profile index, and documents are pulled from a specific location.

Practicalities of Enterprise Search

In theory, it sounds great to be able to search for anything in all the firm's data repositories. But the practicalities make this more complex than at first blush. Here are some considerations.

• Review of business need: While it is possible to search everything, it is less clear how lawyers and staff would use such a system, making it more difficult to define and justify a business case for investment.
• Longevity of storage: E-mail, for many firms, is a temporary storage platform that is automatically archived after a short period.
• Security: Many data platforms in the firm lack security mechanisms that would allow the firm to confidently open collections to enterprise search.
• Filtering or vetting: In the case of materials stored in a DMS, a vetting process will typically identify between five percent and 20 percent of the collection as having valuable reusable content. The corollary noise factor in the case of other data sources, such as e-mail, will likely be much higher, and lawyers typically have a low tolerance for imprecise systems when performing legal research.

Information Retrieval Systems and KM

By examining the requirements and practicalities of information retrieval systems, it is possible to identify the key components of a search system that meets a law firm's knowledge management needs:

• Knowledge Map: A mechanism to select the source repositories for both indexing purposes and information retrieval;
• Vetting Process: A mechanism to synthesize the raw data to identify valuable, reusable material and eliminate confidential or unwanted material;
• Word Search: A tool to search for words and characters, potentially augmented by Boolean, Natural Language, thesaurus and conceptual pattern matching to partially overcome the challenge presented by the richness of language;
• Substantive Indexing: Processes to identify key information and knowledge from the raw source material, such as:
• • Relationships: A profile or index (typically having a one-to-one relationship with the underlying source data) that identifies bibliographic (authorship) and contextual (receivership) information;
  • Patterns: An organization of relationship information (typically having a one-to-many relationship with the underlying source data) that identifies and organizes content and subject matter information; and
  • Associations: An intersection between bibliographic, context and content information (typically having a many-to-many relationship with the underlying source data) that can be used find links between information resources.
• Document Relationships: A process to cross-reference documents using hypertext links and citations; and
• Relevancy Ranking: A mechanism to rank search results by relevancy across disparate collections.

Enterprise Search engines will likely have the features described in a) and c), and because these tools can be applied against a wide range of source collections, such systems offer the advantage of breadth. A knowledge retrieval system will, in addition, require vetting, indexing and cross-reference capabilities in order to pinpoint valuable materials. However, such additional capabilities can operate only against more limited information resources, such as document, precedent and legal research collections.

Choosing the Right Tool

In the end, the answer is that each solution does what it is designed to do. Search engines are optimized for high recall returning as many results as possible, as closely linked to the query as possible. For example, Google returns 1,110,000 results on “merger agreement.” KM solutions are optimized to focus results on the most valuable materials within the firm.

In practice, search engines and knowledge management solutions can live happily together, and complement one another with their different strengths. Search engines are very flexible, and can be pointed at a broad array of data resources; precision search tools offer more structure, but are more limited in the types of information resources that they can handle ' at least today. Both technologies are on a path for convergence. KM systems need the addition of a knowledge map to be able to target a wide variety of source collections, and a mechanism to determine what, if any, additional indexing features should be applied. Enterprise search engines need the addition of vetting and advanced indexing to provide additional structure. For those firms considering which path to follow ' since many will face budgetary constraints precluding the option of choosing both ' it is recommended that the firm considers its business needs and the practical value of each approach.

You've just pulled a steaming pepperoni pizza from the oven and open a kitchen drawer to look for the right tool. Will any tool do? They're all kitchen gadgets. Do you reach for your universal kitchen aid because it claims to do everything ' it slices, it dices? Or, do you reach for the pizza cutter?

There are many tools that are potentially useful ' but finding the right gadget for the job can make the difference between eating pizza now and going hungry. This same quandary can confront attorneys searching their firms' knowledge management (KM) systems for useful intellectual capital using integrated search engine technologies in tandem with specialized KM solutions.

Recently, many firms have taken the innovative step of integrating search engine technologies, such as Google, into their KM systems as a universal or enterprise search utility. This approach allows attorneys to search for materials in addition to documents, such as e-mails, spreadsheets and business support databases. By integrating search engines into their respective intranets, firms enable users to search the entire intranet, document repository or document management system (DMS) for files meeting the search criteria.

On the Web, Google has become the benchmark against which all other search systems are measured, both in terms of ease of use and accuracy.

But does it necessarily apply to legal research? In practice, lawyers need search systems in many situations ranging from well-defined and frequently performed actions to less-predictable and occasional requests. Lawyers may, for example, have need for a search tool for purposes of precedent retrieval, legal research, expert or resource location, electronic discovery, marketing, conflicts checking, etc. In each circumstance, their needs will likely differ. In some cases, they will want a high precision tool; in others, lawyers will need a tool that can cast its net broadly.

Recall v. Precision

The results returned by search engines certainly provide more file and document options, but it is important to understand the difference between “knowledge management” results returned by search engines and results returned from searches on specialized knowledge management tools.

An essential argument for implementing a knowledge management system for many firms is to cut down on time spent looking for past documents that might be useful. For legal research and precedent retrieval systems, it is important that KM systems do not return every single document matching a particular search request, but rather the best material or gold-standard documents.

Another important part of the value proposition of KM systems is their transparency. They should require little, if any, work or input from users or administrators to sort and retrieve useful documents. Automated document vetting should identify the valuable documents containing useful content, and eliminate unwanted or confidential materials. “Trainable” cataloging and search capabilities, approaching artificial intelligence, should “learn” which documents best match a search request. When users run searches for briefs dealing with tax law, the KM system understands the nature of the search, and locates the best examples of documents that are most closely related to the search query. Finally, a KM system should provide more than just assistance finding documents; it should help attorneys review the results by offering smart summaries, and offer features to help attorneys reuse precedent in the drafting process.

Discussing this matter with numerous lawyers confirms that they are satisfied with Google searches for hobbies or vacation plans, but they would not be pleased if a work product retrieval system returned large volumes of materials. In the business context, time is money. Moreover, the value of legal materials may not be clearly discernable from their titles (a search request for a Loan Agreement may return hundreds of similarly titled documents) and, unlike Web collections, legal documents are generally much longer and more complex than typical Web content (each of those Loan Agreements may be 100 pages long) and may take substantial time to read to find the right one.

Breadth v. Depth

The sources from which search engines and knowledge management systems pull information separates the two types of tools. Search engines offer breadth and provide the benefit of returning all types of files, from spreadsheets to e-mails. Search engines can crawl in and out of multiple file locations, including servers, folders, DMS systems and the firm's intranet.

Knowledge management tools, on the other hand, return depth by providing numerous indexing methods to precisely locate valuable and reusable materials. Sophisticated knowledge management systems will have tools to automatically identify valuable materials and filter out unwanted documents. KM systems also have the capability to automatically catalog collections ' they may offer a specified taxonomy, such as the West Key Number System; or firm information administrators may set up a customized framework. Advanced KM systems may also have the capacity to automatically extract key profile information from the documents, such as titles, parties, effective dates and jurisdiction. Searches, then, are run on this catalog and profile index, and documents are pulled from a specific location.

Practicalities of Enterprise Search

In theory, it sounds great to be able to search for anything in all the firm's data repositories. But the practicalities make this more complex than at first blush. Here are some considerations.

• Review of business need: While it is possible to search everything, it is less clear how lawyers and staff would use such a system, making it more difficult to define and justify a business case for investment.
• Longevity of storage: E-mail, for many firms, is a temporary storage platform that is automatically archived after a short period.
• Security: Many data platforms in the firm lack security mechanisms that would allow the firm to confidently open collections to enterprise search.
• Filtering or vetting: In the case of materials stored in a DMS, a vetting process will typically identify between five percent and 20 percent of the collection as having valuable reusable content. The corollary noise factor in the case of other data sources, such as e-mail, will likely be much higher, and lawyers typically have a low tolerance for imprecise systems when performing legal research.

Information Retrieval Systems and KM

By examining the requirements and practicalities of information retrieval systems, it is possible to identify the key components of a search system that meets a law firm's knowledge management needs:

• Knowledge Map: A mechanism to select the source repositories for both indexing purposes and information retrieval;
• Vetting Process: A mechanism to synthesize the raw data to identify valuable, reusable material and eliminate confidential or unwanted material;
• Word Search: A tool to search for words and characters, potentially augmented by Boolean, Natural Language, thesaurus and conceptual pattern matching to partially overcome the challenge presented by the richness of language;
• Substantive Indexing: Processes to identify key information and knowledge from the raw source material, such as:
• • Relationships: A profile or index (typically having a one-to-one relationship with the underlying source data) that identifies bibliographic (authorship) and contextual (receivership) information;
  • Patterns: An organization of relationship information (typically having a one-to-many relationship with the underlying source data) that identifies and organizes content and subject matter information; and
  • Associations: An intersection between bibliographic, context and content information (typically having a many-to-many relationship with the underlying source data) that can be used find links between information resources.
• Document Relationships: A process to cross-reference documents using hypertext links and citations; and
• Relevancy Ranking: A mechanism to rank search results by relevancy across disparate collections.

Enterprise Search engines will likely have the features described in a) and c), and because these tools can be applied against a wide range of source collections, such systems offer the advantage of breadth. A knowledge retrieval system will, in addition, require vetting, indexing and cross-reference capabilities in order to pinpoint valuable materials. However, such additional capabilities can operate only against more limited information resources, such as document, precedent and legal research collections.

Choosing the Right Tool

In the end, the answer is that each solution does what it is designed to do. Search engines are optimized for high recall returning as many results as possible, as closely linked to the query as possible. For example, Google returns 1,110,000 results on “merger agreement.” KM solutions are optimized to focus results on the most valuable materials within the firm.

In practice, search engines and knowledge management solutions can live happily together, and complement one another with their different strengths. Search engines are very flexible, and can be pointed at a broad array of data resources; precision search tools offer more structure, but are more limited in the types of information resources that they can handle ' at least today. Both technologies are on a path for convergence. KM systems need the addition of a knowledge map to be able to target a wide variety of source collections, and a mechanism to determine what, if any, additional indexing features should be applied. Enterprise search engines need the addition of vetting and advanced indexing to provide additional structure. For those firms considering which path to follow ' since many will face budgetary constraints precluding the option of choosing both ' it is recommended that the firm considers its business needs and the practical value of each approach.