A Straightforward Methodology for Remediation Projects

Is your organization creating and accumulating mountains of information? If it is like most, the answer is yes. But in those mountains hides a mixture of content, including: important business records, information subject to legal holds or regulatory requirements, sensitive information (e.g., confidential, private, or trade secret), and outright junk. Remediation is an information governance (IG) process directed at bringing order to information and is a tool that can help you gain control over the undifferentiated mess.

At the Information Governance Initiative's (IGI) last IGI Boot Camp held at Relativity Fest, we explored the topic of remediation and published a short paper, “The Role of Remediation in Information Governance,” which sets out, among other points, an organizing principle, maturity model framework, and methodology for remediation. See , www.iginitiative.com/community. This article highlights some of the key points from that paper.

More than Deletion

First, it is important to understand that remediation is not just deletion, although a full remediation project might include a deletion component. Remediation helps bring order to information and enables critical IG activities like cleaning up, organizing and migrating information. The overarching goals of a full remediation project are keeping the information needed to meet your organization's objectives and obligations (e.g., business, legal, and regulatory); organizing and making it accessible; and defensibly deleting what is no longer useful.

To get started with remediation, you need both visibility into the targeted information ' some idea of what it is ' and the context in which that information exists. Specifically, even if you know what a given piece of information is , you probably can't make a remediation decision about it unless you know something about the business, legal, or regulatory environment in which it lives (e.g., Is this needed for a business reason? Is it subject to a legal hold?). Visibility into the targeted information could come from a number of sources, including file and file system metadata, full content indexing and analysis, or organizational knowledge about what it is. The more you know, the more accurate your remediation decisions will be, but gaining visibility costs time and money. Determining how much you should invest is a business decision involving balancing risks and costs. The organizing principle for remediation decisions is striking the right balance for your organization between the cost of gaining more visibility and the risks of making the wrong decision about that information.

Making the right decision is critical, but what kinds of remediation decisions you will be able to make will turn, in part, on the maturity level of your IG program. In our paper, we explore in more detail the types of remediation decisions that you might be able to make at three broad levels of IG program maturity (initial, repeatable, and advanced). There is some interplay between maturity level and visibility into information. Typically, the more advanced the program, the more visibility. Further, if your IG program is immature, gaining more visibility might not be worth the cost because if you don't have basic policies and procedures in place, you might not know if you can make a given remediation decision. At each decision point, however, you should determine whether there is enough visibility into the information to give you confidence to take a particular remediation step or if you need to dig deeper. Ultimately, this brings you back to the basic organizing principle discussed above, and that is a question that must be answered in a manner tailored to your specific organization. Even so, remediation steps, including some that can whittle away at large portions of your target, are possible no matter what your organization's IG maturity level.

Divide and Conquer

Interestingly, many remediation decisions lend themselves nicely to a binary decision structure that leads to a straightforward remediation methodology that can be adapted to many circumstances. For example, organizations are often faced with remediation decisions like: Should this information be migrated here? Is it okay to delete this? Is this confidential or HIPPA-related content? If you had enough visibility into the information and knew the rules to apply, you could answer each of these questions, yes or no. But even if you don't have enough visibility into all of your data, you can still make progress. In a scenario like this, the first step is to divide the body of information into three buckets:

1. There is sufficient information to be confident that the answer to the remediation question should be “yes.”
2. There is sufficient information to be confident that the answer to the remediation question should be “no.”
3. There is insufficient information to answer the remediation question.

The first “bucketing” decision can be made using any level of visibility into the data deemed appropriate based on the organizing principle, and you might not need to dig into to the content to make the first division (Metadata might be enough). Strategically, based on the organizing principle, it may not make sense to dig deeper for a given remediation decision.

Next, you start to tackle the third bucket. As you dig deeper into the target data in bucket three and gain more visibility, you can move it into buckets one or two. After metadata, you might move next to “data archaeology” (e.g., interviews with employees, legacy documentation and policies, reviews of historical systems configurations) and eventually indexing and content analytics. At each successive level, always return to the organizing principle. Eventually, you may determine that it makes sense to stop and that a full content analysis is not worth it and that the data should be kept for a longer period of time. Of course, none of these decisions can be made in a policy vacuum. The paper discusses common elements of a strong IG policy foundation.

Conclusion

The importance of having solid IG policies and procedures in place before you tackle a remediation project is reflected in the results from the IGI's Annual Report 2014. As part of our research, we asked practitioners and providers what projects they or their clients, respectively, were or were planning to work on over the next 12 months. Remediation projects, like data consolidation/cleanup and migration of unstructured information, ranked in the top three responses for both groups. However, the top response for both was updating policies and procedures, with nearly three quarters of practitioners and 81% of providers saying that they or their clients, respectively, were undertaking such efforts. Similarly, updating policies and procedures was a top answer when the question was what projects would be undertaken if there were budget and authority to do so. These rankings reflect a logical order in developing a functioning IG program. While you can make some progress at different maturity levels, what you can accomplish will be limited by the extent to which your policies and procedures are developed.

Ann Snyder is senior fellow at the Information Governance Initiative (IGI). She is responsible for managing the IGI's research efforts and overseeing their Task Force Program which, in collaboration with members of the IG community, develops practical tools for use by IG practitioners.

Is your organization creating and accumulating mountains of information? If it is like most, the answer is yes. But in those mountains hides a mixture of content, including: important business records, information subject to legal holds or regulatory requirements, sensitive information (e.g., confidential, private, or trade secret), and outright junk. Remediation is an information governance (IG) process directed at bringing order to information and is a tool that can help you gain control over the undifferentiated mess.

At the Information Governance Initiative's (IGI) last IGI Boot Camp held at Relativity Fest, we explored the topic of remediation and published a short paper, “The Role of Remediation in Information Governance,” which sets out, among other points, an organizing principle, maturity model framework, and methodology for remediation. See , www.iginitiative.com/community. This article highlights some of the key points from that paper.

More than Deletion

First, it is important to understand that remediation is not just deletion, although a full remediation project might include a deletion component. Remediation helps bring order to information and enables critical IG activities like cleaning up, organizing and migrating information. The overarching goals of a full remediation project are keeping the information needed to meet your organization's objectives and obligations (e.g., business, legal, and regulatory); organizing and making it accessible; and defensibly deleting what is no longer useful.

To get started with remediation, you need both visibility into the targeted information ' some idea of what it is ' and the context in which that information exists. Specifically, even if you know what a given piece of information is , you probably can't make a remediation decision about it unless you know something about the business, legal, or regulatory environment in which it lives (e.g., Is this needed for a business reason? Is it subject to a legal hold?). Visibility into the targeted information could come from a number of sources, including file and file system metadata, full content indexing and analysis, or organizational knowledge about what it is. The more you know, the more accurate your remediation decisions will be, but gaining visibility costs time and money. Determining how much you should invest is a business decision involving balancing risks and costs. The organizing principle for remediation decisions is striking the right balance for your organization between the cost of gaining more visibility and the risks of making the wrong decision about that information.

Making the right decision is critical, but what kinds of remediation decisions you will be able to make will turn, in part, on the maturity level of your IG program. In our paper, we explore in more detail the types of remediation decisions that you might be able to make at three broad levels of IG program maturity (initial, repeatable, and advanced). There is some interplay between maturity level and visibility into information. Typically, the more advanced the program, the more visibility. Further, if your IG program is immature, gaining more visibility might not be worth the cost because if you don't have basic policies and procedures in place, you might not know if you can make a given remediation decision. At each decision point, however, you should determine whether there is enough visibility into the information to give you confidence to take a particular remediation step or if you need to dig deeper. Ultimately, this brings you back to the basic organizing principle discussed above, and that is a question that must be answered in a manner tailored to your specific organization. Even so, remediation steps, including some that can whittle away at large portions of your target, are possible no matter what your organization's IG maturity level.

Divide and Conquer

Interestingly, many remediation decisions lend themselves nicely to a binary decision structure that leads to a straightforward remediation methodology that can be adapted to many circumstances. For example, organizations are often faced with remediation decisions like: Should this information be migrated here? Is it okay to delete this? Is this confidential or HIPPA-related content? If you had enough visibility into the information and knew the rules to apply, you could answer each of these questions, yes or no. But even if you don't have enough visibility into all of your data, you can still make progress. In a scenario like this, the first step is to divide the body of information into three buckets:

1. There is sufficient information to be confident that the answer to the remediation question should be “yes.”
2. There is sufficient information to be confident that the answer to the remediation question should be “no.”
3. There is insufficient information to answer the remediation question.

The first “bucketing” decision can be made using any level of visibility into the data deemed appropriate based on the organizing principle, and you might not need to dig into to the content to make the first division (Metadata might be enough). Strategically, based on the organizing principle, it may not make sense to dig deeper for a given remediation decision.

Next, you start to tackle the third bucket. As you dig deeper into the target data in bucket three and gain more visibility, you can move it into buckets one or two. After metadata, you might move next to “data archaeology” (e.g., interviews with employees, legacy documentation and policies, reviews of historical systems configurations) and eventually indexing and content analytics. At each successive level, always return to the organizing principle. Eventually, you may determine that it makes sense to stop and that a full content analysis is not worth it and that the data should be kept for a longer period of time. Of course, none of these decisions can be made in a policy vacuum. The paper discusses common elements of a strong IG policy foundation.

Conclusion

The importance of having solid IG policies and procedures in place before you tackle a remediation project is reflected in the results from the IGI's Annual Report 2014. As part of our research, we asked practitioners and providers what projects they or their clients, respectively, were or were planning to work on over the next 12 months. Remediation projects, like data consolidation/cleanup and migration of unstructured information, ranked in the top three responses for both groups. However, the top response for both was updating policies and procedures, with nearly three quarters of practitioners and 81% of providers saying that they or their clients, respectively, were undertaking such efforts. Similarly, updating policies and procedures was a top answer when the question was what projects would be undertaken if there were budget and authority to do so. These rankings reflect a logical order in developing a functioning IG program. While you can make some progress at different maturity levels, what you can accomplish will be limited by the extent to which your policies and procedures are developed.

Ann Snyder is senior fellow at the Information Governance Initiative (IGI). She is responsible for managing the IGI's research efforts and overseeing their Task Force Program which, in collaboration with members of the IG community, develops practical tools for use by IG practitioners.