Analyzing Child Custody Reports

This is the second installment of a four-part series offering a model for attorneys to use when faced with the task of deconstructing a forensic custody assessment.

Effective trial preparation requires that a thorough analysis of the forensic work-product be done in order to determine how to counsel the parent-litigant about the next step in his or her case. A small body of literature has developed that suggests some of the areas most important to consider when analyzing a child custody report, including articles by Austin, Kirkpatrick, & Flens (2011, Family Court Review) and by Gould, Kirkpatrick, Austin, & Martindale (2004, J Child Custody). More recently, the Custody Assessment Analysis System, or CAAS, provided a comprehensive system for the pre-trial analysis of child custody evaluations (CCEs) by attorneys (Wittmann, 2013, MatLaw Corp.).

The inspection of an evaluator's report, focusing on the factors outlined below, produces a “red-flag” analysis: a catalog of threats to reliability that compromise the usefulness of a particular evaluation. The ways in which an assessment may be sturdy and unlikely to yield under attack can also be determined through attention to the assets or strengths of a particular child custody evaluation. The CAAS model suggests four broad lenses for such an analysis, lenses that give structure to our four installments: Management of Professional Relationships, Data Adequacy, Technique Adequacy, and Reasoning Adequacy. The second of these dimensions (data adequacy) is the focus of this installment.

Data Adequacy

The attorney using this lens while examining a CCE report asks the following question: Did the evaluator gather data characterized by sufficient relevance, breadth, and balance to form a foundation for the opinions sought by the court and/or attorneys? That is, what was the quality of the information ' of the data base ' used as a foundation for the opinions that were proffered? There are several errors that can be made as an evaluator decides what information to gather or to set aside, and these errors can have profound implications for the reliability of the overall assessment.

Problems with the Interview Data Set

Were the data gathered via the interviews sufficiently relevant, broad, and balanced? It is important to assess whether the evaluator gathered a sufficient sample of information to answer the questions at hand reliably. For example, was enough attention paid to inquiry about parenting behavior and attitudes, given the centrality of this area for custody assessment? Did the evaluator complete careful inquiry about the personal histories of the litigant-parents (legal, mental health, substance-abuse, child protective, etc.)? Was a thorough assessment of the children and their individual needs, temperaments, etc., conducted? Was there sufficient inquiry about family dynamics, the capacity of both parents to engage in constructive conflict resolution and to shield the children from exposure to intense anger and violence?

It is also important to determine if the evaluator devoted sufficient interview-attention to the roles of other key players; specifically, extended family members, new partners, and siblings who may be aligned with one parent against the other. In their reports, evaluators should articulate the manner in which perceived pros and cons of various access plans were explored, and should relate whatever parental attitudes may have been communicated concerning potential clinical interventions. In some cases, there are very specific issues with which the court is struggling (Examples: Is the mother bipolar? Are there risks of the children being alienated from one parent by the other?). Despite the fact that each case presents different questions, evaluators sometimes utilize a standard interview protocol to which they are accustomed, and fail to make the modifications that would be required for the interview to address difficult issues.

Problems with Testing Data Set

Are the data that were gathered via testing relevant, appropriate, and complete? Here, the attorney, possibly with the aid of a trusted psychological consultant, looks at whether proper tests were chosen, whether important tests were missed, etc. A consortium of associations in the field of psychological measurement has developed the Standards for Educational and Psychological Testing (AERA, 1999), soon to be revised, that can offer a useful starting point for exploring the adequacy of the tests used in a particular CCE. In addition, good resources exist that are focused specifically on test selection and interpretation in custody matters ( e.g., Psychological testing in child custody evaluations, 2005, edited by Flens and Drozd).

In considering the adequacy of the testing conducted in a CCE, the ethnic background of the litigants may be relevant. If one's client is a member of a minority group or of an immigrant population, it is essential to confirm that the comparison group used by the test publishers for the scoring of a particular test includes that ethnic sub-group. In the absence of appropriate normative (comparison) data, the results may not be interpretable for that client. Finally, a foundational issue within this category is the underlying validity and reliability of the tests that were administered. Some assessment methods that evaluators list under report headings such as “tests administered” are not tests, as the term is defined in the Standards for Educational and Psychological Testing, and should not be utilized in forensic contexts. The use of such instruments finds no support in the peer-reviewed research. Finally, the variables measured by a test should have a direct, or close-to-direct correlation with parenting behavior and/or attitudes.

Problems with Collaterals and Records Data Set

Were the data that were gathered via collateral contacts or written records complete and balanced? Sometimes evaluators omit certain key sources of information available to them (teachers, babysitters, pediatricians, etc.) or they fail to balance reasonably the content or the number of the contacts made on behalf of each litigant. The same is sometimes true with respect to written records offered by the litigants. With some custody assessments, it quickly becomes apparent that the nature of the additional data gathered outside of the session with the parties is skewed in a way that may have increased the probability of an unreliable work product.

Implications for Practice

Thoroughly analyzing the adequacy of the data that formed the basis of an evaluator's opinions is akin to inspecting the foundation of a house. When flaws are identified, it is likely that everything resting upon it is weak and untrustworthy. While it is certainly true that gaps or imbalances in CCE data are not uncommon, and may at times be unavoidable, these realities do not lessen the importance of identifying such gaps and imbalances and considering their likely impact on the opinions communicated in the CCE report. It is often one's client who will most quickly spot such deficiencies, by pointing out information s/he offered to the evaluator that appears not to have been considered or data offered by the other parent that were incomplete, distorted, or untrue.

Conclusion

It is critical to try to obtain a copy of the underlying forensic file to complete the analysis described above. Although the rules of discovery vary from state to state, and even, to some extent, from judge to judge, a central aspect of due process involves the right to explore fully the basis of any opinions that may affect liberty interests such as the right to parent one's children. The CCE report is always a filtered description of the data collected (for practical, necessary reasons) and it is only by reading all of the notes taken, all of the test results and interpretive reports, and all of the materials that the evaluator reviewed that an attorney can identify omissions, biases, or distortions that may be relevant to their client's interests.

Jeffrey P. Wittmann, PhD , is a licensed psychologist and trial consultant. See childcustodyforensics.com. David A. Martindale, PhD, ABPP , is the Reporter for the Association of Family and Conciliation Courts' Model Standards of Practice for Child Custody Evaluation. See damartindale.com. Timothy M. Tippins is an Adjunct Professor at Albany Law School and serves on the faculty of the American Academy of Forensic Psychology. Reach him at [email protected]. All three authors are members of this newsletter's Board of Editors.

This is the second installment of a four-part series offering a model for attorneys to use when faced with the task of deconstructing a forensic custody assessment.

Effective trial preparation requires that a thorough analysis of the forensic work-product be done in order to determine how to counsel the parent-litigant about the next step in his or her case. A small body of literature has developed that suggests some of the areas most important to consider when analyzing a child custody report, including articles by Austin, Kirkpatrick, & Flens (2011, Family Court Review) and by Gould, Kirkpatrick, Austin, & Martindale (2004, J Child Custody). More recently, the Custody Assessment Analysis System, or CAAS, provided a comprehensive system for the pre-trial analysis of child custody evaluations (CCEs) by attorneys (Wittmann, 2013, MatLaw Corp.).

The inspection of an evaluator's report, focusing on the factors outlined below, produces a “red-flag” analysis: a catalog of threats to reliability that compromise the usefulness of a particular evaluation. The ways in which an assessment may be sturdy and unlikely to yield under attack can also be determined through attention to the assets or strengths of a particular child custody evaluation. The CAAS model suggests four broad lenses for such an analysis, lenses that give structure to our four installments: Management of Professional Relationships, Data Adequacy, Technique Adequacy, and Reasoning Adequacy. The second of these dimensions (data adequacy) is the focus of this installment.

Data Adequacy

The attorney using this lens while examining a CCE report asks the following question: Did the evaluator gather data characterized by sufficient relevance, breadth, and balance to form a foundation for the opinions sought by the court and/or attorneys? That is, what was the quality of the information ' of the data base ' used as a foundation for the opinions that were proffered? There are several errors that can be made as an evaluator decides what information to gather or to set aside, and these errors can have profound implications for the reliability of the overall assessment.

Problems with the Interview Data Set

Were the data gathered via the interviews sufficiently relevant, broad, and balanced? It is important to assess whether the evaluator gathered a sufficient sample of information to answer the questions at hand reliably. For example, was enough attention paid to inquiry about parenting behavior and attitudes, given the centrality of this area for custody assessment? Did the evaluator complete careful inquiry about the personal histories of the litigant-parents (legal, mental health, substance-abuse, child protective, etc.)? Was a thorough assessment of the children and their individual needs, temperaments, etc., conducted? Was there sufficient inquiry about family dynamics, the capacity of both parents to engage in constructive conflict resolution and to shield the children from exposure to intense anger and violence?

It is also important to determine if the evaluator devoted sufficient interview-attention to the roles of other key players; specifically, extended family members, new partners, and siblings who may be aligned with one parent against the other. In their reports, evaluators should articulate the manner in which perceived pros and cons of various access plans were explored, and should relate whatever parental attitudes may have been communicated concerning potential clinical interventions. In some cases, there are very specific issues with which the court is struggling (Examples: Is the mother bipolar? Are there risks of the children being alienated from one parent by the other?). Despite the fact that each case presents different questions, evaluators sometimes utilize a standard interview protocol to which they are accustomed, and fail to make the modifications that would be required for the interview to address difficult issues.

Problems with Testing Data Set

Are the data that were gathered via testing relevant, appropriate, and complete? Here, the attorney, possibly with the aid of a trusted psychological consultant, looks at whether proper tests were chosen, whether important tests were missed, etc. A consortium of associations in the field of psychological measurement has developed the Standards for Educational and Psychological Testing (AERA, 1999), soon to be revised, that can offer a useful starting point for exploring the adequacy of the tests used in a particular CCE. In addition, good resources exist that are focused specifically on test selection and interpretation in custody matters ( e.g., Psychological testing in child custody evaluations, 2005, edited by Flens and Drozd).

In considering the adequacy of the testing conducted in a CCE, the ethnic background of the litigants may be relevant. If one's client is a member of a minority group or of an immigrant population, it is essential to confirm that the comparison group used by the test publishers for the scoring of a particular test includes that ethnic sub-group. In the absence of appropriate normative (comparison) data, the results may not be interpretable for that client. Finally, a foundational issue within this category is the underlying validity and reliability of the tests that were administered. Some assessment methods that evaluators list under report headings such as “tests administered” are not tests, as the term is defined in the Standards for Educational and Psychological Testing, and should not be utilized in forensic contexts. The use of such instruments finds no support in the peer-reviewed research. Finally, the variables measured by a test should have a direct, or close-to-direct correlation with parenting behavior and/or attitudes.

Problems with Collaterals and Records Data Set

Were the data that were gathered via collateral contacts or written records complete and balanced? Sometimes evaluators omit certain key sources of information available to them (teachers, babysitters, pediatricians, etc.) or they fail to balance reasonably the content or the number of the contacts made on behalf of each litigant. The same is sometimes true with respect to written records offered by the litigants. With some custody assessments, it quickly becomes apparent that the nature of the additional data gathered outside of the session with the parties is skewed in a way that may have increased the probability of an unreliable work product.

Implications for Practice

Thoroughly analyzing the adequacy of the data that formed the basis of an evaluator's opinions is akin to inspecting the foundation of a house. When flaws are identified, it is likely that everything resting upon it is weak and untrustworthy. While it is certainly true that gaps or imbalances in CCE data are not uncommon, and may at times be unavoidable, these realities do not lessen the importance of identifying such gaps and imbalances and considering their likely impact on the opinions communicated in the CCE report. It is often one's client who will most quickly spot such deficiencies, by pointing out information s/he offered to the evaluator that appears not to have been considered or data offered by the other parent that were incomplete, distorted, or untrue.

Conclusion

It is critical to try to obtain a copy of the underlying forensic file to complete the analysis described above. Although the rules of discovery vary from state to state, and even, to some extent, from judge to judge, a central aspect of due process involves the right to explore fully the basis of any opinions that may affect liberty interests such as the right to parent one's children. The CCE report is always a filtered description of the data collected (for practical, necessary reasons) and it is only by reading all of the notes taken, all of the test results and interpretive reports, and all of the materials that the evaluator reviewed that an attorney can identify omissions, biases, or distortions that may be relevant to their client's interests.

Jeffrey P. Wittmann, PhD , is a licensed psychologist and trial consultant. See childcustodyforensics.com. David A. Martindale, PhD, ABPP , is the Reporter for the Association of Family and Conciliation Courts' Model Standards of Practice for Child Custody Evaluation. See damartindale.com. Timothy M. Tippins is an Adjunct Professor at Albany Law School and serves on the faculty of the American Academy of Forensic Psychology. Reach him at [email protected]. All three authors are members of this newsletter's Board of Editors.