One Picture May Not Be Worth 1,000 Words

Wishes v. Perception

If these cautionary statements are applied to family drawings, it follows that if a child produces a drawing in which she places herself close to her father, this may represent a projection of her perception of her relationship with her father, but it may also represent a wish rather than a perception. The child whose father who is either physically absent or emotionally distant might draw the father close to the child. An older child whose drawing skills are somewhat better might draw himself being cared for by a parent who, in reality, provides deficient care.

Reliability

Other complicating factors must also be considered. Even if children's family drawings reflected the children's perceptions of family relationships as accurately as thermometers measure temperature, there is no basis whatsoever for presuming that such perceptions are stable over time. How a particular child views his parents and his relationship with each of them today may be dramatically different from how he will view his parents and his relationship with them six months from today. Finally, even if we could presume that it is always a perception (as opposed to a wish) that is being projected, and even if we could presume that such perceptions are reasonably stable over time, are we to presume that children's perceptions of family dynamics are accurate (or that if they are inaccurate, it does not matter because it is only their perceptions that are of concern to us)?

The problem of temporal stability (or lack thereof) relates to what mental health professionals refer to as test/re-test reliability. Consideration must also be given to what we refer to as inter-judge reliability. There is no evidence that different experts viewing the same drawing will formulate the same opinions concerning the information provided by the drawing.

Inter-Judge Reliability

Three decades ago, Richard Dana, a psychologist who has written extensively on projective techniques, called attention to the fact that interpretations of responses to projective stimuli are vulnerable to eisegesis ' distortion stemming from projection by the examiner (as opposed to the examinee) deriving from the examiner's theoretical biases, emotional investment in certain hypotheses, etc. The late Anne Anastasi, one of psychology's leaders in the field of assessment, opined that the interpretation of responses to projective stimuli “may reveal more about the theoretical orientation, favorite hypotheses, and personality idiosyncrasies of the examiner than it does about the examinee's personality dynamics.”

What Constitutes Expert Evidence?

The scope and quality of the input offered by an expert to a court cannot be judged by the scope and quality of the expert's education and training. In an article written several years ago addressing the cross-examination of experts, I observed that “[t]he defining attributes of an expert opinion relate not to the credentials held by the individual whose fingers type the words or from whose mouth the words flow; rather, the requisite characteristics relate to the procedures that were employed in formulating the opinion and the body of knowledge that forms the foundation upon which those procedures were developed.”

That admissible expertise is defined by methodology ' not by credentials ' was established by the United States Supreme Court in Kumho Tire Co., v. Carmichael, 526 U.S. 137 (1999). Evidence proffered by an expert had been rejected by an Alabama court. In upholding the ruling of the Alabama court, the Supreme Court called attention to the fact that “[t]he District Court did not doubt [the expert's] qualifications. Rather, it excluded the testimony because, despite these qualifications, it initially doubted, and then found unreliable ' the methodology employed by the expert ' ” (at 153).

Standards for Educational and Psychological Testing

The single most authoritative treatise on the subject of the criteria to be employed in selecting and administering psychological assessment instruments and subsequently interpreting the data obtained is the Standards for Educational and Psychological Testing. There are two editions, one published in 1985, the other in 1999. No statement appears in the 1999 edition that might suggest that it supplants the 1985 edition, nothing contained in the 1999 edition alters the content of the 1985 edition, and the importance of each is generally accepted. The 1985 edition's Standard 6.1 reads: “Test users should evaluate the available written documentation on the validity and reliability of tests for the specific use intended.” No documentation exists for human figure drawings.

Standard 6.3, also from the 1985 edition, reads: “When a test is to be used for a purpose for which it has not been previously validated, or for which there is no supported claim for validity, the user is responsible for providing evidence of validity.” No text in which psychological assessment instruments and procedures are discussed provides validity data for human figure drawings, and reviews appearing in the Buros Mental Measurements Yearbook have consistently reported that human figure drawings are neither reliable nor valid.

In the opening pages of the 1999 edition of the Standards for Educational and Psychological Testing, a definition of the word “test” is provided. “A test is an evaluative device or procedure in which a sample of an examinee's behavior in a specified domain is obtained and subsequently evaluated and scored using a standardized process.” First, there is no basis for conceptualizing a child's drawing as a “sample of an examinee's behavior” that is relevant to the issues before the court in a custody/access dispute. Second, though some of those who wish to employ drawings in their assessments have endeavored to create standardized processes, there is no evidence to suggest that any of the systems designed to date demonstrate acceptable inter-judge reliability.

Readers of the 1999 edition of the Standards are reminded that documentation of the type that accompanies accepted assessment instruments is important because it provides test users “with the information needed to make sound judgments about the nature and quality of the test, the resulting scores, and the interpretation based on the test scores.” It is noted that “[t]he greater the potential impact on test takers, for good or ill, the greater the need to identify and satisfy the relevant standards.” Where the issues in dispute involve parenting plans for children, the “potential impact on test takers” is significant.

Though no published data are available for reference, it is my impression that, more often than not, practitioners who rely upon drawings as an assessment technique offer no statements in their reports that might alert readers to the deficiencies in the technique. In the 1999 edition of the Standards the following admonition appears: “Professionals are expected to make every effort to be aware of evidence of validity and reliability that supports or does not support their inferences and to place appropriate limits on the opinions rendered.”

When practitioners employ what is known as the sign approach to figure drawing interpretation, certain features in the drawings are viewed as signs (indicators) of various personality characteristics, life experiences, or personality characteristics. In commenting on the sign approach, several well-respected writers have concluded that the advocates of the sign method often disagree among themselves concerning what a particular feature indicates.

In reviews of drawing techniques prepared for The Buros Institute and published in the Mental Measurements Yearbook, reviewers have consistently concluded that there are no empirical data to support the use of drawings.

Validity

Most simply put, validity addresses the issue of whether we are measuring what we think we are measuring. If the evaluator's goal is to gather information that will shed light on what parenting plan is most likely to be in the long-term psychological best interests of children and if data are gathered that are strongly influenced by day-to-day events (as poor test/re-test reliability data suggest is the case), those data shed no light on the issue of interest.

Discriminant Validity

Where drawings have been utilized for the purpose of identifying someone as a member of a specific group (for example, sexually abused children), it is particularly important that readers of an evaluator's report be familiar with the concept of discriminant validity. The effectiveness of an assessment instrument or technique in discriminating between two classes of examinees defines that instrument's discriminant validity.

Consider this assertion by an evaluator: “I have found the following [features in a child's drawing] to be suggestive of abuse.” In making such an assertion, the evaluator is claiming that he can discriminate between abused children and non-abused children on the basis of certain features of their drawings. Research has repeatedly shown drawings to be lacking in discriminant validity. Even if there were research findings supporting the contention that a certain feature is often observed in drawings produced by abused children, detection of this feature would be useful only if research also demonstrated that the specified feature is observed exclusively in the drawings of abused children and is not observed in the drawings of non-abused children.

What Is the Data Source?

The methods employed by evaluators in gathering information from children differ dramatically from the methods employed in gathering information from adults. Statements made by individuals of any age are viewed as a data source and evaluators interviewing children often find that play, building tasks, drawing tasks, and similar activities facilitate constructive oral interaction. When drawings are utilized as a stimulus for conversation and where it is the statements made that are conceptualized as the data source, it is unlikely that any harm is created by the use of drawings. It is my contention, however, that significant risk of harm is created when no inquiry is conducted and when the drawings themselves and the evaluator's interpretations of those drawings are viewed as a data source.

Conclusion

The psychologists' Ethical Standard 9.02(a) reads:

Psychologists administer, adapt, score, interpret, or use assessment techniques, interviews, tests, or instruments in a manner and for purposes that are appropriate in light of the research on or evidence of the usefulness and proper application of the techniques [emphasis added].” Section (b) reads, in pertinent part: “Psychologists use assessment instruments whose validity and reliability have been established [emphasis added] for use with members of the population tested.

In jurisdictions in which the Daubert criteria are employed in assessing the admissibility of proffered expert evidence, the rejection of findings and opinions based upon drawings should be a foregone conclusion. Not one of the five Daubert criteria are met: 1) the underlying theory is not falsifiable; 2) the procedure has not been endorsed in peer- reviewed journals in the field of forensic psychology; 3) we have no clue concerning the error rate; 4) the use of drawings as a data source is not “sufficiently established to have gained general acceptance in the particular field” in which drawings are being used; and 5) there are no standards controlling the use of drawings as a data source.

In Frye jurisdictions, seeking the exclusion of expert evidence derived from drawings may require the presentation of a persuasive argument. In 1923, when a Federal Circuit Court ruled on the admissibility of data from a systolic blood pressure deception test (Frye v. United States, 295 F. 1013 (D.C. Cir. 1923)), it declared that “while courts will go a long way in admitting expert testimony deduced from a well-recognized scientific principle or discovery, the thing from which the deduction is made must be sufficiently established to have gained general acceptance [emphasis added] in the particular field in which it belongs.” Though opposing experts may express different opinions concerning “general acceptance” or how the court should identify the “particular field” in which “the thing from which the deduction is made” belongs, it is my opinion that the recognition, in 2001, by the Commission for the Recognition of Specialties in Professional Psychology that forensic psychology is a specialty means that forensic psychology (in contrast with clinical psychology, school psychology, etc.) is a particular field within the discipline of psychology and that the general acceptance (or lack thereof) of utilizing drawings as a component of one's evaluative methodology must be examined with reference to forensic psychology.

Finally, it is my understanding (as a mental health professional) that when expert opinion testimony is being proffered, the burden of demonstrating evidentiary reliability lies with the proponent of the proffered testimony and that the suitability of the evidence must be established by a preponderance of the evidence. Those who interpret drawings and employ those interpretations in formulating their opinions in custody evaluations have selected an assessment technique that is not suitable for the use to which it has been put.

David Martindale, PhD, is board certified in forensic psychology by the American Board of Professional Psychology, served as the Reporter for the Association of Family and Conciliation Courts' Model Standards for Child Custody Evaluation and is the co-author, with Jon Gould, of The Art and Science of Child Custody Evaluations. His practice is limited to consulting with attorneys, psychologists, and psychology licensing boards in the areas of child custody and professional ethics and standards. He can be contacted by e-mail at [email protected]. This article first appeared in The Matrimonial Strategist, an Incisve Media sister publication of this newsletter.

Childrens' drawings are sometimes treated as evidence of their perceptions of their family relationships. These pictures may be evaluated for insights into whether a child feels emotionally close to parent who is seeking custody or is afraid of a person accused of sexually abusing the child. But are such pictures really “worth 1,000 words”?

If one fully understands the concept of projection, one is more likely than not to conclude that drawings cannot possibly be viewed as a reliable source of information concerning children's perceptions of themselves, their families, the dynamics within their families, or anything else. Even if it could be demonstrated that the dynamic of projection consistently operates as children produce these drawings, there would still be no basis for relying upon them. The early proponents of projective techniques made it clear that we cannot be certain what perceptions, attitudes, fears, or wishes are being projected.

Practitioners who utilize drawings are generally in agreement that the same-sex figure should be conceptualized as a “self” figure. Even if we presume that drawings “tell” us something about the psychology of children who have produced them, it must be recognized that the self figure may represent the child's perception of himself or herself (with emphasis either on physical or psychological attributes); may represent what the child wishes himself or herself to be (again, with emphasis either on physical or psychological attributes); or, to complicate things further, the drawing may symbolically portray physical or psychological impairment, compensation for a real or perceived shortcoming, or a combination of these factors.

Wishes v. Perception

If these cautionary statements are applied to family drawings, it follows that if a child produces a drawing in which she places herself close to her father, this may represent a projection of her perception of her relationship with her father, but it may also represent a wish rather than a perception. The child whose father who is either physically absent or emotionally distant might draw the father close to the child. An older child whose drawing skills are somewhat better might draw himself being cared for by a parent who, in reality, provides deficient care.

Reliability

Other complicating factors must also be considered. Even if children's family drawings reflected the children's perceptions of family relationships as accurately as thermometers measure temperature, there is no basis whatsoever for presuming that such perceptions are stable over time. How a particular child views his parents and his relationship with each of them today may be dramatically different from how he will view his parents and his relationship with them six months from today. Finally, even if we could presume that it is always a perception (as opposed to a wish) that is being projected, and even if we could presume that such perceptions are reasonably stable over time, are we to presume that children's perceptions of family dynamics are accurate (or that if they are inaccurate, it does not matter because it is only their perceptions that are of concern to us)?

The problem of temporal stability (or lack thereof) relates to what mental health professionals refer to as test/re-test reliability. Consideration must also be given to what we refer to as inter-judge reliability. There is no evidence that different experts viewing the same drawing will formulate the same opinions concerning the information provided by the drawing.

Inter-Judge Reliability

Three decades ago, Richard Dana, a psychologist who has written extensively on projective techniques, called attention to the fact that interpretations of responses to projective stimuli are vulnerable to eisegesis ' distortion stemming from projection by the examiner (as opposed to the examinee) deriving from the examiner's theoretical biases, emotional investment in certain hypotheses, etc. The late Anne Anastasi, one of psychology's leaders in the field of assessment, opined that the interpretation of responses to projective stimuli “may reveal more about the theoretical orientation, favorite hypotheses, and personality idiosyncrasies of the examiner than it does about the examinee's personality dynamics.”

What Constitutes Expert Evidence?

The scope and quality of the input offered by an expert to a court cannot be judged by the scope and quality of the expert's education and training. In an article written several years ago addressing the cross-examination of experts, I observed that “[t]he defining attributes of an expert opinion relate not to the credentials held by the individual whose fingers type the words or from whose mouth the words flow; rather, the requisite characteristics relate to the procedures that were employed in formulating the opinion and the body of knowledge that forms the foundation upon which those procedures were developed.”

That admissible expertise is defined by methodology ' not by credentials ' was established by the United States Supreme Court in Kumho Tire Co., v. Carmichael , 526 U.S. 137 (1999). Evidence proffered by an expert had been rejected by an Alabama court. In upholding the ruling of the Alabama court, the Supreme Court called attention to the fact that “[t]he District Court did not doubt [the expert's] qualifications. Rather, it excluded the testimony because, despite these qualifications, it initially doubted, and then found unreliable ' the methodology employed by the expert ' ” (at 153).

Standards for Educational and Psychological Testing

The single most authoritative treatise on the subject of the criteria to be employed in selecting and administering psychological assessment instruments and subsequently interpreting the data obtained is the Standards for Educational and Psychological Testing. There are two editions, one published in 1985, the other in 1999. No statement appears in the 1999 edition that might suggest that it supplants the 1985 edition, nothing contained in the 1999 edition alters the content of the 1985 edition, and the importance of each is generally accepted. The 1985 edition's Standard 6.1 reads: “Test users should evaluate the available written documentation on the validity and reliability of tests for the specific use intended.” No documentation exists for human figure drawings.

Standard 6.3, also from the 1985 edition, reads: “When a test is to be used for a purpose for which it has not been previously validated, or for which there is no supported claim for validity, the user is responsible for providing evidence of validity.” No text in which psychological assessment instruments and procedures are discussed provides validity data for human figure drawings, and reviews appearing in the Buros Mental Measurements Yearbook have consistently reported that human figure drawings are neither reliable nor valid.

In the opening pages of the 1999 edition of the Standards for Educational and Psychological Testing, a definition of the word “test” is provided. “A test is an evaluative device or procedure in which a sample of an examinee's behavior in a specified domain is obtained and subsequently evaluated and scored using a standardized process.” First, there is no basis for conceptualizing a child's drawing as a “sample of an examinee's behavior” that is relevant to the issues before the court in a custody/access dispute. Second, though some of those who wish to employ drawings in their assessments have endeavored to create standardized processes, there is no evidence to suggest that any of the systems designed to date demonstrate acceptable inter-judge reliability.

Readers of the 1999 edition of the Standards are reminded that documentation of the type that accompanies accepted assessment instruments is important because it provides test users “with the information needed to make sound judgments about the nature and quality of the test, the resulting scores, and the interpretation based on the test scores.” It is noted that “[t]he greater the potential impact on test takers, for good or ill, the greater the need to identify and satisfy the relevant standards.” Where the issues in dispute involve parenting plans for children, the “potential impact on test takers” is significant.

Though no published data are available for reference, it is my impression that, more often than not, practitioners who rely upon drawings as an assessment technique offer no statements in their reports that might alert readers to the deficiencies in the technique. In the 1999 edition of the Standards the following admonition appears: “Professionals are expected to make every effort to be aware of evidence of validity and reliability that supports or does not support their inferences and to place appropriate limits on the opinions rendered.”

When practitioners employ what is known as the sign approach to figure drawing interpretation, certain features in the drawings are viewed as signs (indicators) of various personality characteristics, life experiences, or personality characteristics. In commenting on the sign approach, several well-respected writers have concluded that the advocates of the sign method often disagree among themselves concerning what a particular feature indicates.

In reviews of drawing techniques prepared for The Buros Institute and published in the Mental Measurements Yearbook, reviewers have consistently concluded that there are no empirical data to support the use of drawings.

Validity

Most simply put, validity addresses the issue of whether we are measuring what we think we are measuring. If the evaluator's goal is to gather information that will shed light on what parenting plan is most likely to be in the long-term psychological best interests of children and if data are gathered that are strongly influenced by day-to-day events (as poor test/re-test reliability data suggest is the case), those data shed no light on the issue of interest.

Discriminant Validity

Where drawings have been utilized for the purpose of identifying someone as a member of a specific group (for example, sexually abused children), it is particularly important that readers of an evaluator's report be familiar with the concept of discriminant validity. The effectiveness of an assessment instrument or technique in discriminating between two classes of examinees defines that instrument's discriminant validity.

Consider this assertion by an evaluator: “I have found the following [features in a child's drawing] to be suggestive of abuse.” In making such an assertion, the evaluator is claiming that he can discriminate between abused children and non-abused children on the basis of certain features of their drawings. Research has repeatedly shown drawings to be lacking in discriminant validity. Even if there were research findings supporting the contention that a certain feature is often observed in drawings produced by abused children, detection of this feature would be useful only if research also demonstrated that the specified feature is observed exclusively in the drawings of abused children and is not observed in the drawings of non-abused children.

What Is the Data Source?

The methods employed by evaluators in gathering information from children differ dramatically from the methods employed in gathering information from adults. Statements made by individuals of any age are viewed as a data source and evaluators interviewing children often find that play, building tasks, drawing tasks, and similar activities facilitate constructive oral interaction. When drawings are utilized as a stimulus for conversation and where it is the statements made that are conceptualized as the data source, it is unlikely that any harm is created by the use of drawings. It is my contention, however, that significant risk of harm is created when no inquiry is conducted and when the drawings themselves and the evaluator's interpretations of those drawings are viewed as a data source.

Conclusion

The psychologists' Ethical Standard 9.02(a) reads:

Psychologists administer, adapt, score, interpret, or use assessment techniques, interviews, tests, or instruments in a manner and for purposes that are appropriate in light of the research on or evidence of the usefulness and proper application of the techniques [emphasis added].” Section (b) reads, in pertinent part: “Psychologists use assessment instruments whose validity and reliability have been established [emphasis added] for use with members of the population tested.

In jurisdictions in which the Daubert criteria are employed in assessing the admissibility of proffered expert evidence, the rejection of findings and opinions based upon drawings should be a foregone conclusion. Not one of the five Daubert criteria are met: 1) the underlying theory is not falsifiable; 2) the procedure has not been endorsed in peer- reviewed journals in the field of forensic psychology; 3) we have no clue concerning the error rate; 4) the use of drawings as a data source is not “sufficiently established to have gained general acceptance in the particular field” in which drawings are being used; and 5) there are no standards controlling the use of drawings as a data source.

In Frye jurisdictions, seeking the exclusion of expert evidence derived from drawings may require the presentation of a persuasive argument. In 1923, when a Federal Circuit Court ruled on the admissibility of data from a systolic blood pressure deception test ( Frye v. United States , 295 F. 1013 (D.C. Cir. 1923)), it declared that “while courts will go a long way in admitting expert testimony deduced from a well-recognized scientific principle or discovery, the thing from which the deduction is made must be sufficiently established to have gained general acceptance [emphasis added] in the particular field in which it belongs.” Though opposing experts may express different opinions concerning “general acceptance” or how the court should identify the “particular field” in which “the thing from which the deduction is made” belongs, it is my opinion that the recognition, in 2001, by the Commission for the Recognition of Specialties in Professional Psychology that forensic psychology is a specialty means that forensic psychology (in contrast with clinical psychology, school psychology, etc.) is a particular field within the discipline of psychology and that the general acceptance (or lack thereof) of utilizing drawings as a component of one's evaluative methodology must be examined with reference to forensic psychology.

Finally, it is my understanding (as a mental health professional) that when expert opinion testimony is being proffered, the burden of demonstrating evidentiary reliability lies with the proponent of the proffered testimony and that the suitability of the evidence must be established by a preponderance of the evidence. Those who interpret drawings and employ those interpretations in formulating their opinions in custody evaluations have selected an assessment technique that is not suitable for the use to which it has been put.

David Martindale, PhD, is board certified in forensic psychology by the American Board of Professional Psychology, served as the Reporter for the Association of Family and Conciliation Courts' Model Standards for Child Custody Evaluation and is the co-author, with Jon Gould, of The Art and Science of Child Custody Evaluations. His practice is limited to consulting with attorneys, psychologists, and psychology licensing boards in the areas of child custody and professional ethics and standards. He can be contacted by e-mail at [email protected]. This article first appeared in The Matrimonial Strategist, an Incisve Media sister publication of this newsletter.