Evaluation

Learning About Ourselves
Evaluation of CASA Representation by Caliber Associates

We have a longstanding commitment to measuring success in our work.  But measuring success is complicated. There are many factors, often outside our control, that can affect a child's chances of success in life. There are also many ways our volunteers may impact a child's life which do not easily lend themselves to documentation.

At this year’s national conference, we reported on the results of a national evaluation of CASA programs.  The study, titled “NCASAA Evaluation Project,” was conducted by Caliber Associates.  Included in the study were 25 programs that submitted data from their COMET programs for analysis.  In addition to the COMET data sites, the study also compared data from the National Survey of Child and Adolescent Wellbeing, sponsored by the Children’s Bureau, in an effort to look at wellbeing indicators not usually available on a local program level.

Some findings from the study describe what we already know.  For example, judges tend to assign CASA volunteers to the most difficult cases.  The study found that the children to whom CASA volunteers are assigned are much more likely to be at severe risk than those who do not get assigned a volunteer. 

There are also some findings that are very positive in terms of volunteer impact.  The study found, for example, that CASA volunteers are “highly effective” at making recommendations to the courts.  Also, consistent with the National CASA standards, volunteers spend the largest portion of their time in contact with the child.

In addition, children with CASA volunteers—as well as the children’s parents—received more services, though more research is needed to determine if this was due to CASA advocacy.  The finding is positive for CASA children, because they are the ones who need services the most.

Other findings might appear to be neutral, until you look at the study more closely.  The study found that the CASA cases were in the system about as long as the non-CASA cases.  This is not surprising, given that the CASA cases were more serious.  It might have been expected that these children would have spent longer in care without a volunteer.

On several measures of child well-being, the study found that cases involving CASA volunteers were similar to the non-CASA cases, despite the difference in case severity.  While it is valuable to learn that differences could not be documented, we are even more interested in exploring further ways to do just that.  Again, because the CASA cases were more serious, this finding does not reflect badly on our work.

Finally, there are findings that are potentially negative, though not fully explained in the study.  One has to do with volunteer hours; the study reported surprisingly low hours per child per month—about 3.2 hours.  To anyone who knows CASA, this is an absurdly low figure.  Our annual national survey consistently shows volunteers putting in at least twice as much time.  The researchers think this is a problem of missing case activity in the COMET records, not a true reflection of the time volunteers spent.

This finding does suggest that we have to have to look at documenting volunteer hours.  However, we know that CASA volunteers are heavily involved in child-related case activities, and the last thing we want to do is to turn them into data input people.  This is an issue which will need further discussion over the coming months.

Another finding within the Caliber Evaluation reports differences in time spent on African American children’s cases. To better understand this finding, we commissioned further follow up analysis by Caliber Associates. This additional analysis shows that the race of the volunteer is not significant in determining the number of hours spent per month on cases, and that African American families within the study have a significantly higher number of siblings, which would help explain why less time might be spent per child.  Finally, there are statements in the report that can easily be misinterpreted.  One is that CASA kids are more likely to be in foster care than non-CASA kids.  This is not surprising, since it is generally children in foster care to whom we are assigned in the first place.  CASA involvement did not cause this placement into care.

Another way to look at the value of what we do is to ask key participants in the child protection process what they think of our work.  Last year, we reported on the completion of a “National CASA Consumer Satisfaction Survey” by Dr. Pat Litzelfelner.  That study surveyed judges, lawyers, biological and foster parents, and caseworkers about the work of CASA volunteers.  Not surprisingly, judges and lawyers were very satisfied with our work.  What I found very interesting, though, was that the parents of abused and neglected children also very much appreciate CASA volunteers.  Parents ranked CASA volunteers positively on every measure, including their helpfulness to the parents themselves.

This may surprise some who are unfamiliar with CASA.  But an understanding approach to families, while maintaining objectivity and independence, are key values of the program.  A statement in the current CASA volunteer training summarizes the organization’s policy: “It is important to be objective, to remain detached enough to see clearly the child’s and family’s situation and needs.”

This objective yet independent orientation shows in the results of the cases in which volunteers are involved.  According to our annual program survey, our cases close most often because the parents are able to have the children returned to them.

Some keys to the volunteer’s role are their ability to focus on just a few children at a time, the time they have available for a variety of activities (most importantly, spending time getting to know the child), and thinking critically about everything in the child’s life when making recommendations to the court.  One of the great things about the CASA/GAL network is that we ask critical questions not just in the cases, but about ourselves.  That is what we did in this evaluation.  The results tell us we need to keep doing just that; challenging ourselves and asking the kinds of critical questions that help us constantly improve our work for children.

To follow up on these research studies, we are developing a new research agenda to explore some of the unanswered questions and—just as importantly—to identify best practices to improve our advocacy based on this work.

Click here for research summary (PDF)
Click here for full report (PDF)

If you have any questions about this study, please contact Jim Clune, Communications Chief staff@nationalcasa.org