Quick Jump to Article Topic Headings:
 Problem
Proposal
Process
Postlude
Outcomes
Problem
Concern over poorly performing public services is growing. People believe that our public
systems, including education and social services, are failing.
A common response is to demand more regulation. We assume that rules will promote
effective programs by guarding our resources and interests. This focus on control leads us
to measure success by how well the rules were followed. Accountability is determined by
asking, "Did you do what they told you to do?"
Current program evaluations measure such things as attendance or the number of services an
agency offers, rather than evaluating any changes in client status. For example, a local
health clinic may claim that it now serves more children, but that does not necessarily
mean that more children are healthier.
Proposal
Outcome evaluation is regulation by results. The accountability question becomes,
"Did it work?" Instead of auditing regulatory compliance or counting the number
of customers served, outcome evaluation tracks actual changes in status or condition. A
program is successful if it improves the well-being of the consumer.
Using results to gauge success may sound simple, but defining goals, finding reliable
measurements, and getting support can be difficult. Understanding the issues, obstacles,
and benefits of outcome evaluation is the first step for agencies that want to provide
more effective services for their customers.
Process
(7 points)
1.Pick Your Partners
Outcome evaluation usually requires collaboration. Everyone with a stake in the results
should participate, including formal support agencies (child welfare, schools, health) and
informal support groups (neighborhood groups, churches, youth groups). Most important,
make sure you involve consumers of the service; they know the most about what they need.
For more information on building collaborations, see the Building Blocks for Change"
report, "Collaborating for Missouri's Children." (Order information below or see
HandsNet: Children, Youth & Families/Comprehensive Strategies/Collaboration/How to
Start a Collaborative, 7/12/95).
Top
2.Define Your Vision, Select Outcomes and Indicators
You are the architects of a new community. Create a rendering that illustrates your shared
vision. Draw a picture of the way things should be. What living conditions are acceptable?
What skills do successful adults need?
As architects you must develop a blueprint that specifies the elements required for
construction. Just as a house needs electrical, plumbing, and ventilation systems, a
successful adult needs good health, a solid education, and marketable job skills. These
core outcomes or goals are the foundation for your vision.
There are an infinite number of potential outcomes. You must set priorities, focusing on
the most critical areas. Stress goals that are important to both supporters and skeptics.
Choose outcomes that the general public will understand. Include positive (developmental)
and negative (problem reduction) goals. Avoid outcomes that are vague or too ambitious
like "increasing the ability to appreciate diversity."
Now that you have a vision and a blueprint, you need a construction schedule to mark your
progress. In outcome evaluation these markers are called component indicators. They
provide evidence of current status and measure changes in status over time.
Progress on a physical project, like a building, can be assessed just by looking at the
site, but progress in the field of social services is much more difficult to measure. The
indicators that you select are critical to tracking your progress reliably. Avoid
measures, like attendance figures, that do not show results. Pick measures that are
compiled by reputable sources. For example, if increasing the number of healthy births in
your community is a core outcome, one indicator is the number of low-birthweight births
that occur in your area. This data is recorded annually by your state health department, a
source that is credible to most interested parties and the general public.
The Center for the Study of Social Policy has a list of sample core outcomes and
indicators for children and their families. A portion of this list follows the text of
this article.
Top
3.Collect the Information
Collect current data for each indicator. The initial data compilation gives a clear
picture of the current status and serves as a baseline for evaluating progress.
Make sure that the same information will be collected in the future so that you can make
comparisons over time. Ask how often the information is updated. This knowledge will help
you plan ongoing program assessment.
If there is no reliable data source, you may need to find other indicators, or note
weaknesses when publicizing results. Your group can encourage the collection of additional
data on important indicators, like immunizations for children under two.
Top
4.Service Plan
The next step is to determine the resources needed to achieve the outcomes. Look for
programs and services that have a demonstrated impact on your outcomes. Make sure you
understand what elements make these services effective so that you can apply them to your
unique environment. Canvass the targeted area and develop a map of services that already
exist and their locations. This map will provide you with invaluable information about the
community, its culture and values, its resources and deficits.
Building Blocks for Change, "Going Places with Community Mapping," offers
detailed instructions for completing a service map of your area. (Order information below,
or see HandsNet: Children, Youth & Families/Comprehensive
Strategies/Collaboration/How-to Comm. Svcs. Mapping, 3/9/95)
You can now develop a plan for expanding the capacity of existing services, adding new
services and eliminating ineffective ones. Specify the tactics you will employ to achieve
your goals. This plan is your strategic vision for service delivery.
Top
5.Develop Resources
Implementation of your strategic service plan will depend on the availability of
resources. All stakeholders need to contribute in some way: clients donate labor, public
and private agencies supply services or financial support, and the general public
furnishes goods or approval for funding.
Creative financing strategies must be investigated. State, county, and local resource
should be tapped. Local businesses may be willing to fund or underwrite specific projects.
Find private foundations that have a history of supporting programs similar to those you
have identified.
In some cases, collaborative partners have contributed funds to a single pool. That pool
was tapped to pay for services. As groups merge funds, funding sources will begin to
target specific outcomes instead of individual groups. Some funders are already moving in
this direction.
Don't forget other types of resources like labor or in-kind services. Local colleges or
universities may donate student or faculty time. They may happily undertake data
collection and analysis tasks.
6.Realistic Expectations
Shifting to outcome evaluation is a long term process. Do not expect change overnight.
Your community may not be ready for a change of this magnitude. Start with your own
agency. Present the case to your funders. Educate other service providers.
Even if the community is ready and you have convened your collaborative, expect an
extended process. It may take two years just to agree on outcomes, indicators, and funding
mechanisms. Keep the momentum going by setting intermediate, achievable goals that move
towards the chosen results. Building capacity for quality prenatal services, for example,
is a step toward increasing healthy births.
Consider relevant community conditions. The level of neighborhood safety or the
availability of adequate, affordable housing will impact how prepared children are for
school. Planning must also acknowledge the impact of external factors, like inflation or
low interest rates, that are beyond community control.
Adopting outcome evaluation does not mean that all regulations are obsolete. Retain rules
that safeguard against fraud, poor service, or discrimination.
Top
7.Publicize Results
Agencies depend on public backing. Outcome evaluation provides a unique opportunity to
develop broad-based support within the community.
Cultivate relationships with members of the media. Some will be sympathetic to your
efforts; others will need to be educated about the issues. Work with both to get your
story out in print, on radio, and on television.
You may begin by announcing the formation of your collaborative. Explain what groups are
involved and what you are trying to accomplish. This stimulates community input from the
very beginning.
Keep the information flowing as you work on defining outcomes. Issue periodic news
releases and make sure they are used. Your project will gain visibility. This also
encourages public interaction.
Present outcome goals to the community. Stage a news conference for the announcement.
People are more supportive when they understand the objectives.
Use your baseline information to illustrate the current status of the community. This
gives a snapshot of local strengths and weaknesses. Broadcasting this picture raises
community awareness of the issues.
After the publication of the baseline information, explain your plan for improving
outcomes. Be as specific as possible. Include tasks, responsibilities, and time frames.
This is to ensure that the public understands what to expect.
Publish a schedule for periodic testing to compare benchmarks and stick to it. Continuous
information keeps the community involved. As progress is made, public trust and support
grows.
Be honest about results. Indicators may look worse before they look better. If potential
dropouts stay in school, the dropout rate may improve while test scores fall. Candid
evaluation increases your credibility and the collaborative gains political clout.
Advocate for outcome evaluation at the city, county, state, and federal level. As such
evaluations become more common, policy will support changes in service and funding
practices. Promotion of results-oriented evaluation principles will spawn additional
publicity and support.
Top
Postlude
•Judging programs by their effectiveness makes sense. It also provides some other
distinct advantages to current practice.
•Outcome evaluation encourages cross-systems collaboration. As agencies work
together, they eliminate barriers to effective service.
•Programs driven by results allow professionals to use their judgment, based on
experience. This flexibility improves service response to the needs of children and
families.
•Demonstrable benefits translate into increased resources. Communities support
productive efforts. Funders see tangible, meaningful results. In competition for scarce
resources, agencies willing to be held accountable for achieving agreed-upon outcomes will
have an advantage.
•Child welfare, social services, and mental health systems that support children only
touch a small portion of their daily lives. Programs must gives families the tools to
sustain themselves for the rest of the day. As families gain strength, they will learn to
function independently within their community. Outcome evaluation is a way to determine if
services effectively prepare family systems for self-sufficiency.
•In turn, the family system is supported by the development of a shared, community
vision. By linking families to their environment, outcome accountability promotes a
"culture of responsibility" that lets whole communities take charge of the
well-being of their children.
Outcomes and Indicators for Children and Families
Core Outcome: More Healthy Births Component Indicators:
•Lower rates of low birthweight births •Lower rates of late or no prenatal care
•Lower rates of births to school-age mothers and fathers
Core Outcome: More Children Ready for School Component Indicators:
•Immunizations complete •No preventable or untreated health problems including
vision and hearing defects •Children not abused or neglected •Children living in
own family or stable foster care
Core Outcome: More Children Succeeding in School Component Indicators:
•Academic achievement measures •Lower rates of dropout, truancy, grade
retention, suspensions and expulsions •Appropriate receipt of special education
services
Core Outcome: More Youngsters Avoiding Risks Component Indicators:
•Lower rates of school-age parenting •Lower rates of substance abuse •Lower
incidence of involvement in violent or criminal behavior including abuse, suicide,
homicide, drug dealing •Lower rates of idleness (not in school/not employed)
------------------------------------------------------------------------
For
more information, or to order the Building Blocks" series, contact:
Citizens for Missouri's Children
------------------------------------------------------------------------
Date posted with permission: Mon Sep 11 10:03:58 1995
HandsNet on the web is a service of HandsNet, Inc.
|
