---

Editor's Note: This month's Highlights from the Field and Family Matters! columns focus on community-driven evaluation. While the following Highlights article gives some step-by-step advice on forming your own Community Evaluation Team (CET), also take a look at our Family Matters! article to find out what family members have to say about their involvement in community evaluation.

As people come together to support youth and families, decisions about what to do have to be made and prioritized. Stakeholders such as families, youth, community members, and professionals should be involved in making these decisions. Community evaluation teams (CETs) provide a forum to bring these stakeholders together to work on the evaluation and continuous quality improvement processes of a system of care.

CETs are also an important part of a program/system's best practices. In theory, programs and systems in system of care communities are driven by consumer needs rather than by available services. By involving individuals whom the program/system serves in the data-based decisionmaking that informs ongoing program development, you help create a direct link between community needs and system services. This is crucial for implementing system of care values and principles.

The CET is also a powerful vehicle for communication and empowerment for parents and professionals alike. Learning how to communicate evaluation findings empowers the evaluation team. The rest of the community is also empowered, both by the information they receive and through seeing youth, family members, and professionals working with data that directly affect their system of care.

I. Background: Developing Allegheny County 's CET 

Community Connections for Families (CCF), a Substance Abuse and Mental Health Services Administration (SAMHSA) system of care grant project in Allegheny County , PA , implemented a CET in the summer of 2001. CET development was spearheaded by Dr. James Rast, a consultant on continuous quality improvement (CQI) who had successfully implemented CET's in the past and felt that the idea could be adapted to the CCF model.

While other CETs, such as the Tampa CET spotlighted in this month's Family Matters! column, consist solely of family members, the CET in Allegheny County comprises youth, community members, and systems partners in addition to family members. These members form a core team to help guide the project in making evaluation and CQI decisions.

II. What are Community Evaluation Teams?

A CET is a group of individuals who come together from diverse backgrounds to discuss and interpret evaluation findings and to devise creative ways to make those findings useful for families, decisionmakers, and communities. CCF CET comprises approximately 20 individuals, including the CCF evaluation team, family members from all five of the partner communities, CCF professionals, and system partner professionals.

The CET provides a forum for parents and professionals to come together in an environment of mutual learning. Some of this learning is formal. For example, members of the CCF CET have participated in several trainings, including the Federation of Families (FoF) World of Evaluation Training and CCF's Evaluation 101. Other aspects of learning emerge through the rich discussions that occur when people of various cultural backgrounds come together.

The CET is responsible for using data to guide decisionmaking about the system of care. CETs are formed to help everyone involved understand what is really happening. This may involve using data as part of advocacy work, teaching others about data, and comparing verbal “real-life” experiences with the system of care to existing data. At CCF, the CET is responsible to CCF's Advisory Committee. By taking a utilization-focused approach to evaluation (i.e., information is collected only if it is useful), CET members inform others, based on data and on their personal experiences, about what is really happening in the system of care. The people on the CET are those who are directly involved in the system of care in some way. This involvement empowers families and members of the community to affect the direction of change and gives current decisionmakers the information that will lead to successful decisions.

III. What a CET Does

A CET can take on several roles and participate in a variety of evaluative activities. CCF is just now realizing that forming subcommittees of the CET can provide a useful way to help members target their areas of interest and stay focused on tasks around these activities:

nterpreting Data—The evaluation staff provides the CET with charts and aggregated numbers. The CET discusses trends they see, how these trends can be addressed, and what the trends might mean.

• Telling Others About Data—CET members are champions of evaluation and quality improvement. That is, they are passionate about the work they do as part of this team. As a direct result, they use data to communicate with other families, professionals, and community members in an effort to create awareness and systems change.
• Training—As members of the CET complete evaluation trainings, they, in turn, train new CET members and others in the community at large.
• Defining Indicators and Benchmarks of Success—The CET can help evaluators and program planners to define indicators of various program aspects and how the program should measure success in those areas. For example, the CCF CET formed a subcommittee to create a mission and to create and define 12 core values. The CET plans to review these values annually for updates.
• Monitoring Language—The CET can monitor outcome reports, surveys, etc., for language changes, thus ensuring the language is family friendly and clear. For example, the CCF CET reviewed and revised for content a child and family team satisfaction survey.

VI. How To Start a CET

Starting a CET is fairly simple, but it does take a lot of planning. Aspects to consider include:

• Times of Meetings—Can family members and professionals attend at the times meetings are held? Should meeting times alternate between morning, afternoon, and evening?
• Location of Meetings—Is the location easily accessible? Is transportation an issue?
• Recruitment—How will you find people to bring to the table? Are they truly interested? Can you recruit from existing family organizations?
• Tasks—What types of tasks can the group do? What will they be responsible for?
• Facilitation—Who will facilitate the group? Will you have parent and professional cofacilitators?
• Training—What trainings need to occur to get the group ready to face the challenges of the tasks they face? How often will training be offered?
• Making It Fun—How will you make the group fun for those who attend?
• Incentives—Will you offer stipends to parents who sit on the team? Can you offer lunch or dinner to participants?

V. Family Involvement and Roles Within the CET

Family members on the CCF CET have helped to choose which topics the team should address. For example, the CET is currently concentrating on education data, such as dropout rates, detentions, and suspensions, so that team members can educate families on how to use these numbers when advocating for their children in schools. Family members have also helped define CCF's mission and values and are crucial to the monitoring of those values. Finally, they have helped change language on surveys so that it is clear and family friendly.

Family members who join the CET can take on several roles:

• Collaborators—Families work with professionals to foster an atmosphere of mutual learning and respect (a “no shame, no blame attitude”).
• Decisionmakers—Family members help make decisions about areas the program can target for quality improvement activities, which trainings to offer, etc.
• Drivers—Family members may drive discussion around evaluation information, set agendas, and facilitate the CET and/or its subcommittees.
• System Monitors—Having family members at the table to interpret data provides an invaluable perspective on the system; without families’ help, professionals may overlook this perspective.
• Trend Setters—Family members are leaders by communicating their understanding of data and CQI.

VI. Lessons Learned

CCF has learned, and continues to learn, many things from the implementation of a CET. Some of the most important lessons gleaned from the process are described below.

• Time and Hard Work—Planning is crucial, and good planning takes time. It may take 4–6 months to develop a core team of individuals dedicated to the evaluation process. It also takes time to allow the team to decide what they are interested in and where they would like their focus to be. It is important to remember that working and evaluating as a team is an ongoing learning experience that it is well worth the rewards in the end.
• Ongoing Training— One lesson learned at CCF was that some family members on the CET were ahead of the professionals in terms of their knowledge of evaluation. This happened because several family members attended the FoF World of Evaluation Training. Hence, CCF devised an Evaluation 101 curriculum for professionals and family members who did not have the opportunity to attend the FoF training.
• Keeping Focused— It is important that the group continually has defined tasks so that it keeps its momentum and stays focused. Evaluation and quality improvement are broad topics, and they need to be narrowed and planned out so that the group can be productive.
• Administrative Support— Having the support of key administrators from the program or system plays an important role in the CET's success. With administrative support, the CET has greater credibility and also has a place to voice issues. This support also increases the CET's opportunities to implement quality improvement change.
• Champions— Leaders are important to the success of any team, system, or program. Every member of the CET is a leader and champion of evaluation and quality improvement in a system of care. These dedicated members and active participants make the CET a valuable asset to the system of care.

VII. Next Steps: Raising the Bar