Family and Youth Collaboration FAQ September 2007

September 2007

Question:
Back in February you provided some information on building the capacity to have a structured and monitored process across systems to incorporate family and youth involvement at all levels of our system of care. Thank you so much for that information as it has proven to be very helpful as we begin to partner with professionals and providers in our community. There is an emerging issue related to evaluation and the importance of families and youth being involved in that process as well. In our community we have 2 full-time positions to staff the evaluation team. Our Project Director has stated that one of those full time slots had to go to a person who had the appropriate credentials to serve as the Lead Evaluator and the other position could be considered for family participation in evaluation. We have had a discussion to divide the other full-time slot into part-time positions to facilitate hiring 2-3 family members to be a part of the evaluation team. Should I utilize the previous information on family involvement or do you have different information regarding families and youth in evaluation?

Answer:
First of all we’re glad that you found the February 2007 information in the newsletter that related to family involvement and their roles helpful. In that article we did mention that evaluation was one of the key areas where family and youth involvement was crucial. For the past 10 years or more, communities have been expected to involve family members and youth in the evaluation process. One of the three core values in a system of care is that the service system be family focused, family driven and child-centered. This particular value holds system of care sites accountable to being family-driven systems to involve family members in not only planning for their child’s services, but to also hold them accountable to involving families at all levels. Successful systems of care communities have families involved at the administrative level which involves budgeting, planning, policy making, and evaluation. To get an understanding of evaluation let’s look at the Federation of Families for Children’s Mental Health’s definition. What is evaluation? Evaluation is a very specific process used to judge how well a program, service, or system is working. Different from our other ways of knowing and judging, such as intuition or opinion, it is a formal and systematic inquiry resulting in a judgment. The results are intended to be used to improve the program, service, or system evaluated.

Where did it come from and why is it so important?

Evaluation is important to families and youth because it is used to identify which programs work, how they can be improved, and what should be funded. Evaluation drives resources and is a key component of all social programs – public and private. Now that we are aware of the definition of evaluation and its origin and importance, let’s take a journey into how families and youth could be involved, as well as some of the challenges that could come about.

A number of benefits of involving family members and youth on evaluation teams have been identified. Teams have found that family/youth members involved in conducting the evaluation and research share common experiences with those who are part of the evaluation project. These family evaluators are more likely to gain the trust of the research participants and they are more likely to get accurate information from participants. Family evaluators can share their insights with the team in order to verify findings and seek explanations for unexplained results. In addition, family participation in research and evaluation improves the usability of instruments, and the information gathered is more relevant to the needs of the families. Dissemination efforts can be enhanced by the involvement of family members. Families can help to design more user-friendly formats for findings that increase the dissemination of results to a wider range of people, especially other family members. While some researchers and evaluators may recognize the value of working with family/youth evaluators on research and evaluation teams, few have experience working with family members in this capacity.

Traditionally, evaluators and researchers have not had experience working closely with people experiencing the issues that are the focus of their research. In the traditional research model, evaluators and researchers are trained to be objective scientists, keeping control of the research process and distancing themselves from the “subjects,” i.e., those who were being researched. In this traditional model, the participants make no contribution to the formulation of the research questions. In contrast, the new approach to family/youth-evaluator partnerships in systems of care requires evaluators to collaborate with family members and youth in all aspects of the evaluation of services.

Benefits of family/youth involvement in evaluation:

The information is authentic and useful.
Families/youth that are interviewed feel comfortable and valued by being interviewed and having data collected by someone who has “been there and done that” and feel less scrutinized, not blamed and shamed when interacting with a fellow family member or youth. Families and youth can add a level of trust to the interview process.
Family/youth partners can be a great resource when data needs interpretation, especially from a family/youth’s point of view.
Families/youth are very well received when being part of a team that does presentations. They are able to connect with other family members or youth who may be participants or make it more real for professionals who may be in the audience.

Partner Roles for families/youth in evaluation:

Partnering in the development of the projects;
Partnering in developing instruments and surveys;
Collection of data;
Presenting data;
Partnering with the dissemination of the data;
Partnering in the interpretation of the data;
Partnering with the analysis of the data;
Training other family/youth evaluators;
Equal partners in decision making;

Challenges/barriers for families/youth in evaluation:

Evaluators not trained to work with families and youth;
Hiring policies are based on academic qualifications;
Concerns that family and youth involvement affects their ability to be objective;
Scheduling of meetings conflicts between family/youth and professionals;
Treatment of families and youth should be respectful, but hierarchical mindsets still exist;
Families and youth getting adequate salaries or timely reimbursements;
Professionals' anticipation of stress related to families and youth having dual roles as advocates and evaluators;
Failure of the broader community (i.e. providers) to value family/youth’s role on evaluation teams;
Families/youth are reluctant to get involved with research and evaluation.

Helpful strategies for evaluators:

Recognize and appreciate the strengths of family/youth evaluators;
Willingness to work as a team;
Willingness to communicate openly about differences in perspectives;
Willingness to have a flexible working environment;
Willingness to conduct on-the-job training and technical assistance to family and youth members of the team;
Promote a culture of mutual learning.

Here’s hoping that your questions around involving families and youth in evaluation have been answered, and if not, please feel free to contact us here at the Technical Assistance Partnership at gpalmer@ffcmh.org or kwilliams@ffcmh.org

And of course you may contact Elaine Slaton, the Director of Evaluation and Training at the Federation of Families for Children’s Mental Health at eslaton@ffcmh.org.

Technical Assistance Partnership for Child and Family Mental Health

Family Involvement and Advocacy Frequently Asked Questions

Home About Us About Systems of Care Contact Us Events & Presentations Newsletter Consultant Pool Special Topics Column Grantee Locator Webinars Resource Specialties Child Welfare Education Family Involvement Juvenile Justice Mental Health Primary Care Substance Abuse Sustainability Youth Involvement Regional Technical Assistance Region I Region II Region III Region IV Region V Region VI Communities of Practice Cultural and Linguistic Competence Early Childhood Transition-Aged Youth	Family Involvement and Advocacy Frequently Asked Questions
	September 2007 Question: Back in February you provided some information on building the capacity to have a structured and monitored process across systems to incorporate family and youth involvement at all levels of our system of care. Thank you so much for that information as it has proven to be very helpful as we begin to partner with professionals and providers in our community. There is an emerging issue related to evaluation and the importance of families and youth being involved in that process as well. In our community we have 2 full-time positions to staff the evaluation team. Our Project Director has stated that one of those full time slots had to go to a person who had the appropriate credentials to serve as the Lead Evaluator and the other position could be considered for family participation in evaluation. We have had a discussion to divide the other full-time slot into part-time positions to facilitate hiring 2-3 family members to be a part of the evaluation team. Should I utilize the previous information on family involvement or do you have different information regarding families and youth in evaluation? Answer: First of all we’re glad that you found the February 2007 information in the newsletter that related to family involvement and their roles helpful. In that article we did mention that evaluation was one of the key areas where family and youth involvement was crucial. For the past 10 years or more, communities have been expected to involve family members and youth in the evaluation process. One of the three core values in a system of care is that the service system be family focused, family driven and child-centered. This particular value holds system of care sites accountable to being family-driven systems to involve family members in not only planning for their child’s services, but to also hold them accountable to involving families at all levels. Successful systems of care communities have families involved at the administrative level which involves budgeting, planning, policy making, and evaluation. To get an understanding of evaluation let’s look at the Federation of Families for Children’s Mental Health’s definition. What is evaluation? Evaluation is a very specific process used to judge how well a program, service, or system is working. Different from our other ways of knowing and judging, such as intuition or opinion, it is a formal and systematic inquiry resulting in a judgment. The results are intended to be used to improve the program, service, or system evaluated. Where did it come from and why is it so important? Evaluation is important to families and youth because it is used to identify which programs work, how they can be improved, and what should be funded. Evaluation drives resources and is a key component of all social programs – public and private. Now that we are aware of the definition of evaluation and its origin and importance, let’s take a journey into how families and youth could be involved, as well as some of the challenges that could come about. A number of benefits of involving family members and youth on evaluation teams have been identified. Teams have found that family/youth members involved in conducting the evaluation and research share common experiences with those who are part of the evaluation project. These family evaluators are more likely to gain the trust of the research participants and they are more likely to get accurate information from participants. Family evaluators can share their insights with the team in order to verify findings and seek explanations for unexplained results. In addition, family participation in research and evaluation improves the usability of instruments, and the information gathered is more relevant to the needs of the families. Dissemination efforts can be enhanced by the involvement of family members. Families can help to design more user-friendly formats for findings that increase the dissemination of results to a wider range of people, especially other family members. While some researchers and evaluators may recognize the value of working with family/youth evaluators on research and evaluation teams, few have experience working with family members in this capacity. Traditionally, evaluators and researchers have not had experience working closely with people experiencing the issues that are the focus of their research. In the traditional research model, evaluators and researchers are trained to be objective scientists, keeping control of the research process and distancing themselves from the “subjects,” i.e., those who were being researched. In this traditional model, the participants make no contribution to the formulation of the research questions. In contrast, the new approach to family/youth-evaluator partnerships in systems of care requires evaluators to collaborate with family members and youth in all aspects of the evaluation of services. Benefits of family/youth involvement in evaluation: The information is authentic and useful. Families/youth that are interviewed feel comfortable and valued by being interviewed and having data collected by someone who has “been there and done that” and feel less scrutinized, not blamed and shamed when interacting with a fellow family member or youth. Families and youth can add a level of trust to the interview process. Family/youth partners can be a great resource when data needs interpretation, especially from a family/youth’s point of view. Families/youth are very well received when being part of a team that does presentations. They are able to connect with other family members or youth who may be participants or make it more real for professionals who may be in the audience. Partner Roles for families/youth in evaluation: Partnering in the development of the projects; Partnering in developing instruments and surveys; Collection of data; Presenting data; Partnering with the dissemination of the data; Partnering in the interpretation of the data; Partnering with the analysis of the data; Training other family/youth evaluators; Equal partners in decision making; Challenges/barriers for families/youth in evaluation: Evaluators not trained to work with families and youth; Hiring policies are based on academic qualifications; Concerns that family and youth involvement affects their ability to be objective; Scheduling of meetings conflicts between family/youth and professionals; Treatment of families and youth should be respectful, but hierarchical mindsets still exist; Families and youth getting adequate salaries or timely reimbursements; Professionals' anticipation of stress related to families and youth having dual roles as advocates and evaluators; Failure of the broader community (i.e. providers) to value family/youth’s role on evaluation teams; Families/youth are reluctant to get involved with research and evaluation. Helpful strategies for evaluators: Recognize and appreciate the strengths of family/youth evaluators; Willingness to work as a team; Willingness to communicate openly about differences in perspectives; Willingness to have a flexible working environment; Willingness to conduct on-the-job training and technical assistance to family and youth members of the team; Promote a culture of mutual learning. Here’s hoping that your questions around involving families and youth in evaluation have been answered, and if not, please feel free to contact us here at the Technical Assistance Partnership at gpalmer@ffcmh.org or kwilliams@ffcmh.org And of course you may contact Elaine Slaton, the Director of Evaluation and Training at the Federation of Families for Children’s Mental Health at eslaton@ffcmh.org.