Youth Frontiers Evaluation Overview
Brooklyn Center Evaluation Project
Scope of project: A three-year partnership funded by the Charlson Foundation which provided for the delivery of all YF retreats to both students and educators in the Brooklyn Center school district. The goal of the evaluation of this project was to determine the impact of YF programs on school climate, student character, and staff morale.
Key Dates: Year 1 (spring 2006, and school year 2006-07), Year 2 (2007-08), Year 3 (2008-09).
Data Collection Strategies: Student and teacher perceptions regarding the impact of the YF student retreats were collected by means of survey research from students and educators who participated in the retreat experiences. Survey tools used were:
- School as a Caring Community Profile (SCCP-II)
- 7-day post retreat questionnaires
- 60-day post retreat questionnaires
Key Findings / Measures of Impact:
Students learn how to treat each other better. (Teach Character)
- 69% of students on the Kindness Retreat felt that they had learned new skills and strategies to better handle bullying behavior.
- 1/2 of students believe that their class is a kinder or more respectful class than before the retreat experience.
Students are more engaged and willing to stand up for others. (Engage Bystanders)
- Over 1/2 of the students on the Courage Retreat reported that they felt more empowered to do what they believe is right and act with courage in moments of decision.
- 1/3 of students believe that there has been a reduction in bullying.
- 73% of students on the Kindness Retreat felt more connected to their class since the retreat.
Student discipline referrals and incident reports decline.
- During the first two years of this project, discipline incidents and discipline referrals declined by nearly 60%.
YF delivers a consistently high quality retreat experience.
- 89% of students on the Kindness Retreat, 76% of students on the Courage Retreat, and 76% of students on the Respect Retreat rated the retreats' overall quality as "Excellent" or "Very Good."
Data Analysis: Information regarding attendance, academic achievement data such as the MCA-IIs (Minnesota Comprehensive Assessments), and data from the school district's climate survey will be included in the final report when the district's Accountability Report is published in October 2008.
Project Evaluator: Edmund Sass, Ed.D, Professor of Education, College of Saint Benedict/Saint John's University.
YF School Client Evaluation Project
Scope of project: YF will better understand how and why schools utilize YF programs; and to what extent schools could report on the sustainable effects of the retreat experience.
Key Dates: Client interviews conducted March - May 2008; final report and analysis in progress.
Data Collection Strategies: Client interviews were conducted with YF clients (schools) who had used YF retreats for at least two years. Interviews were conducted by a YF staff member (in person or over the phone) and the primary contact at a school - typically the principal, assistant principal, dean, classroom teacher, counselor, or social worker.
Key Findings / Measures of Impact:
Most common reported trends or themes
- YF expands the common vocabulary of students and teachers.
- Students receive appropriate repetition and reinforcement of content on retreats. (They get to practice how to stand up for themselves or others.)
- YF retreats assist teachers with behavior management and problem solving strategies.
- YF retreats are intentionally woven into the school's curriculum and instruction.
- YF complements and enhances schools' instructional strategies and/or core values.
- YF retreats support the Social and Emotional Learning framework.
- YF retreats build community and build bridges between students.
- Peer-peer education is powerful and has a lasting impact.
Data Analysis:
Inter-rated reliability substantiates the validity of these findings. In essence, if several school principals independently report on the positive, sustainable effects of the YF retreats, this is statistically significant.
(Additional supporting studies of literature in progress.)
Project Evaluator: Bill Johnston, independent research consultant and adjunct professor of research methods at the Hamline University Graduate School of Management.
Upcoming CAREI Evaluation Project
YF is partnering with CAREI (Center for Academic Research for Educational Improvement) at the University of Minnesota. CAREI is an internationally recognized leader in the field of educational research and evaluation.
- The CAREI evaluation is a three-year study (2008-2011), which will measure the impact of YF programs on behavior, attitudes, and perceptions as they pertain to efforts to create positive school communities and help students realize the importance of living a life of character.
- A variety of data collection strategies will be utilized over the course of the three-year project.
- CAREI will submit annual reports that will summarize evaluation activities and preliminary findings coupled with supporting and relevant review of literature.
CAREI background:
CAREI's mission is to improve the quality of education for all learners through neutral, rigorous research and evaluation of educational contexts and initiatives.
Some of CAREI's research and evaluation projects in progress are being conducted for St. Paul Public Schools, Minnesota Department of Education, The Children's Theatre Company, and the Wallace Foundation.
Dr. Kyla Wahlstrom, Director of CAREI, wrote the evaluation plan for YF. Dr. Wahlstrom will also be overseeing this project and authoring the final report analysis.
YF Evaluation Summary as of August 2008
Lessons we have learned or lessons that have been reinforced by our recent evaluation efforts:
- Schools are intentionally using our retreats and programs to complement and enhance their curriculum and instruction.
- Teachers recognize that they are primarily responsible for following up on the retreat content after the retreat experience.
- Teachers are using very creative ways to call upon the lessons learned and the language taught at the retreat. (We are a touchstone.)
- Students are learning new skills at the retreat and are feeling empowered by having the time during the retreat experience to practice those new skills. (Closing activity)
- Students leave the retreat feeling better about themselves and their peers.
- The retreat leaders (upperclass students) are great role models and peer educators. This helps to build community that permeates throughout the entire school.
- Kids remember the retreat.
- School climate is improving when we analyze key indicators such as the decline in discipline incidents, improved student attendance, and growth in academic achievement.
- Schools acknowledge that partnering with YF is critical in building community and improving school climate.
|
