Youth and Families Serving Youth in At-risk Environments How Are We Doing? by Barbara E. Schlatter and Marta K. Moorman Haven't you heard? It's the buzz-phrase of the 1990syouth at risk. There is an urgency among recreation professionals today to address the needs of youth in at-risk environments through recreation programming. This is not a new idea. At the turn of the century, the harbingers of the public park and recreation movement began providing meaningful activities for children who resided in settlement housing. Families who migrated to the cities from rural areas sought work in factories. Many of the children worked in the same factories as their parents. The problem was that the children did not know how to spend their free time once their factory jobs were over for the day. Park planners began designing parks specifically with the needs of these "at-risk" children in mind. Our definition of at-risk youth is probably not much different from the way our park and recreation forbearers envisioned the youth of the late 1800s and early 1900s. Today, however, we consider youth at risk as those who experience personal, family, economic, or community situations which may increase the possibility of negative behaviors. Examples of negative behaviors include getting in trouble in school, dropping out of school, joining a gang, becoming sexually active, and using alcohol and drugs. These behaviors can lead to criminal activity, substance abuse, teen pregnancy, and sexually transmitted diseases. Programming for At-risk Youth Programs that target youth in at-risk environments are designed with the intent to provide meaningful activities and positive role models. These activities are often recreational in nature; however, it is not uncommon to see a room full of kids doing their homework at a recreation community center. Some recreation agencies are attempting to meet the needs of at-risk youth by providing tutorial programs, counseling services and computer labs, in addition to traditional recreation programming. The primary goals of these programs are to improve self-esteem and help individuals develop life skills. These lofty goals sound great on paper, but measuring the extent to which they are achieved is another matter. The Problem of Evaluation The question is, How successful are these programs in terms of meeting the needs of at-risk youth? Are the programs effective in helping kids stay out of trouble with the law, stay out of gangs, or stay substance-free? These important questions must be incorporated into evaluation methods during program development. Too often, however, the evaluation methods used to measure program success are simplistic and non-comprehensive. By the time a program is almost over, evaluation of the program is discussed as an afterthought. Evaluation plans are often thrown together with little thought, sometimes on the next to the last day of the program! The results include participant attendance records, summaries of participant and parental opinions about the program, and financial reports which indicate funding increases made to those "successful" programs. These evaluation methods tell us little about the overall success of the program. To claim that a program is successful, we must be able to document changes in a child's knowledge, attitudes, and behaviors as a result of their participation in the program. Never could this be more crucial for programs serving at-risk youth. Hence, appropriate methods must be designed simultaneously as programs are created. Evaluation should not Illinois Parks & Recreation July/August 1996 29 be an afterthought. Rather, it should be an integral part of the program development process. A Review of Program Evaluation We live in an era in which accountability is essential. Long gone are the days of programs that are sustained solely by an agency's good name. Instead, measurable goals and objectives are now the norm. Appropriate data must be collected and analyzed, and the results must be related to the goals and objectives. A key component of accountability is evaluation. Evaluation is used to measure the extent to which program goals and objectives are met. This is generally done in two ways: process evaluation and summative evaluation. Process evaluation occurs during the course of the program with the intent of improving the program before it is over. An example is the evaluation of program structure. Is the noncompetitive basketball league for fifth and sixth graders truly noncompetitive or has it gravitated towards competition? This type of evaluation occurs throughout the duration of the program to ensure that the program goals and objectives are being met. Summative (or outcome) evaluation is conducted after the program is over. It involves the collection and analysis of data regarding the overall effectiveness of the program measured against its stated objectives. Evaluation of Programs Serving At-risk Youth There are several ways to improve evaluation methods for programs that serve youth in at-risk environments. First, process evaluation should include perceptions about specific programs from advisory board members, program staff, participants, and parents. Also, qualitative methods of process evaluation such as program observation should be used throughout the course of the program (3). Summative (outcome) evaluation should be systematic in nature and should measure the extent to which participant behaviors and attitudes have changed. This can be done by asking participants, local officials, program leaders and parents about their perceptions of participant behaviors. Information that is outside the realm of the community recreation agency should also be obtained (i.e., results from standard achievement tests, arrest and probation records or police records, and academic grades) (3). Many program planners agree that incorporating these evaluation methods into program development provides more comprehensive information about overall program success. It is also fair to suggest that many planners feel that doing so requires extra time and effort, which they may not have. A Survey of Illinois Park Districts A telephone survey was conducted in September of 1995 to determine the extent to which Illinois park districts with programs targeting at-risk youth use the evaluation methods described above. It was hypothesized that recreation and park programs that serve at-risk youth rely primarily on evaluation methods such as head counts, parental perceptions of the program, and participant perceptions of the program. The 1994-1995 IAPD/IPRA Membership Directory and Buyers' Guide was used to select a random sample of 25 percent. Of the 263 park districts listed, 66 were selected using a table of random numbers. Eight of the 66 districts did not respond to three phone-call attempts and therefore were dropped from the sample, leaving a total of 58 in the sample.
30 Illinois Parks & Recreation July/August. 1996 Telephone survey respondents were either directly responsible for program development or were assistants to the program directors. Thirteen park districts, or 22 percent, stated that they had specific programs for at-risk youth. By comparing the presence of at-risk recreation programs with population size, it was found that these programs tend to be concentrated in larger population centers (greater than 70,000). Table 1 lists the percentage of park districts using various evaluation techniques as described in the article by Witt, Crompton, and Baker (3). Respondents were asked to indicate how often they used these evaluation techniques for their programs serving at-risk youth. The choices were always, sometimes, or never. The findings of this survey confirm the hypothesis that program evaluation is concentrated primarily in the measurement of participant attendance, parent testimonials, and participant testimonials about the program. Only a few programmers attempted to measure program success by reviewing participant's academic records and police records, and by soliciting local elected official's perceptions of programs. However, more than half of the respondents indicated that they always use process evaluation methods. These findings suggest that, overall, Illinois park districts are evaluating programs satisfactorily serving at-risk youth. To improve their evaluation methods, park districts must begin incorporating outside statistics such as academics and police records. Respondents were asked to cite the top three reasons why they believe it is difficult to evaluate programs for youth in at-risk environments. The primary reason offered was not enough staff available to conduct thorough evaluations. The second reason was that there was not enough money. The third reason was that there was not enough time. Historically, these are the three main reasons why it's difficult to evaluate any recreation program. It is important to remember, however, that sound evaluation strategies are the key to justifying recreation programs, especially those that serve at-risk youth. Strategies for Program Development What can be done to improve evaluation for programs that serve at-risk youth? The following three strategies can be incorporated into program development and evaluation. Establish program goals based on the needs of the population. The best way to do this is to conduct a needs assessment. Find out what the prospective participants would like for free-time activities. Don't be surprised if the results show that kids are interested in nontraditional programming such as study halls and awareness seminars about things like substance abuse, issues about sex, and career options. Once the needs assessment is completed, write program goals based on those needs. Write measurable goals and objectives for programs that serve youth in at-risk environments. Recall that objectives usually fall into one of three domains: cognitive, affective, and psychomotor. If program goals and objectives are measurable, the evaluation plan should fall into place. For example, a cognitive program objective might state, "By the end of the noncompetitive basketball program, participants will increase knowledge of basketball rules by scoring 80 percent or higher on a rules test." Given this objective, the programmer knows that an instrument must be developed to measure knowledge of basketball rules. Network with local officials and agencies in order to gain access to school and police records. Appropriate records must be accessed in order to claim that programs for at-risk youth significantly reduce crime, teen pregnancy, substance abuse, and the number of high school dropouts. Conclusion As recreation and leisure professionals, we are challenged to be proactive in our evaluation methods of programs that serve at-risk youth. Recreation professionals must move beyond claims that programs are successful because there are a lot of participants. Programs that reflect the needs of the population must be created. The goals and objectives of these programs must be measurable so that evaluation methods can be created simultaneously during program development. Finally, work with local officials and solicit their insights about programs serving at-risk youth. Seek out appropriate records that will substantiate the successes of your programs. With these strategies in mind, we can enter the next millennium fully equipped to meet the needs of our most precious resourceyouth. Barbara E. Schlatter is an assistant professor in the Department of Health, Physical Education, and Recreation at Chicago State University, Chicago, Illinois. Marta K. Moorman is a doctoral candidate in Recreation at the University of Arkansas, Fayetteville, Arkansas. References (1) Cranz, G. 1982. Politics of Park Design. Cambridge, MA: Massachusetts Institute of Technology. (2) Jessor, R. 1992. "Risk Behavior in Adolescence: A Psycho Social Framework for Understanding and Action." Adolescents at Risk: Medical and Social Perspectives, p. 19-35. (3) Witt, P., Crompton, J., & Baker, D. April 1995. "Evaluating Youth Recreation Programs." Journal of Physical Education, Recreation, and Dance, p. 27-30. Illinois Parks & Recreation July/August, 1996 31 |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
