Journal Issue: Transition to Adulthood Volume 20 Number 1 Spring 2010
What Conclusions Can Be Drawn?
It is difficult to draw cross-cutting lessons from the evaluations in table 1 because there are many programs and not many unambiguously positive results. For example, the data do not support clear conclusions about whether paid work, a residential structure, or other program design elements are associated with more positive results in random-assignment studies. It is possible, however, to make a few general points.
First, although sustained positive effects would obviously be preferable, short-term effects are not unimportant. When programs achieve short-term increases in earnings or other outcomes, those effects are not erased if the program and control groups have similar outcomes later. Although many programs assert that they can alter the long-term trajectories of their participants, it is worth considering whether it is reasonable to expect even the strongest youth programs to produce effects that can still be measured many years later. Results like those achieved in the Career Academies evaluation, where earnings gains persisted eight years after students had completed high school, are very rare—and the Academies that were tested did not serve a highly disadvantaged group of young people. Some experts have raised the question of whether it is more appropriate to think of time-limited programs for dropouts as inoculations, whose effects may last forever, or as vitamins, whose effects wear off if they are not taken consistently.
Second, it is important to note that almost all the programs (and the control groups as well) involved youth who had volunteered to participate—and who thus had at least some motivation to change their lives. In fact, some of the programs extensively screened applicants and accepted only those who demonstrated strong motivation and commitment. Thus the young people who ended up in the control groups likely sought out other programs in the community and received some of the same kinds of services that program group youth received. The study results could thus be interpreted to mean that the tested programs did not do much better than other programs in their communities, but that all of the programs were relatively effective for motivated participants. That said, most of the evaluations also found that outcomes were relatively poor for both research groups. For example, in the Job Corps study, the average employed sample member earned only about $10,000 a year during the later years of the follow-up period. Similarly, in the JOBSTART study, only about 65 percent of sample members worked at all in the final year of the study period, and those who worked earned less than $9,000, on average. In other words, regardless of their effects, the programs' outcomes leave much room for improvement.
Third, it is possible that the difficulty in achieving sustained increases in economic outcomes may be traced, in part, to the educational goal of most programs—to help participants pass the GED exam. Many studies have concluded that the labor market does not, in fact, view the GED as equivalent to a high school diploma. In other words, GED holders earn significantly less than people with regular high school diplomas. Some studies have even questioned whether GED holders earn more than uncredentialed dropouts, though some recent studies suggest that the GED does have an economic payoff, at least for dropouts with low skills—although the payoff may take several years to appear. Studies have also shown that postsecondary education pays off as much for GED holders as for high school graduates, but only a small minority of GED holders complete even one year of postsecondary education.20 These data may help explain why programs that substantially increased GED receipt did not lead to longer-term gains in employment or earnings.
Fourth, some youth experts have pointed to broader limitations of some of the program models, particularly those tested during the 1980s and early 1990s. Some have argued that these programs failed to engage youth long enough to make a lasting difference, in part because restrictions on federal funding under the Job Training Partnership Act system did not allow the programs to offer stipends or opportunities for paid work experience.21
Others maintain that some of the earlier youth programs were "deficit focused"—that is, they defined participants by their problems and sought to "fix" them. These experts recommend that programs should not only provide participants with training or a job, but also expose them to a range of settings, activities, and relationships that are thought to promote healthy development across a wide range of domains. One study identified these domains as cognitive, physical, social and emotional, ethnic identity, civic engagement, and career.22 Young people from higher-income families are more likely than their lower-income counterparts to have positive experiences in these developmental areas in their families, schools, and communities. Programs may help to fill these gaps by exposing youth to responsible, caring adult role models; by creating a safe, positive group identity among participants; and by giving young people opportunities to act as leaders and to contribute to the broader society.
Among current programs, for example, YouthBuild helps young people work toward their high school diploma or GED while simultaneously learning job skills as they build or rehabilitate housing for homeless and low-income people. It emphasizes service and leadership development by giving young participants a key role in running the program. Participants also receive stipends or wages. Similarly, Service and Conservation Corps, descendants of the Depression-era Civilian Conservation Corps and of the American Conservation and Youth Service Corps evaluated in 1990 (see table 1), combine intensive community service with job training and education. Crews of participants work on conservation, urban infrastructure, and human services projects and receive stipends. A third such program, ChalleNGe, includes a five-month residential phase built around eight "core components" designed to promote positive development: service to community, leadership and followership, responsible citizenship, health and hygiene, life-coping skills, physical fitness, job skills, and academic excellence. In a final such example, City Year, participants devote a full year to community service and civic engagement, wearing uniforms to build a team identity and make their work highly visible to the community. Because the program is open to young people from a wide range of backgrounds, participants may be exposed to people quite different from themselves.
It is widely believed that programs built on positive youth development principles are more effective than others. Although this may well be the case, the evidence from rigorous evaluations is too thin to prove or disprove the hypothesis; several of the programs noted have not been rigorously evaluated. Moreover, it may be difficult to achieve consensus about which particular programs reflect youth development principles and which do not.
Tight structure and accountability may also be critical for young people who have grown up in chaotic environments and may help to counteract the potentially negative effects of placing many at-risk young people together in a program setting—effects sometimes called "deviant peer influences" or "peer contagion."23 For example, the ChalleNGe program adopts a "quasi-military" approach: participants are divided into platoons and squads, live in barracks, have their hair cut short, wear uniforms, and are subject to military-style discipline. Their day is highly structured, with almost no "down time." Most of the staff are military veterans, retirees, or National Guard members. Although the program uses military structure, discipline, facilities, and staff to accomplish its objectives, program participation is voluntary, with no requirements for military service during the program or afterwards. Some experts have suggested that elements of the ChalleNGe model could be applied in non-residential settings and, indeed, a few military-style public high schools already operate in the United States.24
Finally, strong youth programs are focusing more on the transition for program graduates. Program effects may decay over time in part because youth have trouble maintaining momentum after they leave the structured, supportive program environment and confront a world where opportunities are limited. In addition, youth programs may have difficulty building strong links with employers, colleges, or other "post-program" resources for their participants. As noted, a number of youth programs have begun to build links to postsecondary education for their participants. Others have an "open door" policy that allows youth to maintain contact with the program for as long as they want or need to.25 The ChalleNGe program includes a formal one-year Post-Residential Phase that is built around a structured mentoring program. Youth nominate their own mentors, who are then screened, trained, and supported by program staff. Studies suggest that well-implemented mentoring programs can have positive impacts for some young people, though the studies did not examine mentoring for high school dropouts.26