The Future of Children, Princeton - Brookings: Providing research and analysis to promote effective policies and programs for children.

The study seeks to answer four questions: (1) How well does actual program performance conform to the HSP model? (2) How successful is the program in achieving desired outcomes for parents and children? (3) How does fidelity of program implementation influence outcomes? and (4) How do benefits compare with direct and indirect program costs? The framework for the study and the measures employed are based on the program's conceptual framework, which is illustrated in Figure 1.

The evaluation is a true experiment, with random assignment of at-risk families to the home-visited and control groups. There are three study groups: the HSP group, the main control group, and a testing control group. Families in the HSP and main control groups are followed and tested annually to measure outcomes. Testing control group families are evaluated only at three years to assess whether the study's intensive data collection has influenced outcomes.

The evaluation includes all HSP group families for three years, regardless of whether they stay in the program or leave prematurely. Control group families are not eligible for HSP for three years following the index birth.

The evaluation focuses on the six HSP sites serving geographically defined communities on Oahu. The sites are operated by three community agencies—HFSC, Child and Family Service (CFS), and Parents and Children Together (PACT)—with each agency operating two sites. Sites on Oahu rather than the Neighbor Islands were selected to allow close monitoring of fieldwork while containing costs.

Families were recruited from November 1994 through December 1995. EID staff identified at-risk families following the usual protocol. An at-risk family was eligible for Healthy Start services if the family lived in the HSP target community and was not already known to CPS. A family was eligible for the evaluation if the mother understood English well enough to be interviewed without a translator and if the family's HSP site was open to accept new referrals. Overall, 897 at-risk families met HSP and study inclusion criteria.

When an eligible family was identified, the EID worker described HSP and the evaluation and obtained the mother's signed, informed consent to take part. By study protocol, EID workers called the evaluation office for group assignment of all HSP-eligible families, not just those eligible for the evaluation. This served two purposes. First, it prevented the opportunity for HSP staff to bypass the random assignment procedure. Second, it gave both evaluation-eligible and ineligible families the same likelihood of enrollment in HSP. Using a table of random numbers, the evaluation staff informed the EID worker of the family's group assignment.

Of families eligible for the study, 730 (81%) gave initial consent to participate and were randomized into HSP and control groups. Mothers were to be interviewed by evaluation staff at the hospital before discharge or at home within a month of delivery if a hospital interview was not possible. Overall, 684 (94%) of those randomized were interviewed at baseline; the remainder declined participation. Thus, 76% of eligible families completed baseline interviews and became study participants; this is comparable to HSP participation rates on Oahu prior to the study. Study participants were slightly younger than eligible nonparticipants, and were more likely to be at extremely high risk, to have given birth prematurely, and to have completed their EID assessments in the hospital rather than by telephone.

There are 373 families in the HSP group, 270 in the main control group, and 41 in the testing control group. Baseline characteristics of the HSP and main control groups (those followed at years one and two) are described in Table 1. HSP and control group mothers were comparable at baseline in most respects. Twelve percent of mothers had no relationship with their children's fathers, 36% were friends with the fathers or going together, 29% lived with the fathers, and 24% were married. Two-thirds of mothers in both groups were high school graduates, but HSP group mothers were more likely to have worked in the year prior to the index child's birth (52% versus 44%, p=.05). HSP group mothers were slightly less likely to have poor general mental health at baseline (43% versus 50%, p=.05) and to have reported partner violence in the year prior to the index birth (43% versus 52%, p=.02). Two-thirds of families had household incomes below the poverty level (p=.34). From data not presented in the table, 71% of HSP and control group fathers had graduated from high school and 67% were working, and 42% of families had problems with substance abuse (all p>.40).

Follow-up data are collected through structured parent interviews; developmental testing of the child; observation of the home environment and parent-child interaction; and review of HSP records, pediatric records, CPS reports, and health care insurer files. Precautions to minimize the threat of biased measurement include the independence of evaluation staff from HSP itself, the prevention of interviewers from knowing families' group status, the use of objective measures, and the use of multiple data sources to supplement participants' memories.

Eighty-eight percent of participating families were followed at one year, 88% at two years, and 83% at both points. Families lost to follow-up did not differ significantly from those followed at one year. At two years, follow-up rates were slightly higher for native Hawaiian families and lower for families from other Pacific islands.