|
In the course of its deliberations described above,
the Committee sought to develop and consider a wide array of design
options in coming up with its final recommendations. These options,
included here to reflect the full range of discussion in the Committee,
are sketched briefly below. The options were developed by individual
members or groups of members in order to further the deliberations
of the group, not to offer a single, comprehensive research design.
As indicated in Chapter IV that outlines the research framework,
some but not all elements of several of these options are included
in the final plan.
Option I. Two Stage Randomized Design with Quasi-Experimental
Component
How the Option Would Work. In stage one, a
nationally representative sample of sites would be chosen from the
full list of all Head Start programs using a stratified and clustered
random sampling technique. Basic information about all of these
programs would be gathered. In stage two, a random subsample of
the stage one sites would be chosen, and these sites would then
randomly assign children to Head Start and control groups. By matching
the sample and population profiles on measured variables, the national
evaluator could then extrapolate impact findings of the subsample
to the larger nationally representative sample.
Sites that do not participate in random assignment
would become part of a quasi-experimental study. This quasi-experimental
study would randomly select Head Start families to participate in
the research, but all would be enrolled in Head Start. The control
group would be a matched comparison group from the community who
would not have received Head Start.
How the Option Relates to the Recommendations.
This option is one example of a specific strategy that would be
permitted under the design criteria identified by the Committee
in its recommendations.
Option II. Random Assignment of Sites to Traditional
Head Start and an Enhanced Head Start
How the Option Would Work. Sites would be randomly
selected from the total universe of Head Start sites. Half of the
sites selected would be randomly assigned to the intervention and
the other half to the control group. The control group would be
Head Start as currently implemented. The intervention group would
be the basic Head Start model with program options (e.g., an added
focus on literacy services; two-year vs. one-year Head Start; full-day
vs. part-day Head Start; various curriculum models). These program
options would not be unusual efforts but programmatic approaches
that reflect what is currently being carried out in strong Head
Start programs and what could reasonably be expected to occur nationally
once their effectiveness is demonstrated. There would be a sequence
of these studies with randomization at the site level so that new
information about various program options could continuously be
used to reshape the core Head Start program.
How the Option Relates to the Recommendations.
This option is not included in the basic impact design, although
the concerns that led to the development of the option (primarily,
concerns regarding contamination, the feasibility of random assignment,
and obtaining clear estimates of the important sources of variation
within Head Start) are extensively represented in the recommendations.
Instead, the Committee has asked the Department to review elements
of this option as part of the broader research agenda.
Option III. Purposive Sampling of Programs + Nationally
Representative Quasi-Experimental Component
How the Option Would Work. Part one includes
a purposive sample of programs selected through a competitive process,
or through a sampling frame that identifies programs according to
specific strata. These programs would then engage in random assignment
of children. The second part includes a study of a nationally representative
sample of Head Start programs with a naturalistic comparison group
in a subset of randomly selected sites made up of children in the
community who would qualify for Head Start but are not served by
the program. The studies would be linked conceptually by using common
measures and measurement points. Because of this conceptual link,
researchers may be able to extrapolate from the purposive to the
nationally representative sample.
How the Option Relates to the Recommendations.
Like the first option above, this approach meets some the Committee's
design criteria.
Option IV. "Part Start" Randomized Design
How the Option Would Work. Sites that have
unserved children would be included in a pool of sites to be randomized.
The randomly selected study sites would be given an expansion grant
or other incentives to serve more children. Children would be randomly
assigned to a full-program group or a partial-program group where
they would receive only partial Head Start services (e.g., health
and nutrition services; support of the family service coordinator;
and child care subsidy). Comparisons would be made between these
two groups of children.
How the Option Relates to the Recommendations.
The Committee did not recommend this option in its full form, recommending
that children be randomly assigned to Head Start and non-Head Start
groups, rather than partial Head Start groups. However, the concern
underlying this option, that staff and parents need an incentive
to participate in the experiment, is reflected in the Committee's
recommendation that the Department should consider what kinds of
incentives can appropriately be offered to families.
Option V. Consortium of Randomized Designs
How the Option Would Work. Sites, in partnership
with local researchers, would be competitively selected to participate
as part of a consortium carrying out a set of coordinated randomized
designs. A national research organization would provide technical
assistance to the consortium and would conduct a cross-site analysis.
If the number of sites was small, the purpose of this analysis would
include gleaning information from the experiences of the sites to
inform a larger, more rigorous experimental design if one is needed
in the future.
How the Option Relates to the Recommendations.
The Committee did not recommend the initial form of this option,
which involved a very small number of sites, out of concern that
such a small group of sites would not be sufficiently diverse to
yield a useful national answer. In this initial form, the Committee
felt that the option was more suitable as part of a feasibility
study than as an impact study itself. If the number of sites was
large enough, this approach could potentially represent another
acceptable approach under the Committee's recommended framework.
Option VI. Model Head Start with Random Assignment
How the Option Would Work. Communities not
currently served by Head Start would be identified and a model Head
Start program would be built with all the attributes believed to
be most effective. Eligible children would be randomly assigned
to the new program. Because the program is new to the community,
random assignment can be implemented with fewer concerns for denying
services to some.
How the Option Relates to the Recommendations.
This option is not part of the Committee's framework for the impact
research, but it could be considered as part of the Department's
overall research portfolio. In addition, the concerns that prompted
the design of this option, particularly concerns regarding the alternative
services available to control group children, are addressed by several
of the Committee's recommendations.
Option VII. Highest Quality Head Start with Average
Head Start as Comparison
How the Option Would Work. Compare Head Start
programs that provide an extraordinarily high quality Head Start
experience to a random selection of average Head Start programs.
How the Option Relates to the Recommendations.
This option is not part of the Committee's research design for impact,
but it could be part of the Department's overall research portfolio.
Whether through this design or not, the Committee urges the Department
to continue a vigorous research agenda relating to Head Start quality.
Option VIII. National Early Childhood Data Collection
Study
How the Option Would Work. Build on one of
the existing national studies, such as ECLS-B or ECLS-K, ensuring
that there is a sufficient subsample of Head Start children to compare
outcomes for Head Start children and comparable non-Head Start children.
How the Option Relates to the Recommendations.
The Committee recommends that the impact research must include the
fullest possible use of ongoing research, to supplement the random
assignment sites and the possible quasi-experimental strategy. Therefore,
this approach is a key part of the Committee's recommendations.
|
