|
Head Start provides comprehensive early child development services
to low-income children, their families, and communities. Since 1965,
the program-enjoying bipartisan support-has served nearly 18 million
children and their families (Administration on Children, Youth and
Families, 1998), providing preschool-aged children with education,
nutritious meals, and access to health, mental health, and social
services that support their early development. Further, the program
has influenced the development and direction of a broad range of
early child development services across the nation through its role
as a national laboratory. The program has provided support to low-income
families seeking enriching experiences for their children, and it
has provided direction and leadership to the fields of early child
development and education.
With increased attention to outcomes and accountability for federal
resources (Government Performance and Results Act of 1993)2,
the Head Start program has been challenged to document its effectiveness
in new ways. A series of reports from the U.S. General Accounting
Office (U.S. GAO, 1997; U.S. GAO, 1998) and concern among congressional
leaders about the lack of rigorous experimental designs testing
the effectiveness of the program caused Congress to specifically
call for the formation of an independent panel of experts to review
and make recommendations on the design of a study or studies that
provide a "national analysis" of the impact of the Head Start program
(Head Start Amendments of 1998).
The Head Start Amendments of 1998 provide specific guidance about
how Congress envisions impact research on Head Start. For example,
the legislation calls for research that "uses rigorous methodological
designs and techniques (based on the recommendations of the expert
panel), including longitudinal designs, control groups, nationally
recognized standardized measures, and random selection and assignment,
as appropriate." The legislation states that "the research shall
be conducted as a single comprehensive assessment or as a group
of coordinated assessments designed to provide, when taken together,
a national analysis of the impact of Head Start programs." The legislation
states that the assessment or coordinated assessments include "comparisons
of individuals who participate in Head Start programs with control
groups (including comparison groups) composed of (i) individuals
who participate in other early childhood programs (such as public
or private preschool programs and day care); and (ii) individuals
who do not participate in any other early childhood program." Impact
is defined as "a difference in an outcome for a participant in the
program that would not have occurred without the participation in
the program" and is to be examined at three points in time: "on
the date participants leave Head Start programs; at the end of kindergarten;
and at the end of first grade." The final report of the research
study or studies is to be transmitted to Congress by September 30,
2003 (Head Start Amendments of 1998).
The Advisory Committee on Head
Start Research and Evaluation
On March 23, 1999, the Department of Health and Human Services
Secretary signed the Charter establishing the Advisory Committee
on Head Start Research and Evaluation. The Charter reiterates the
requirements spelled out in the Head Start Amendments, as amended
by the 1998 reauthorization. It also states that as part of the
process, the Committee shall:
-
Review existing and ongoing research and evaluation studies
that document the impact of Head Start programs;
-
Assess the benefits and feasibility of alternative research
designs and techniques to determine if, overall, Head Start
programs have impacts consistent with their primary goal of
promoting school readiness by enhancing the social and cognitive
development of low-income children;
-
Report to the Secretary on recommendations for a study or studies
of the impact of Head Start services, including appropriate
designs, techniques, methods of analysis, and consideration
of sources of variation;
-
Advise the Secretary on the progress of the study or studies
of the impacts of Head Start programs; and
-
Review and comment to the Secretary, if the Committee so desires,
on the first and second interim and final impact study reports
of the organization(s) selected for carrying out the independent
research.
The Advisory Committee includes 30 individuals with expertise in
areas of program evaluation and research, education, early childhood
care and education, policy, and economics. Many of the Committee
members have a long and rich involvement with Head Start and other
early childhood programs; other members are new to Head Start but
have extensive backgrounds in research methodology. A biographical
sketch of each Committee member is included at the end of this report.
The Committee is chaired by Olivia A. Golden, Ph.D., Assistant Secretary
for Children and Families of the Department of Health and Human
Services.
The Committee met in April, June, and July 1999 in order to fulfill
the first phase of the Committee's charge: issuing recommendations
to the Secretary for a study or studies that provide a national
analysis of the impact of Head Start. In addition to these meetings,
individual Committee members volunteered to draft design and issue
papers that were circulated between meetings and helped the Committee
reach the conclusions that are included in this report. All meeting
agendas and papers prepared by Committee members were posted on
the Department's web page for public access.
The deliberations were thoughtful and intense. Overall, the Committee
finds the complexity of the task very challenging: Committee members
recognize that there is no simple design for measuring the impact
of Head Start. Some design choices threaten external validity (i.e.,
representativeness or the ability to generalize from study participants
to the Head Start population as a whole); others threaten internal
validity (i.e., causality or the ability to attribute observed outcomes
to participation in Head Start). Some research designs would require
changes in the practices and procedures for enrolling the most needy
children in Head Start; other designs would require that services
be denied to eligible children. Deciding precisely what to recommend,
primarily as it relates to the trade-offs between external and internal
validity, was a challenging task for the Committee.
This report-the first phase of the Committee's work-represents
the deliberations of the Committee and its recommendations to the
Secretary regarding a framework for studying the impact of Head
Start. Chapters include:
-
The Growth of Head Start and Other Early Childhood Options.
This chapter provides background information on the growth of
Head Start and other early childhood options that Committee
members considered as they discussed the most credible and feasible
methods for measuring the impact of Head Start.
-
Previous and Current Research on Head Start and Early Childhood.
This chapter summarizes findings from previous research on Head
Start and other early childhood studies, and outlines the components
of the current Head Start research agenda that provide information
about impact, quality, and outcomes.
-
Recommended Framework for Studying the Impact of Head Start.
This chapter reflects deliberations of the Committee related
to its charge-issuing recommendations to the Secretary regarding
the design of an impact study (or set of coordinated assessments)
of Head Start. Thus, it includes the research questions, criteria,
outcomes, measurement issues, and overall research design recommended
by the Committee.
-
Rationale for the Recommendations: Addressing Key Challenges.
This chapter highlights challenges for designing an impact study
or studies of Head Start that influence the generalizability
of the findings and the ability to determine causality. Members
of the Committee believe that with careful planning and ongoing
consideration, these challenges could be addressed in an impact
study or set of studies.
-
Next Steps: Implementing the Recommendations. This chapter
outlines specific steps for the Department of Health and Human
Services to take in carrying out a plan for studying the impact
of Head Start.
|
