|
In 1993, the Final Report of the Advisory Committee on Head Start
Quality and Expansion concluded:
"A series of substantial and careful reviews has reported
that Head Start produces benefits for the children and families
experiencing the program (Bronfenbrenner, 1974; Datta, 1979; McCall,
1993; McKey et al., 1985; Zigler & Styfco, 1993). The evidence
is clear that Head Start produces immediate gains for children
and families. The evidence on the long-term impact of the program
has been the subject of some debate" (U.S. Department of Health
and Human Services, 1993).
The Advisory Committee on Head Start Quality and Expansion also
stated:
"Head Start is entering an historic period of reexamination,
improvement in quality, and expansion of services. The size of
the program, its comprehensive services, and diversity of the
population it serves, and the fact that it is federally funded
suggest a role for Head Start as a national laboratory for best
practices in early childhood and family support services in low-income
communities. Because Head Start needs to expand and renew itself
in order to assume its role as a state-of-the-art 'technology,'
there is a concomitant and compelling need for a new, expanded,
and formal role for Head Start research" (U.S. Department of Health
and Human Services, 1993).
These statements were made as a result of a review of previous
research and belief in the potential of Head Start to shape the
early childhood field. Since the report of the Advisory Committee
on Head Start Quality and Expansion, steps have been taken to
expand the Head Start research agenda and better understand the
relationship between quality and outcomes. This chapter provides
a brief review of major research findings on Head Start and related
early childhood initiatives, as summarized by past literature
reviews, conclusions of previous expert panels, and ongoing research
initiatives that are most relevant to the question of program
impact. The Committee did not deliberate explicitly on the relative
quality, value, or utility of these past findings, but rather
used this history as a context for shaping its deliberations about
the future.
Previous Research on Head Start and Other Early
Childhood Programs
The results of the first national impact study of Head Start,
conducted by the Westinghouse Learning Corporation in the late
1960s, concluded that summer Head Start programs had little or
no effect on children, and that full-year Head Start programs
benefited children's school achievement, but such effects tended
to "fade out" by the third grade (Westinghouse Learning Corporation,
1969). Although these results created repercussions in the Head
Start policy community, researchers recognized that the study
was seriously flawed through the use of a post-test-only design
(so that it was impossible to adequately control for initial differences
between Head Start and comparison children), and outcome measures
that narrowly focused on cognitive development at the expense
of the full range of developmental outcomes that represented the
goals of Head Start. Subsequent small studies illustrated that
there were immediate benefits from participation in Head Start
on IQ tests or other cognitive instruments (Bissell, 1971; Smith,
1973; Miller & Dyer, 1975; Zigler, Abelson, & Trickett, 1982).
In 1981, the Department undertook a multi-year effort to synthesize
all the early research on Head Start, both published and unpublished.
More than 200 reports were studied, and 76 of these became part
of the meta-analysis (McKey, et al., 1985)5.
The Synthesis and Utilization Project concluded that:
".children enrolled in Head Start enjoy significant
immediate gains in cognitive test scores, socioemotional test
scores and health status. In the long run, cognitive and socioemotional
test scores of former Head Start students do not remain superior
to those of disadvantaged children who did not attend Head Start.
However, a small subset of studies find that former Head Starters
are more likely to be promoted to the next grade and are less
likely to be assigned to special education classes. Head Start
also has aided families by providing health, social, and educational
services and by linking families with services available in the
community. Finally, educational, economic, health care, social
service, and other institutions have been influenced by Head Start
staff and parents to provide benefits to both Head Start and non-Head
families in their respective communities" (McKey, et al., 1985).
More recently, a study using data from the National Longitudinal
Survey of Youth compared scores from tests of receptive vocabulary
for Head Start children with those of siblings who did not attend
Head Start but either had no preschool or attended another type
of preschool. The study found large and significant gains in test
scores for both White and African American children over their
siblings; there were also gains for Hispanic children (Currie
& Thomas, 1995; Currie & Thomas, 1996). Furthermore, for White
children, effects of Head Start were greater than effects of attending
other preschool programs. However, among African American children,
the gains were quickly lost. The study also illustrated that both
White and African American children who attend Head Start or other
preschools gain greater access to preventive health services (Currie
& Thomas, 1995).
In addition to this specific research on Head Start, many other
studies have been conducted in the early childhood field that
provide some indication about the effectiveness of high quality
services, including the following:
-
Longitudinal data from the High/Scope Perry Preschool Project
and other model preschool programs showed that despite the
apparent fade-out of certain effects, children served in preschool
programs showed positive longer-term effects on such important
outcomes as special education placement, high school graduation,
and arrest rates (The Consortium for Longitudinal Studies,
1983; Schweinhart, Barnes, & Weikart, 1993).
-
Other program evaluations provide clear support that the
early intervention high quality child care experiences enhanced
children's cognitive and language development, at least for
the duration of the intervention (Haskins, 1989; IHDP, 1990;
Ramey et al., 1992; Lazar & Darlington, 1982; O'Connell &
Farran, 1982). Compared with those in the control groups,
low-income children who attended high quality child care centers
displayed higher cognitive scores during the preschool years
(Lazar & Darlington, 1982; IHDP, 1990; Burchinal, Lee, & Ramey,
1989). For some of the most intensive early childhood programs,
cognitive, academic, and social benefits have endured into
adolescence and early adulthood (Garber, 1988; Zigler, Taussig,
& Black, 1992; Campbell & Ramey, 1994; Yoshikawa, 1994; Yoshikawa,
1995; Campbell, 1999). In addition, compared with control
group children, children who received early interventions
were more likely to be promoted in school, graduate from high
school, and become productive young adults (Lazar & Darlington,
1982; Schweinhart, Barnes, & Weikart, 1993). In contrast,
control group children were more likely to be retained in
grade, be placed in special education, and drop out of school
(Lazar & Darlington, 1982).
-
Barnett's recent review of 36 studies of early childhood
programs (including some Head Start programs) concluded that
they can produce large effects on IQ during the early childhood
years and sizable persistent effects on achievement, grade
retention, special education, high school graduation, and
socialization. He found that the effects depend on program
quality and are larger for well-designed intensive early childhood
care and education interventions than for ordinary child care
(Barnett, 1995).
While this research is based on a wide range of programs, many
of the other efforts share features with Head Start, including
the population served, goals, program strategies, and conceptions
of high quality and best practices in serving young children and
families. As such, their findings have provided support for continued
investments in early childhood programs and Head Start.
In response to a congressional request for a review of the literature
about Head Start's impact, the General Accounting Office (GAO)
examined studies of Head Start participation in 1976 or later
to determine what the studies suggested about the impact of the
program. In a 1997 report to Congress, the GAO stated:
"Although an extensive body of literature exists on
Head Start, only a small part of this literature is program impact
research. This body of research is inadequate for use in drawing
conclusions about the impact of the national program in any area
in which Head Start provides services such as school readiness
or health-related services. Not only is the total number of studies
small, but most of the studies focus on cognitive outcomes, leaving
such areas as nutrition and health-related outcomes almost completely
unevaluated. Individually, the studies suffer to some extent from
methodological and design weaknesses, such as noncomparability
of comparison groups, which call into question the usefulness
of their individual findings. In addition, no single study used
a nationally representative sample so that findings could be generalized
to the national program."
And in a 1998 report, the GAO recommended to the Secretary of
the Department of Health and Human Services:
"To determine whether the Head Start program is making
a difference in the lives of those it serves, we recommend that
HHS assess the impact of regular Head Start programs by conducting
a study or studies that will definitively compare the outcomes
achieved by Head Start children and their families with those
achieved by similar non-Head Start children and families."
These conclusions and recommendations by the GAO stimulated considerable
attention and debate and were one of the factors leading to the
charge of this Advisory Committee.
Call for a Revitalized
Head Start Research Agenda
Beginning in the late 1980s and continuing to date, the Department
has regularly sought the advice of experts to identify the best
ways to conduct research on Head Start including approaches to
studying the effectiveness of the Head Start intervention and
the relationship between quality and outcomes. In addition to
frequent consultation with leading researchers, the Department
brought together three expert panels that helped identify areas
for future research and helped shape a revitalized Head Start
research agenda.
Advisory Panel for the Head Start Evaluation
Design
In 1990, the Advisory Panel for the Head Start Evaluation Design
Project-commonly referred to as the "Blueprint Committee"-was
created to conduct a systematic analysis of the research needs
relevant to the future of Head Start and to recommend a series
of options for the evaluation of the Head Start program. As a
result of their deliberations, the committee called for the establishment
of an overall research strategy and a set of guiding principles,
rather than specific studies or design alternatives. The overall
strategy and general principles were organized around two principal
questions: (1) Which Head Start practices maximize benefits for
children and families with different characteristics under what
types of circumstances? and (2) How are gains sustained for children
and families after the Head Start experience? The Department subsequently
organized various research and evaluation activities within this
general framework.
Advisory Committee on Head Start Quality and
Expansion
In 1993, the Department created the Advisory Committee on Head
Start Quality and Expansion with the goal of reviewing the program
and making recommendations for improvements and expansion. The
Advisory Committee's report recommended strengthening the role
of research. Specifically, the Advisory Committee called for five
major actions: (1) build a strong and enduring infrastructure
for Head Start research to ensure that Head Start is able to carry
out its leadership role on an ongoing basis, (2) conduct new Head
Start research focusing on quality and other policy issues, (3)
conduct longitudinal research on children and families served
in Head Start programs, (4) expand the partnership between research
and practitioners by encouraging better communication and better
utilization of data, and (5) develop a long-term research plan
for Head Start which places Head Start in the broader context
of research on young children, families, and communities, ensures
a commitment to ongoing themes, and has the flexibility to respond
to new and emerging issues.
National Academy of Sciences Roundtable on
Head Start Research
In 1994, the Department funded the National Academy of Sciences
to convene a Roundtable on Head Start Research. The Roundtable
was charged with identifying directions for research on Head Start's
families. The Roundtable identified three broad areas that had
not been adequately explored by research: (1) the challenges posed
to Head Start by the increasing ethnic and linguistic diversity
of the families it serves; (2) the need to embed research on Head
Start within its community context, paying specific attention
to the effects on Head Start and its families of violent environments;
and (3) the implications of the changing economic landscape and
the structure of income support policies for the poor for how
Head Start works with families, and what it means to offer families
a high quality program.
Taken together, these expert panels helped refine and expand the
ongoing set of Head Start research activities in order to maintain
its role as a national laboratory for early childhood research.
They did not recommend a study of impact as conceived by the Head
Start Amendments of 1998, in part because members of the various
committees believed the short-term impact of Head Start had been
adequately documented by the Synthesis and Utilization Project.
Current Research
on Head Start
The recommendations from the Blueprint Committee, the Advisory
Committee on Head Start Quality and Expansion, and the National
Academy of Sciences Roundtable, along with other sources of input,
were taken into consideration as the Administration on Children,
Youth and Families of the Department developed the current overall
Head Start research agenda. This revitalized agenda addresses
six broad areas: (1) focusing research on quality, (2) conducting
longitudinal research on children and families, (3) evaluating
services for infants and toddlers, (4) studying emerging innovative
strategies, (5) studying special subpopulations, and (6) developing
and enhancing capacity for research on Head Start in partnership
with the larger early childhood and development community6.
The studies being conducted in each of these broad areas will
provide useful information about Head Start and child development
in general. The question of impact is part of, but not central
to, much of the current research efforts.
Current Head Start research efforts most relevant to the question
of impact are the:
-
Family and Child Experiences Survey (FACES);
-
Early Head Start Research and Evaluation Project;
-
Head Start/Public Schools Early Childhood Transition Demonstration
Project;
-
Early Childhood Longitudinal Study-Kindergarten Cohort (ECLS-K)
and the Early Childhood Longitudinal Study-Birth Cohort (ECLS-B)7;
and
-
Head Start Quality Research Centers (QRCs).
Family and Child Experiences Survey
The Family and Child Experiences Survey (FACES) is a study of
3,200 families with children enrolled in 40 nationally representative
Head Start programs. The study began in 1997 and will be collecting
longitudinal data on these children through first grade as part
of Head Start's responsibility to gather information for the Government
Performance and Results Act of 1993. In addition, an embedded
case study is being conducted of 120 families randomly selected
from the larger FACES sample. The overall purpose of FACES is
to provide descriptions of the characteristics, experiences, and
outcomes for children and families served by Head Start and to
observe the relationships among family and program characteristics
and outcomes.
FACES will be able to compare the developmental status of Head
Start children with their same-aged peers in the following ways:
-
Comparison of Head Start children's scores with overall age
norms on the Peabody Picture Vocabulary Test, the Woodcock-Johnson
Psychoeducational Battery-Revised (Letter-Word Identification,
Applied Problems, and Diction tasks), the McCarthy Scales
of Children's Abilities (Draw a Design and Numerical Memory
subtests), and the Test of Language Development phonemic analysis
subscale;
-
Comparisons of the rate of development shown by Head Start
children with the rate of development of all preschoolers
of the same age using national normative information;
-
Comparison with children from low-income families who have
not attended Head Start but who are part of other studies
using the same measures; and
-
Comparisons among Head Start children who participate in
the program for varying duration.
In addition to the above comparisons, FACES will be able to relate
differences in children's development and family behavior to program
quality measures and other aspects of the Head Start centers and
programs the children attend.
Further, FACES has incorporated portions of the ECLS-K assessment
instruments (a description of ECLS-K follows) into the FACES kindergarten
and first grade follow-up. This will allow the tested achievement
of Head Start graduates to be compared with the achievement of
a large, nationally representative comparison group who attend
other programs (e.g., publicly funded prekindergarten programs).
Some of the latest findings from the FACES study include:
-
Early Childhood Environment Rating Scale scores were consistently
good over two years, within the national sample of 40 programs.
At three time points, approximately 75 percent of observed
classrooms were rated good quality or higher. No classrooms
scored below a minimal level of quality, unlike many studies
of other preschool and child care settings (Administration
on Children, Youth and Families, 1998a; Resnick & Zill, 1999).
-
Observed Head Start classroom quality is linked to child
outcomes. For example, children score higher on early literacy
measures when they experience richer teacher-child interaction,
more language learning opportunities, and a classroom well
equipped with learning resources (Administration on Children,
Youth and Families, 1998a).
-
Children in the highest quarter of the Head Start sample
scored close to the national mean on vocabulary, math, letter
identification, and dictation tasks at the end of Head Start,
although the median score for Head Start children was approximately
10 points below the national mean. Children in Head Start
made significant gains in some areas (i.e., vocabulary knowledge
and social skills) compared to national norms. At the same
time, there are other areas (i.e., letter recognition and
problem behavior) where current progress seems inadequate,
suggesting Head Start programs could be doing more (Zill,
Resnick & McKey, 1999).
-
By the end of kindergarten, however, Head Start children
showed significant gains in knowing letters, writing letters,
and writing their names compared to national norms. They also
improved in awareness of word sounds and familiarity with
books and print conventions. The top quarter again scored
at national norms at the end of kindergarten, similar to the
findings at the end of Head Start (Zill, Resnick & McKey,
1999).
The Department is building on the FACES study in two other respects.
First, the Administration on Children, Youth and Families is conducting
a feasibility study to identify and assess methods for contacting
and interviewing families who are eligible for Head Start within
the FACES neighborhoods but not currently served by the program.
The findings (available by the summer of 2000) will assist in
identifying potential comparison groups for future evaluation
studies and will provide information for the improvement of recruitment
procedures for individual programs. In addition, the Administration
on Children, Youth and Families is funding the design of a study
of quality enhancements in Head Start programs, including enhancements
in areas of letter recognition, reading concepts, and emerging
literacy. This study is intended to explore potential causal links
between program/classroom characteristics and child performance.
The report on the research design will be available by the summer
of 2000.
Early Head Start Research and Evaluation Project
The Early Head Start (EHS) Research and Evaluation Project is
a study of approximately 3,000 low-income families with infants
and toddlers served by the EHS program. EHS, initiated by the
Federal government in 1995, represents a new phase of Head Start
that serves low-income pregnant women and families with infants
and toddlers. Seventeen programs were selected from the first
two cohorts to participate in the national evaluation; 16 of the
programs are participating in additional site-specific research.
These programs were selected from nearly 50 that applied to become
part of this research effort and are highly similar to the rest
of the programs from which they were selected. All programs in
the national evaluation recruited twice as many families as they
could serve, and the evaluation contractor randomly assigned families
either to the EHS program or a control group.
The EHS impact study will provide information on the overall
impact of the program on children and families; differential effects
for families with certain characteristics; differential impacts
related to differences in program implementation, program theories
of change and quality of child development services; and how within-program
variations in services delivered affect child and family outcomes.
An interim report will be available in 2001 followed by a final
report on program impact in 2002.
In addition, a longitudinal study is being planned that will
follow the children and families in these EHS research sites.
The longitudinal study will follow children through entry to kindergarten
and will answer a number of questions, including the effects on
children of continuous, five-year quality early childhood experiences
(e.g., participation in EHS followed by participation in the traditional
Head Start program) compared to less intense, discontinuous, or
low-quality program experiences.
Head Start/Public Schools Early Childhood Transition
Demonstration Project
The Head Start/Public Schools Early Childhood Transition Demonstration
Project-commonly known as the "Transition Project"-is a longitudinal
study of 7,515 former Head Start children, their caregivers, teachers,
and principals through third grade. The study was designed to
determine the effects of the transition demonstration on children,
families, the Head Start program, the public school system, and
the community; and to assess the effectiveness of the transition
concept as a means for the maintenance and enhancement of early
gains achieved by Head Start children and families. Demonstration
grants were awarded to 31 sites. Grantees were required to randomly
assign schools to either a demonstration or a control condition.
A consortium was formed among the national and local evaluators
to develop the design for the national evaluation. At the same
time, 22 of the sites were given funds to collect the core measures
on non-Head Start graduates who were attending schools in the
study. There was considerable variability, however, among the
sites in sample selection and the ability to follow these children
over time. The final report on the evaluation is expected in the
fall of 1999.
Early Childhood Longitudinal Survey-Kindergarten
Cohort
The Early Childhood Longitudinal Survey-Kindergarten Cohort (ECLS-K)
is a longitudinal study of a nationally representative sample
of children from the beginning of kindergarten through fifth grade
to test hypotheses about the effects of a wide range of family,
school, community, and individual variables on children's development,
early learning, and early performance in school. Started in the
fall of 1998, the study includes approximately 18,000 children
enrolled in 931 schools nationwide. There are about 2,933 children
in the sample whose parents report they have been in Head Start
at some point. These reports will be verified. Data will be available
beginning in the spring of 2000.
The longitudinal data will include information on an array of
child development measures that are directly related to school
readiness and social competence. These include: direct assessments
of early reading skills; direct assessment of early math skills;
direct assessment of general knowledge; teacher ratings of children's
approaches to learning, social skills, and problem behavior; direct
assessments of fine and gross psychomotor skills; direct measures
of children's height and weight, as well as parent and teacher
reports on children's health status. These measures will be available
at kindergarten entry, at the end of the kindergarten year, at
the beginning of first grade (for a subsample) and at the end
of first grade. In addition, ECLS parent questionnaires are collecting
demographic and socioeconomic descriptors of the children and
families, making it possible to match and control for differences
between low-income children who have or have not attended Head
Start. These descriptors include: parent education levels, family
income, race and ethnicity, disability status, minority language
status, family structure, parents' employment status and history,
number of siblings, welfare dependence, and others.
Early Childhood Longitudinal Survey-Birth Cohort
The Early Childhood Longitudinal Survey-Birth Cohort 2000 (ECLS-B)
will provide detailed information on children's development, health,
and early care and education on a nationally representative sample
of 15,000 children born in 2000 who will be followed longitudinally
from birth through the end of first grade. The design will capture
data about children's homes, communities, health care, non-parental
care, and early childhood programs. Preliminary data will be available
in the spring of 2002. The complete results measuring children
from birth through first grade will be available in 2008. A Head
Start substudy will enhance the information about the types and
quality of care received by approximately 1,200-2,400 young children
from low-income families and the consequences of differences in
care quality for children's development and later academic achievement.
Further, it will provide information about the decisions that
families make related to selection of care and education settings,
including Head Start.
Head Start Quality Research Centers
In response to concerns that the most rigorous methodological
approaches were needed to measure the effectiveness of Head Start,
in 1997 the Department asked the four Head Start Quality Research
Centers (QRCs) to test the feasibility of conducting randomized
studies within Head Start programs at their respective sites8.
While the experiences of the QRCs varied, a pattern emerged of
similar opportunities and challenges in their investigations of
the effectiveness of using randomized designs to test the impact
of Head Start. For example, the QRCs found that:
-
Implementation of the research was critically dependent on
the development of trusting working relationships between
researchers and Head Start program administrators and other
staff, necessitating ample time for researchers and Head Start
staff to plan such research. Obtaining endorsement of the
study and its goals at many program levels (Policy Council,
director, coordinators, teachers, and family service workers)
helped increase compliance with study procedures.
-
The programs that cooperated in random assignment of children
to Head Start and non-Head Start groups were extraordinarily
open and responsive to the procedures of random assignment.
Random assignment was so challenging for some programs, even
willing ones, that it could not be implemented. This raises
serious questions about the feasibility of a study that involves
both random selection of programs and random assignment of
children.
-
All QRCs found it a significant challenge to locate a sufficiently
large sample of Head Start eligible children to create a control
group. Some focused their efforts on programs that typically
have long wait lists; others worked with programs where staff
were willing to recruit significantly more children than in
previous years. Random assignment was accomplished most smoothly
when the program's typical recruitment and enrollment procedures
were changed very little.
-
The QRCs experienced varying rates of attrition from the
control groups. In some cases, the compliance rates of children
and families randomly assigned to the control group was very
low.
-
Head Start programs operate with widely varying formats (center-based,
home-based, full-day, half-day); procedures (when and how
recruitment and enrollment take place); and policies (some
children may have priority enrollment). The QRC feasibility
studies had to adapt their research design to these differences.
-
Study designs needed to acknowledge the increased presence
of state-operated and other prekindergarten programs and other
child care options and plan for ways to observe the control
group children in these environments.
-
Assessing the environments of control group children in the
same way as Head Start children was sometimes difficult. Other
child care centers and family child care homes attended by
some of the control group children were more likely to refuse
an observation visit than were Head Start classrooms.
-
Data on quality of the classrooms and attendance of the children
were needed in the analysis plan in order to gauge implementation
efforts.
-
Implementation of the research placed additional work demands
on Head Start program staff. Attention should be given to
the staffing needs and training requirements to carry out
such research, by providing incentives or adding additional
staff who are dedicated to the research activities.
The four QRC pilot studies were not nationally representative,
but this effort to test the feasibility of conducting a random
assignment design in Head Start programs provided important information
that should be considered as the research design for studying
the impact of Head Start is developed further.
Many members of the Committee agree with the Congress and the
General Accounting Office that previous and ongoing Head Start
research, while offering promise for understanding the relationship
between quality and outcomes, should be supplemented in order
to more concisely answer the question of impact. They believe
that the array of current projects will provide considerable data
on Head Start effects on an ongoing basis, chiefly by comparing
outcomes to national norms. However, in its discussions of the
specific mandate from the Congress for a more conclusive national
analysis of impact, the Committee has concluded that additional,
well-designed research on impact, within the context of the broader
research agenda, is needed to respond to policymakers and to inform
the field. As Zigler (1999) states: "After 35 years, Head Start
deserves a study with an experimental design that permits causal
conclusions." The challenge the Committee grappled with was how
to design a study that would be credible, feasible, and provide
information that would help advance thinking and programming in
Head Start and early childhood. This chapter highlighted the previous
research efforts and set the stage for understanding what is already
being addressed through the revitalized research agenda.
|
