Preschool Curriculum Evaluation Research

AGENCY:

Office of Educational Research and Improvement, Department of Education.

ACTION:

Notice of proposed priority.

SUMMARY:

The Assistant Secretary proposes a priority for Preschool Curriculum Evaluation Research (PCER). The Assistant Secretary may use this priority for competitions in fiscal year (FY) 2003 and in later fiscal years. We take this action to implement rigorous evaluations of preschool curricula that will provide information to support informed choices of classroom curricula for early childhood programs. We intend this priority to focus support on research that will determine, through randomized clinical trials, whether one or more curricula produce educationally meaningful effects on children.

DATES:

We must receive your comments on or before October 30, 2002.

ADDRESSES:

Address all comments about this proposed priority to Heidi Schweingruber, U.S. Department of Education, 555 New Jersey Avenue, NW., room 602-c, Washington, DC 20208-5501. You may fax your comments to (202) 219-1402. If you prefer to send your comments through the Internet, use the following address: heidi.schweingruber@ed.gov.

FOR FURTHER INFORMATION CONTACT:

Heidi Schweingruber. Telephone: (202) 219-2040 or via the Internet, heidi.schweingruber@ed.gov.

If you use a telecommunications device for the deaf (TDY), you may call the Federal Information Relay Service (FIRS) at 1-800-877-8339.

Individuals with disabilities may obtain this document in an alternative format (e.g., Braille, large print, audiotape, or computer diskette) on request to the contact person listed under FOR FURTHER INFORMATION CONTACT.

SUPPLEMENTARY INFORMATION:

Invitation To Comment

We invite you to submit comments regarding this proposed priority. We also invite you to assist us in complying with the specific requirements of Executive Order 12866 and its overall requirement of reducing regulatory burden that might result from this proposed priority. Please let us know of any further opportunities we should take to reduce potential costs or increase potential benefits while preserving the effective and efficient administration of the program.

During and after the comment period, you may inspect all public comments about this proposed priority in room 602-c, 555 New Jersey Avenue, NW., Washington, DC, between the hours of 8:30 a.m. and 4 p.m., Eastern time, Monday through Friday of each week except Federal holidays.

Assistance to Individuals With Disabilities in Reviewing the Rulemaking Record

On request, we will supply an appropriate aid, such as a reader or print magnifier, to an individual with a disability who needs assistance to review the comments or other documents in the public rulemaking record for this proposed priority. If you want to schedule an appointment for this type of aid, please contact the person listed under FOR FURTHER INFORMATION CONTACT.

General Information

The Secretary believes that research that provides evidence about the effectiveness of preschool curricula for supporting school readiness is essential to improving educational opportunity for all children.

The proposed priority is based on responses to the first PCER competition held in FY 2002. In reviewing the content of applications and feedback from applicants, we have revised the priority. For background information, you may view the original notice soliciting applications for the FY 2002 PCER competition on the Internet at: http://www.ed.gov/legislation/FedRegister/announcements/2001-4/121701b.html.

We will announce the final priority in a notice in the Federal Register. We will determine the final priority after considering responses to this notice and other information available to the Department. This notice does not preclude us from proposing or funding additional priorities, subject to meeting applicable rulemaking requirements.

Note:

This notice does not solicit applications. In any year in which we choose to use this proposed priority, we invite applications through a notice in the Federal Register. When inviting applications we designate the priority as absolute, competitive preference, or invitational. The effect of each type of priority follows:

Absolute priority: Under an absolute priority we consider only applications that meet the priority (34 CFR 75.105(c)(3)).

Competitive preference priority: Under a competitive preference priority we give competitive preference to an application by either (1) awarding additional points, depending on how well or the extent to which the application meets the competitive priority (34 CFR 75.105(c)(2)(i)); or (2) selecting an application that meets the competitive priority over an application of comparable merit that does not meet the priority (34 CFR 75.105(c)(2)(ii)).

Invitational priority: Under an invitational priority we are particularly interested in applications that meet the invitational priority. However, we do not give an application that meets the invitational priority a competitive or absolute preference over other applications (34 CFR 75.105(c)(1)).

Priority

Preschool Curriculum Evaluation Research

Background

The importance of early child-care and preschool experiences in supporting cognitive development and other skills essential to a successful transition into school is a focus of the administration's early childhood initiative—Good Start, Grow Smart. This initiative calls attention to the need for preschool programs to enhance their instructional content in order to ensure that young children start school with the skills that will lead to continued academic success.

The evidence that would allow informed choices of classroom curricula for early childhood programs is weak. Rigorous preschool program evaluations that exist are for programs designed and delivered decades ago. The results from historical evaluations of preschool curricula and current research on the learning and development of young children provide some insights into general features of successful preschool programs. However, they give little guidance for selecting from among the ever-expanding list of available preschool curricula. The proposed priority is intended to address the lack of rigorous, systematic evaluation of preschool curricula currently in use.

PCER is intended to build on recent initiatives aimed at evaluating the preschool experiences of children. These initiatives include The Family and Child Experiences Survey (FACES) undertaken by Head Start, and the Early Childhood Longitudinal Survey-Kindergarten (ECLS-K) and the Early Childhood Longitudinal Survey-Birth Cohort (ECLS-B), both ongoing projects of the National Center for Education Statistics within OERI.

The outcomes of greatest interest to PCER are those skills that are most highly predictive of academic success in the early years of elementary school and that are most amenable to influence by factors within the realm of classroom curricula and practice. These outcomes include language development, pre-reading and pre-math abilities, cognition, general knowledge, and social competence.

The curricula of primary interest to PCER are those with sufficient standardized training procedures and published curriculum materials to support implementation of the curriculum by entities other than, and at a distance from, the curriculum developers. In addition, the curricula of interest to PCER are those that focus on the child outcomes described in the preceding paragraph and those with instructional approaches supported in the scientific literature on learning and instruction.

An applicant is not expected to compare different well-articulated, well-implemented preschool curricula, though we do not discourage these. Rather, we anticipate that the typical applicant will propose to implement a well-articulated, well-implemented curriculum and compare it to the prevailing approach, which is likely to be a home-grown, unlabeled preschool experience that lacks specific instructional goals and a detailed curriculum.

PCER is not intended to support the development of new curricula, nor to support research on interventions for children from birth to 3 years of age. These efforts are the focus of other programs of research to be sponsored by other Federal agencies participating in the Interagency Task Force in Early Childhood Development.

Grantees would coordinate with a national evaluation coordinator, funded separately by OERI, to ensure that evaluations carried out in different locations follow consistent protocols and use a core set of comparable measures. The cross-site data collected by the national coordinator would be returned to grantees in a timely manner for their own use. For the purposes of planning proposed studies and calculating participant burden, a full listing of the measures and procedures used in the 2002-2003 PCER data collection can be found at: http://pcer.rti.org.

Description of Priority

Each applicant must propose to (a) implement one or more pre-kindergarten (pre-K) curricula, with attention to fidelity of the curriculum implementation; and (b) coordinate with a national coordinator the assessment of children and classrooms in the fall and spring of the pre-K year, and in the spring of the kindergarten and first grade year.

(a) Specifically, an applicant must—

(1) Provide a letter of cooperation from participating preschool programs for the purposes of conducting the research. In the letter of cooperation, representatives of the preschool would have to clearly indicate and accept the responsibilities associated with participating in the study. These responsibilities must include—

(i) Agreement to provide a sufficient number of preschool sites and classrooms to participate in the study; and

(ii) Agreement to the random assignment of children or classrooms to the curriculum being evaluated versus one or more comparison approaches;

(2) Provide an on-site coordinator to manage all aspects of data collection, curriculum implementation, and interaction with the national coordinator;

(3) Be prepared to obtain informed consent of parents of children participating in the study, and of all teachers and other administrators from whom data will be collected;

(4) Be prepared to provide all necessary materials and professional development to teachers and staff to implement the curriculum to be evaluated in the intervention classrooms;

(5) Be prepared to make all on-site arrangements necessary for the national coordinator to assess participating children and classrooms;

(6) Be prepared to conduct face-to-face interviews with parents and provide incentives for parent participation in the interviews;

(7) Be prepared to work with the national evaluation contractor for the collection of cross-site data, in coordination with any local data collection activities; and

(8) Be prepared to send at least one representative to attend up to two meetings each year of all of the grantees, national coordinator, and Federal staff. The applicant's budget must include travel funds for these purposes.

(b) An applicant must also do the following:

(1) Be able to guarantee access to a minimum of 10 classrooms with a total of 150 children. The national coordinator can accommodate data collection using the core PCER battery for a maximum of 20 classrooms and 300 children for each applicant. An applicant that proposes to include more than this maximum must include the costs of additional data collection in its budget.

(2) Propose to include only children who are of an age in the first year of the study to be eligible for entrance into public kindergarten in the second year.

(3) Either focus on preschools that serve children from low-income backgrounds or assure that these children are present in significant numbers within the preschool classrooms that are sampled.

(4) Employ random assignment in the evaluation design. A preschool program that is to be a site for curriculum implementation must agree to cooperate fully with the random assignment as a condition for the applicant to receive an award.

To facilitate random assignment, we encourage applicants to consider the use of incentives for schools and families. These may include, but are not limited to: compensation for additional preschool staff time required to cooperate with the research effort; funding for a new classroom; provision of additional resources to enable a program to conduct new activities; securing vehicles for transportation; and stipends to families.

(5) Provide a convincing rationale for its intervention being likely to improve children's outcomes compared with the practices used in the control or comparison conditions. In this regard and for all the projects, we require a reasonable assumption that children in the intervention classrooms will experience neutral to positive outcomes—rather than negative outcomes—compared with children in the control classrooms.

(6) Follow children who participate in studies of PCER curricula that generate educationally meaningful effects at the end of the pre-K year into kindergarten and first grade. The national coordinator will assess all children at follow-up in both the intervention and control or comparison conditions. However, each grantee would be responsible for making arrangements for these assessments, including obtaining parental permission and negotiating access to children for testing in their schools.

The applicant must address how it will provide access to children for follow-up testing. The grantee would also be responsible for conducting interviews with parents, using the established PCER parent interview, each time children and classrooms are assessed by the national coordinator.

(c) Evaluation Design: The applicant must propose an evaluation design that includes the following:

(1) A description of the control condition and the intervention condition or conditions.

(2) An explanation of procedures for random assignment and discussion of procedures for tracking fidelity to the assignment and potential sources of contamination.

(3) The logic of sampling so as to capture, to the degree possible, diversity in the preschool population to be studied. Core variables an applicant should consider for capturing diversity include: race, ethnicity status or language status or both; household income; and parental education.

(4) A discussion of possible variations in the structure of the participating preschool program or programs (part-day or full day, public or private, profit or non-profit, etc.) and how the applicant will take these variations into consideration in the evaluation design.

(5) A discussion of how the applicant will document implementation of and fidelity to the curriculum.

(d) Partners and consultants: An applicant that is not a research organization must obtain the services of at least one consultant who is an established researcher and who has committed enough time to the project to assure the integrity of the local evaluation and to participate in all required meetings.

An applicant that is a research organization may involve curriculum developers or distributors in the project, from having the curriculum developers as full partners in its proposal to using off-the-shelf curriculum materials without involvement of the developer or publisher. Involvement of the curriculum developer or distributor must not jeopardize the objectivity of the evaluation and must not involve a level of professional training or support for the curriculum that rises above that available to ordinary adopters of the curriculum.

In addition, an applicant that is a commercial curriculum developer must indicate in the budget summary the value of any nonfederal resources that would be devoted to the research project, such as their curriculum products.

Additional Considerations

In any given year, the Secretary may, in the application notice, do either or both of the following:

(a) Give competitive preference to applicants proposing to evaluate preschool curricula not currently under study by existing PCER grantees.

(b) Request or require proposals that incorporate complementary research studies to further knowledge of the mechanisms by which curricula support children's learning. The complementary research may address a range of issues related broadly to curriculum effectiveness, such as the impact of curriculum implementation on preschool staff, the influence of individual differences in children on program impact, the development of instrumentation, or other related topics.

Complementary research provides an opportunity to identify outcomes that, because of data constraints, are not explored in the core evaluation or are specific to an individual site. It expands the possibilities for multiple measures of the same variable, and for the development of new measures. Complementary research designs may involve continued pre-K implementations and ongoing research in the pre-K setting for some or all years of the grant while children in the first cohort are being followed into first grade.

Two areas of complementary research are of particular interest:

(1) Studies that address how individual or background differences in children interact with the curriculum to influence developmental outcomes. These studies would address the question: For which children under which conditions is the curriculum most successful?

(2) Studies that compare different versions of the curriculum or different approaches to implementation in order to identify key features of the curriculum and approaches that might improve effectiveness and ease of implementation. These studies would address the question: Under what circumstances does the curriculum achieve the greatest impact?

Intergovernmental Review

This program is not subject to Executive Order 12372 and the regulations in 34 CFR part 79.

Electronic Access to This Document

You may view this document, as well as all other Department of Education documents published in the Federal Register, in text or Adobe Portable Document Format (PDF) on the Internet at the following site: http://www.ed.gov/legislation/FedRegister.

To use PDF, you must have Adobe Acrobat Reader, which is available free at this site. If you have questions about using PDF, call the U.S. Government Printing Office (GPO), toll free, at 1-888-293-6498; or in the Washington, DC area at (202) 512-1530.

Note:

The official version of this document is the document published in the Federal Register. Free Internet access to the official edition of the Federal Register and the Code of Federal Regulations is available on GPO Access at: http://www.access.gpo.gov/nara/index.html.

(Catalog of Federal Domestic Assistance Number 84.305J Preschool Curriculum Evaluation Research Program)

Program Authority: 20 U.S.C. 6031.