Results of a Three Tiered Model Implementation:
The Heartland Early Literacy Project (HELP)
Results: Phonemic Segmentation and Oral Reading Fluency Gains Expressed as Z-Scores
There are a number of different ways to present and summarize results from this project. One clear and standard way to demonstrate the magnitude of the effectiveness of improvements in teaching and learning is through the use of effect sizes. Effect sizes illustrate, on a common scale, the relative growth in student performance over time. In this analysis, growth for each school building is examined.
The following specific procedures were used to demonstrate growth in student skills:
- Each Heartland school building that has participated in the HELP project for 2 or 3 years has been included in this analysis. A list of districts and buildings included in this analysis are presented in Table 1. Buildings just entering the HELP process in the 01-02 school year or the 02-03 school year were not included in this analysis due to the lack of available growth data (at least two years of data are required in order to calculate an effect size).
- For each participating district, their first year of participating in HELP was designated as "year 1" regardless of which school year that year represented.
- Data were collected from all kindergarteners on phonemic segmentation fluency (PSF) in the spring of each participation year. Data were collected on oral reading fluency (ORF) in the spring of each year.
- For each school, data on both indicators (PSF and ORF) were averaged for each year of participation.
- Next, a z-score was calculated for year 1 to year 2 growth (85 buildings included) for each building, a z-score was calculated for year 2 to year 3 (36 buildings included) growth and year 1 to year 3 growth (36 buildings included). z-scores were calculated using the following standard formula
Post Mean-Pre Mean/Pre Standard Deviation - In the z-score calculation, the standard deviation used in the calculation was the standard deviation from the sampling distribution of means in year 1. A second analysis was run with the population standard deviation for all students in our sample for each measure during spring of year 1. Similar results were found.
- A mean and median z-score was calculated across all buildings for - year 1 to year 2 data, year 2 to year 3 data, and year 1 to year 3 data.
- Results from these analyses are presented in tabular format (Tables 3 and 4).
- To depict the range of effect that HELP has had in different schools, a frequency distribution was created for end of year Kindergarteners on the PSF measure and for end of year first graders on the ORF measure. These graphics are presented in Figures 7 and 8.
Table 3. Growth in Kindergarten Phonemic Segmentation Fluency, Represented in z-Score Units, Heartland Early Literacy Project 2002-2003.
| Yr1-Yr2 z-Score | Yr 1- Yr 3 z-Score | |
| Mean Z Score | 0.71 | 1.08 |
| Median Z Score | 0.70 | 1.25 |
| Number of Scores | 85 | 36 |
| Low Z Score | -3.76 | -0.77 |
| High Z Score | 3.93 | 3.29 |
Table 4. Growth in First Grade Oral Reading Fluency, Represented in z-Score Units, Heartland Early Literacy Project 2002-2003.
| Yr1-Yr2 z-score | Yr 1- Yr 3 z-scores | |
| Mean Z Score | 0.26 | 0.39 |
| Median Z Score | 0.32 | 0.36 |
| Number of Scores | 86 | 32 |
| Low Z Score | -2.15 | -0.68 |
| High Z Score | 2.49 | 2.47 |
Cohen (1977) has described the magnitude of effect sizes in the social sciences. Effects of 0.20 are described as small, effects of 0.50 are described as moderate and effects of 0.80 are large. On this scale, the median effects for phonemic awareness can be described as very large. The median effect for reading fluency would be considered more moderate.
In addition to depicting project results as summary statistics, school-level z-scores were used to create a graphic illustrating the distribution of "amounts of growth" experienced across the project. For PSF, a broad range of results were experienced by individual schools. Three schools' performances actually declined across a 3-year period participating in the project. The remaining schools all experienced positive growth in PSF
Figure 7. Frequency distribution of end of kindergarten PSF effect sizes for HELP project schools from first to third year implementation.
Figure 8. Frequency distribution of end of first grade ORF effect sizes for HELP project schools from first to third year implementation.
Previous Page | Next Page
Oral Reading | Identification Rates
