Results
Identifying Nonresponders
One purpose of this study was to explore a dual-discrepancy approach to identifying nonresponders to generally effective intervention. A series of analyses was conducted to determine
- the success of the screening process for identifying a very low-performing risk pool,
- the reliability and validity of the progress monitoring measures, and
- the sensitivity of the dual-discrepancy approach in identifying nonresponders to PALS while minimizing the numbers of false positives and false negatives.
As indicated, we first identified average readers and a risk pool of low-performing readers, using Rapid Letter Naming scores and teacher judgment. To determine whether this screening process accurately identified a group of very poor readers, one-way ANOVAs comparing at-risk and average readers were run on all of the pretreatment measures. The average readers statistically significantly, and dramatically, outperformed the risk pool on all measures, indicating that this screening process successfully earmarked a group of students who were performing reliably and substantially below their average peers (see Table 1).
Table 1: Comparisons of At-Risk and Average-Performing Students on Pretreatment Measures
| Measure | At-Riska (n= 174) | Averagea (n= 87) | ||||
| M | (SD) | M | (SD) | F | (df) | |
| Rapid Letter Naming | 34.02 | (13.40) | 48.75 | (10.37) | 80.74** | (259) |
| Rapid Letter Sound | 20.60 | (13.96) | 31.75 | (11.51) | 41.36** | (259) |
| Segmentation | 20.40 | (14.68) | 26.14 | (12.61) | 9.70** | (259) |
| Word ID | 9.20 | (9.54) | 17.63 | (9.75) | 44.63** | (259) |
| Word Attack | 3.45 | (5.27) | 6.92 | (4.74) | 26.87** | (259) |
| Timed Blending | 12.44 | (7.47) | 16.52 | (6.66) | 18.58** | (259) |
| Untimed Blending | 14.78 | (7.74) | 18.91 | (6.06) | 18.94** | (259) |
| Spelling | 8.83 | (3.48) | 11.63 | (3.02) | 40.49** | (258) |
| **p< .01 | ||||||
aPretreatment data were not available for 2 at-risk students and 1 average-performing student.
The second step was to monitor the progress of the risk pool during the first seven weeks of PALS. To determine criterion validity and test-retest reliability, Pearson rs were calculated among the monitoring levels and pre- and posttreatment scores. Dolch levels correlated strongly with the Word Identification and Word Attack subtests, the near- and far-transfer fluency measures, and the comprehension measure, with coefficients of .82, .52, .92, .93, and .73, respectively. Nonword Fluency levels correlated well with the same measures, with coefficients of .65, .51, .78, .80, and .54, respectively. Each of these correlations was significant at the p < .01 level. In addition, test-retest correlations for the Dolch and Nonword Fluency measures were calculated. These correlations were .88 for Dolch and .87 for Nonword Fluency. Results suggest the monitoring measures were valid indicators of reading-related skills and were stable over time.
Finally, to determine whether the dual discrepancy approach successfully differentiated unresponsive at-risk students from responsive at-risk students and average readers, the CBM levels and slopes of these three groups were compared using ANOVAs. Not surprisingly, there were statistically significant differences among the three groups (Dolch level: F [2, 250] = 47.73; p < .001; Dolch slope: F [2, 250] = 19.13; p < .001; Nonword Fluency level: F [2, 250] = 64.97; p < .001; Nonword Fluency slope: F [2, 250] = 3.99; p < .05). As shown in Table 2, follow-up comparisons indicated that both average-performing and responsive at-risk readers had statistically significantly higher CBM levels and slopes than nonresponders. Effect size contrasts revealed that these differences were not only reliable but, for the most part, large in magnitude, indicating the dual-discrepancy criterion successfully identified a very low-performing group of readers.
Table 2: Comparisons of At-Risk Nonresponders, At-Risk Responders, and Average-Performers in December
| Measures | Comparisons | ||||||||
| Responders vs. Nonresponders | Average Performers vs. Nonresponders | Average Performers vs. Responders | |||||||
| Mean Difference | SE | ESa | Mean Difference | SE | ESb | Mean Difference | SE | ESb | |
| Dolch | |||||||||
| Level | 8.41*** | 1.86 | 0.85 | 18.91*** | 1.96 | 1.09 | 10.51*** | 1.74 | 0.61 |
| Slope | 0.57*** | 0.15 | 0.57 | 1.00*** | 0.16 | 0.82 | 0.43** | 0.14 | 0.35 |
| NWF | |||||||||
| Level | 17.34*** | 2.17 | 1.35 | 25.72*** | 2.28 | 1.51 | 8.38*** | 2.04 | 0.49 |
| Slope | 0.97** | 0.34 | 0.41 | 0.60 | 0.36 | 0.26 | -0.36 | 0.32 | -0.15 |
Note. SE = Standard error; ES = Effect size; NWF = Nonword Fluency.
aEffect sizes were calculated by dividing mean difference by the SD of the responders.
bEffect sizes were calculated by dividing mean difference by the SD of the average performers.
**p< .01, ***p< .001
Previous Page | Next Page
(Method: Procedures) | (Results: Effectiveness)
