Results
Comparing Treatment Effectiveness
Significance tests. A second purpose of this study was to compare the effects of three increasingly individualized treatments on nonresponders' reading performance. A three-factor nested design was used to analyze pre- and posttreatment group differences after 13 weeks of additional treatment. Treatment (PALS vs. Modified PALS vs. Tutoring) was nested within PALS program (Standard PALS vs. PALS + Fluency). Nonresponder status (low vs. very low) was nested within treatment. The pretreatment data were analyzed with 2 x 3 x 2 (PALS program x treatment x nonresponder status) ANOVAs. There were no statistically significant pretreatment main effects or interactions.
The posttreatment data were analyzed with 2 x 3 x 2 (PALS program x treatment x nonresponder status) analyses of covariance (ANCOVAs). ANCOVAs were used because, although there were no statistically reliable pretreatment group differences, we suspected that possible between-group differences may not have been detected due to low statistical power. Thus, we used ANCOVA with December Dolch level as the covariate because it was a more proximate pretreatment measure than those administered in October. Because there were no statistically significant main effects or interactions of interest, we simplified the design of our analysis to one-between group ANCOVAs. Table 3 shows means and standard deviations of the pre- and posttreatment and growth scores of the three groups on all measures. No statistically significant between-group differences were found on any of these measures. Respective Fs (df 2, 52) are as follows: Rapid Letter Naming = .26, p = .77; Rapid Letter Sound = 0.59, p = .56; segmentation = 0.07, p = .93; Word Identification = 1.47, p = .24; Word Attack = 0.79, p = .46; timed blending = 1.48, p = .24; untimed blending = 1.03, p = .36; standard spelling = 1.13, p = .33; developmental spelling = 0.53, p = .59; near transfer = 0.35, p = .70; far transfer = 0.30, p = .74; or comprehension = 0.78, p = .46.
Table 3: Reading Performance by Nonresponder Treatment
| Measures | PALS (n= 21) | Modified PALS (n= 15) | Tutoring (n= 20) | |||
| M | (SD) | M | (SD) | M | (SD) | |
| Pre- and Post-test | ||||||
| Rapid Letter Naming | ||||||
| Pre | 30.14 | (7.64) | 27.60 | (12.84) | 27.30 | (12.83) |
| Post | 47.43 | (17.11) | 44.47 | (19.23) | 45.80 | (19.19) |
| Growth | 17.28 | (13.96) | 16.87 | (13.50) | 18.50 | (15.83) |
| Rapid Letter Sound | ||||||
| Pre | 13.29 | (8.44) | 18.20 | (15.62) | 12.90 | (9.42) |
| Post | 41.71 | (11.15) | 48.27 | (22.75) | 45.50 | (15.54) |
| Growth | 28.42 | (9.11) | 30.07 | (16.44) | 32.60 | (13.59) |
| Segmentation | ||||||
| Pre | 13.62 | (13.00) | 18.53 | (13.19) | 15.05 | (13.68) |
| Post | 35.52 | (13.39) | 33.87 | (14.91) | 35.25 | (14.04) |
| Growth | 21.90 | (12.40) | 15.33 | (12.61) | 20.20 | (13.03) |
| Word Identification | ||||||
| Pre | 2.86 | (2.85) | 5.40 | (5.53) | 4.65 | (6.05) |
| Post | 20.62 | (9.56) | 20.47 | (10.13) | 25.60 | (9.51) |
| Growth | 17.76 | (9.74) | 15.07 | (8.33) | 20.95 | (9.78) |
| Word Attack | ||||||
| Pre | 1.10 | (1.87) | 2.27 | (3.08) | 1.40 | (2.46) |
| Post | 6.71 | (5.26) | 8.67 | (5.55) | 8.95 | (5.36) |
| Growth | 5.62 | (5.10) | 6.40 | (4.45) | 7.55 | (4.99) |
| Timed Blending | ||||||
| Pre | 9.67 | (7.07) | 10.00 | (6.01) | 10.75 | (7.01) |
| Post | 19.38 | (8.54) | 19.00 | (5.94) | 22.70 | (7.14) |
| Growth | 9.71 | (6.70) | 9.00 | (6.23) | 11.95 | (7.56) |
| Untimed Blending | ||||||
| Pre | 12.67 | (7.64) | 13.67 | (7.79) | 13.00 | (7.68) |
| Post | 20.81 | (7.69) | 20.13 | (5.71) | 23.20 | (6.87) |
| Growth | 8.14 | (6.30) | 6.47 | (5.48) | 10.20 | (7.73) |
| Standard Spelling | ||||||
| Pre | 6.86 | (2.01) | 7.47 | (2.59) | 6.95 | (3.33) |
| Post | 12.67 | (3.37) | 11.27 | (3.61) | 12.45 | (2.86) |
| Growth | 5.81 | (3.28) | 3.80 | (2.75) | 5.50 | (3.75) |
| Developmental Spelling | ||||||
| Pre | 28.43 | (16.56) | 32.87 | (18.92) | 30.65 | (26.18) |
| Post | 74.29 | (27.25) | 67.60 | (23.18) | 75.05 | (22.18) |
| Growth | 45.86 | (24.87) | 34.73 | (19.88) | 44.40 | (30.02) |
| Post-test Only | ||||||
| Near-Transfer Fluency | 18.26 | (12.55) | 19.07 | (12.71) | 22.20 | (10.68) |
| Far-Transfer Fluency | 19.29 | (12.22) | 19.90 | (13.69) | 22.95 | (10.08) |
| Comprehension | 0.38 | (0.55) | 0.70 | (0.92) | 0.68 | (0.82) |
In addition to comparing pre- and posttreatment performance, repeated measures ANOVAs were conducted on the CBM levels and slopes to compare nonresponders' performance during the first seven weeks of PALS to their performance during the additional 13 weeks of treatment. Students' CBM levels and slopes in April were significantly greater following the PALS, Modified PALS, or Tutoring treatments than their levels and slopes in December (Dolch level: F [2, 53] = 133.45, p < .001; slope: F [2, 53] = 60.38, p < .001; Nonword Fluency level: F [2, 53] = 313.25, p < .001; slope: F [2, 53] = 65.59, p < .001). This indicates that students made greater gains during spring implementation of the three treatments than during the first seven weeks of PALS implementation. However, there were no statistically significant interactions between treatment and time; that is, the higher levels and slopes in April cannot easily be attributed to the treatments.
Power analysis and effect sizes. A power analysis was conducted to determine whether the treatment groups were sufficiently large to yield statistically significant between-group differences that may have existed. Assuming that the difference between groups would be small to moderate (i.e., an effect size of .30), the sample size needed for a power level of .70 is about 103 students per group. Posttreatment data were available for 21, 15, and 20 students in PALS, Modified PALS, and Tutoring, respectively. Thus, the statistical tests used were low powered for detecting relatively moderate differences. Whereas one might reasonably suggest we could have been more planful, it should be noted that a study of this kind cannot guarantee a specific number of participants. We attempted to maximize the number of nonresponders in our selection process by starting with a relatively large sample (N = 323); nevertheless, it was impossible to ensure an appropriately large sample of nonresponders.
Because statistical analyses were relatively low powered, we calculated effect sizes as another means to explore the importance of the study. We used effect sizes for ANCOVA, again relying on December Dolch level as the covariate. Moderate effects, reported in Table 4, were found
- favoring Tutoring vs. PALS on Word Identification, Word Attack, blending, and comprehension,
- favoring Tutoring vs. Modified PALS on Word Identification, blending, and spelling; and
- favoring Modified PALS vs. PALS on Rapid Letter Sound, Word Attack, and comprehension.
Table 4: Posttreatment Effect Sizes by Nonresponder Treatment
| Measures | Comparisons | ||
| Tutoring vs. PALS | Tutoring vs. Modified PALS | Modified PALS vs. PALS | |
| Rapid Letter Naming | -0.09 | 0.07 | -0.16 |
| Rapid Letter Sound | 0.23 | -0.17 | 0.40 |
| Segmentation | -0.02 | 0.09 | -0.11 |
| Word ID | 0.43 | 0.44 | -0.01 |
| Word Attack | 0.38 | 0.05 | 0.33 |
| Blending | 0.44 | 0.49 | 0.05 |
| Spelling | -0.06 | 0.31 | -0.37 |
| Near Transfer Fluency | 0.22 | 0.18 | 0.05 |
| Far Transfer Fluency | 0.21 | 0.17 | 0.03 |
| Comprehension | 0.32 | -0.02 | 0.34 |
Note. Effect sizes were calculate using effect size for ANCOVA (Wilson, 1996).
Previous Page | Next Page
(Results: Nonresponders) | (Results: After Treatment)
