This paper is available in alternate formats: | Download Word | Download pdf |
The Work Group developed a series of policy recommendations, for consideration by RTP policymakers, as appropriate, to support a process for reviewing IDEA-Part D products and practices. The Work Group endorsed many aspects of the former JDRP process and suggested that it serve as the basis for design of the new system. However, they made a number of suggestions to correct some of the weaknesses of the former JDRP system and to adapt it to the current educational and political context. Because of limited time and lack of consensus regarding the Federal role in dissemination, the majority of recommendations apply to the review process.
The Work Group recommendations generally involve suggestions for (a) identifying the distinguishing characteristics of an OSEP-sponsored review process, (b) communicating information about the review process and its outcomes to IDEA stakeholders, and (c) establishing operational procedures to support the initiation of the review process internally within RTP, OSEP, and ED. Exhibit 1 contains a summary of these Work Group recommendations, each of which is then discussed next.
Members believed the review process should focus on interventions as well as programs. In some cases interventions may be more likely to be adopted than a multi-faceted program. When complex interventions are reviewed, the developer should identify the critical aspects of the intervention.
Members also recommended that in addition to approving individual practices or interventions, the review process should also identify common elements across approaches and deemed effective highlight commonalties and differences in effectiveness across interventions. Data on approved interventions may also be used to identify areas where more research is needed for use in OSEP planning.
Face-to-face reviews add cost to the process, but members familiar with the JDRP process believed something was lost when they moved away from face-to-face reviews. The face-to-face aspect of review seems important to several members, especially in the initial trials of the review process.
EXHIBIT 1: Work Group Recommendations
CHARACTERISTICS OF THE REVIEW PROCESS
The review process should not focus solely on programs, but should also consider interventions, which may be more likely to be adopted than a multi-faceted program.
The review process should identify common elements across approaches and highlight commonalties and differences in effectiveness across interventions.
Though it adds cost to the process, the face-to-face aspect of review seems important, especially in the initial trials of the review process.
The JDRP/PEP criteria and their emphasis on outcomes should be considered as part of the new system, but they should be expanded to include evidence of effectiveness with diverse students in diverse settings and determining what type of support is necessary to ensure treatment effectiveness.
COMMUNICATION WITH IDEA STAKEHOLDERS
Because of the need to publicize the impact of IDEA-Part D supported activities, the review should be limited to interventions that have received IDEA support at some stage of their development.
The printed materials describing the JDRP/PEP process were very well done and extremely valuable to the field. Any new review process should endeavor to provide the same clarity and breadth in its description of criteria and process.
The publication and dissemination of the successor to "Educational Programs that Work" should be high profile and intense. An attempt should be made to reach every SEA, LEA, and parent group in the country.
Publishers' role in the validation process either as consumers or developers should be explored along with other entities such as educator unions, professional associations, parent groups, and technical assistance providers.
OPERATIONAL ACTIVITIES AT OSEP
OSEP should begin the process of developing an internal review system and concurrently pursue the Department's interest in an agency-wide effort.
There is a need to create cohesion around effective practices in all IDEA-Part D activities. OSEP's model demonstration and outreach competitions should be reconfigured to support the validation process. Likewise, Federal technical assistance, personnel preparation, state improvement grants, and monitoring activities should reflect the importance of including research-validated practices in training and improvement efforts.
There must be different types of expertise and different membership on the review panel to accommodate the variety of research paradigms that the system must handle.
There is the possibility of a public/private partnership between the agency and private foundations to fund a review and dissemination activity.
Finally, members recommended the JDRP/PEP criteria and their emphasis on outcomes should be considered as part of the new system, but they should be expanded to include evidence of effectiveness with diverse students in diverse settings. Evidence of effectiveness should also include statements concerning the type and extent of support that are needed to achieve the reported results and the contextual variables that affect implementation. When intermediate outcomes are used as evidence of effectiveness, they must be linked to long-term outcomes through a sound theoretical framework.
The Work Group members recognized the need to publicize the impact of IDEA-Part D supported activities, and for this reason believe the review should be limited to interventions that have received IDEA support at some stage of their development. This includes commercially available materials.
A consistent theme among the Work Group members is the high quality and utility of the printed materials describing the JDRP/PEP process. These materials guided applicants through the process and were perceived as extremely valuable to the field. Members recommended that any new review process should endeavor to provide the same clarity and breadth in its description of criteria and process.
The publication and dissemination of the successor to "Educational Programs that Work" should be high profile and intense. An attempt should be made to reach every SEA, LEA, and parent group in the country with this publication.
Members cited the role of publishers in the validation process either as consumers or developers as potentially valuable, and recommend that OSEP begin to explore this opportunity. They also suggest linking the new process to other entities such educator unions, professional associations, parent groups, and technical assistance providers.
Members believed OSEP should begin the process of developing an internal review system. At the same time they encourage OSEP to concurrently pursue the Department's interest in an agency-wide effort.
The Work Group also talked about the need to create cohesion around effective practices in all IDEA-Part D activities. They recommended OSEP reconfigure its demonstration and outreach competitions to support the validation process and create a continuum of funding to support research-based practices through development, validation, and replication. Similarly, other Federal initiatives such as technical assistance, personnel preparation, state improvement grants, and monitoring activities should reflect the priority of using research-validated practices.
The review panel will likely be involved in evaluating a variety of practices and products that provide evidence of effectiveness reflecting different research paradigms and methods. In responding to this complexity, members believed the review panel should consist of panelists representing different types of expertise to accommodate the variety of research paradigms the system will encounter.
Finally, the Task Force briefly discussed the possibility of a partnership between the agency and private foundations to fund the review and dissemination initiative. There are several foundations with distinguished track records of supporting educational activities with a disability focus. Investing in this project might give them the kind of visibility they desire. Where as there may be regulatory obstacles preventing such a partnership, it would seem that the possibility of financial support justifies an exploration of the strategy.
The Work Group members proposed a series of actions to be considered by the RTP and OSEP over the next few weeks and months.
First, the Work Group agreed to provide feedback on a summary report of the meeting provided to them by AIR. This report would then become a written document summarizing the Work Group deliberations and could be used by RTP policymakers, as needed, to help identify and organize subsequent Work group activities.
Second, the Work Group recommended that OSEP staff investigate resources, including Federal staff time and reviewer costs that would be needed to establish potential of a review process. A second work group, or a subgroup of the first, could be assigned to develop specifications for a pilot review process that considered criteria of effectiveness, as well as application and review procedures. OSEP staff could create an opportunity to pilot the process in a special competition for outreach or model demonstration projects, to be held in the near future.
Finally, the Work Group recommended that the Research to Practice Division consider publicizing its planned initiative for an IDEA-Part D product and practice review process. For example, as a means of obtaining input from a broader cross-section of the field, findings of the Work Group would be presented at the Research Project Directors' Meeting in July. Work group members agreed to participate in this session, as appropriate and requested by RTP.
Funding for NRCLD is provided by the U.S. Department of Education, Office of Special
Education Programs. Award No. H324U010004. Opinions expressed herein are those of the National
Research Center on Learning Disabilities and do not necessarily represent the position of the U.S.
Department of Education.
© 2007 National Research Center on Learning Disabilities
Last Updated: November 19, 2007
Contact: nrcld@ku.edu