This paper is available in alternate formats: | Download Word | Download pdf |
After the introductory charge by Lou Danielson (RTP), Lizanne DeStefano (University of Illinois) summarized research and theory on scalability. This presentation was followed by a session in which each Work Group member reflected upon their personal experiences in scaling up research-validated practices. The Work Group followed this by engaging in a group-wide discussion where they built on Dr. DeStefano's presentation, personal reflections, and readings from the literature to identify issues and strategies to be considered in going to scale with effective practices. Following this discussion the Work Group divided into three small groups to discuss the Federal role in this process, using the issues and strategies identified by the large group as a guide. The small groups reported to the larger group the issues and recommendations generated by their discussions. Finally, taking each of the discussions from the entire day into consideration, including personal reflections and both large and small group discussions, the Work Group as a whole identified recommendations to be formally proposed to RTP policymakers, and each member voted for their top choice.
This section on Work Group deliberations summarizes information presented by Dr. DeStefano on the concept of scalability, group member personal reflections, issues and strategies identified by the Work Group, and recommendations generated by the small group discussions of the Federal role in moving effective practices to scale. The formal recommendations of the Work Group as a whole, as derived from these discussions, are then presented in the concluding section of the report.
The meeting began with a presentation by Lizanne DeStefano, summarizing research and theory on scalability. Issues covered during this presentation included the nature of successful change, designing support for change, promoting research-validated practices, and the Federal role in bringing effective practices to scale.
While many children with disabilities have benefited from the accomplishments of successful teachers, schools, and programs, replicating this success on a larger scale has proven to be a difficult challenge. Scaling up effective practices involves change. Much of the literature says change efforts are most successful when there exists a school-wide vision and school climate conducive to learning, enthusiastic and knowledgeable teachers, high-quality curricula and instructional strategies, a high level of engagement, shared decision-making, and parental support and involvement. Little is said about what to do in the absence of these factors.
The literature stresses that the scope and magnitude of the change needs to be considered when planning incentive structures and resources to support it. External (e.g., standards, professional development communities) and internal (e.g., shared vision, researcher/practitioner collaborations) structures are identified in the literature as providing valuable support for change.
The literature says there is need to create intentional processes for reproduction of success. For example, reforms that purport to change practice should embody an explicit theory about how human beings and organizations learn to do things differently. Promoting research-validated practices is a key issue. Mechanisms are needed to promote the trustworthiness, usability (relevance and feasibility) and accessibility of research validated practices. Developers should address the feasibility of the innovation in terms of scope and magnitude of change, the demand characteristics, and amount of professional development and technical assistance needed to support long-term implementation.
The literature talks about the lack of consideration of the unique circumstances of bringing research-validated practices to scale in special education. This is a critical problem given that the majority of instruction for special education students occurs in general education with general education teachers. Local leaders could potentially help. It seems they play a critical role in bringing research-validated practices to scale, though little has been has been written about it.
Another issue discussed in the literature is the Federal role in enhancing large-scale adoption of research-validated practices. This role is complex and that involves many activities such as research, model demonstration, outreach, personnel preparation, technical assistance and dissemination, monitoring, formula and discretionary funding, and accountability. Thus, Federal support of going to scale involves pulling several policy levers that consistently promote best practices in the field.
Each of the Work Group members shared their personal experiences with scaling up research-validated practices. These reflections are briefly summarized below (see Attachment C for a more detailed and comprehensive review of these reflections).
The Work Group members drew from rich and diverse experiences in scaling up research-based practices to identify a beginning set of ideas and issues about going to scale. Some members talked about their experiences with the Joint Dissemination Review Panel (JDRP) process. They perceived the JDRP as providing an important mechanism for reviewing and supporting large-scale adoption of effective IDEA-Part D practices. In a related discussion, members reflected on the value of defining standards of practice as a way of institutionalizing effective practice. They believe publishing criteria of effectiveness in and of itself has an impact by raising the standards for effective practice. Other members talked about issues involved in implementing programs and practices developed with IDEA-Part D support. For example, they reflected on how there are usually specific practices, and essential elements of those practices, that need to be implemented and scaled up, rather than the entire program.
A wide range of other issues were discussed, although not in as much depth or detail as those issues listed above. These additional issues included parent involvement, targeting general and special educators, and key characteristics of the scaling up process.
Members' personal reflections set the stage for a group-wide discussion which generated ten issues considered important in bringing research-validated practices to scale. The Work Group also identified possible solutions or strategies to address these issues. The ten issues and strategies are summarized in Exhibit 1. The four overarching themes derived from these ten issues are then discussed next.
Identifying and disseminating research-validated practices
Work group members discussed how a system designed to identify and disseminate effective IDEA-Part D practices would assist developers and local practitioners in bringing validated practices to scale (JDRP-like process). They believe there should be some way of legitimizing and disseminating what we know about effective practices, and identified this as an appropriate Federal role. In a related vein, they talked about how standards of practice developed through such a process can provide an external reference for change. They believe professional organizations and the Federal government should play a role in defining and promoting standards of practice. The standards of practice would be more generic than a particular intervention, but would define good practice in special education. The standards would serve as the benchmark for monitoring and professional development activities. They should also guide local district and school selection of effective practices.
Exhibit 1: Work Group Issues and Strategies on Scaling Up
| Issue | Strategy |
| A system to identify and disseminate research-validated practices would assist developers and implementers in bringing research-validated practices to scale. |
|
| Standards of practice provide an external reference for change. | Professional organizations and the Federal government should play a role in defining and promoting standards of practice. |
| Developers should differentiate between essential and negotiable elements of an intervention. |
|
| It is necessary to identify incentives or leverage points to encourage local districts to adopt effective practices. |
|
| An effective strategy for going to scale should be designed to support change over time, considering political and institutional barriers. |
|
| Change should be both top down and bottom up--systemic in nature. |
|
| Technical assistance and professional development systems must be constructed to support the change process. |
|
| Any process for going to scale must recognize that most children with disabilities are educated in general education classrooms. General and special educators are both audiences. |
|
| Advocacy/Stakeholder groups can facilitate the scaling up process. |
|
| Marketing strategies are important. |
|
Using incentives and leveraging points to encourage broad adoption of best practices developed with IDEA-Part D support
The Work Group agreed it is necessary to identify incentives or leverage points for encouraging local districts and schools to adopt practices that have been developed and validated with IDEA-Part D support. They discussed how accountability systems should focus on student outcomes, but should also include expectations for standards of practice linked to positive results. They identified multiple leverage points: state, district, and classroom. School accountability systems can provide strong incentives for change at both state and local levels. Continuous monitoring should be focused on implementation of best practices. Furthermore, group members talked about how consumer driven accountability requires that both parents and teachers have access to information about best practices. In a related issue, members discussed how large-scale assessment systems do not always accurately represent students with disabilities. If they are to drive the system, they must be rethought.
Facilitating local implementation and adaptation of interventions
Focusing on individual interventions, the Work Group discussed how developers should differentiate between essential and negotiable elements of an intervention for scaling up. Members talked about how it is important to start with the implementation of a core set of elements and a theory of how they should work. Co-collaboration can develop an intervention beyond the core for a particular context. In fact, group members emphasized how perfect fidelity to program design is unlikely and not necessarily desirable. They emphasized the importance of local communities adapting an intervention to their particular needs and circumstances to facilitate meaningful and relevant implementation that resonates with local needs, and the potential for local adaptation to develop a sense of ownership at the local level. The group further said continual monitoring and research should be conducted to identify if desired outcomes continue to be met. It is also important to determine which adaptations are not desirable.
The Work Group discussed the relationship between implementation, scaling up and change through a number of different perspectives. They believe change should be both top down and bottom up. This relates to adapting an intervention to local needs. Several members said that while a central vision defined by developers of an intervention is important, the process of change should gradually de-emphasize the central vision and shift to local staff driving practice and adapting the intervention to their circumstances and needs.
Using resources to support change over time
Work Group members believe that going to scale should be designed to support change over time, considering political and institutional barriers. One group member reported that many developers demand at least 80 percent "buy-in" from local staff before they will begin working with a school. These issues also relate to designing monitoring and accountability systems aligned with desired change, and using both staff selection and technical assistance to reinforce priorities for change. As staff leave a district or school the process for selecting new staff can focus on recruiting, screening, and eventually hiring personnel who have the skills and values aligned with the goals for change. Group members also talked about how technical assistance and professional development systems can be designed to support the change process. The group believes technical assistance structures should have the following characteristics to effectively support change and scaling up:
Members emphasized that to effectively encourage change, both polices and technical assistance should be coherent and consistently aligned with the goals for change.
Bringing multiple audiences into the change process
The Work Group discussed how scaling up requires effective marketing and dissemination strategies for reaching a wide range of audiences. They talked about how any process for going to scale must recognize that most children with disabilities are educated in general education classrooms. According to the Work Group, general and special educators are both key audiences, and training for both general educators and special educators needs to be addressed. School improvement teams were identified as important target audiences. Members said improvement teams are effective entities for fostering change because they usually represent general and special educators as well as parents. They also see advocacy groups facilitating the scaling up process. In reaching these audiences group members believe marketing strategies are important. Materials should be multi-media and easily accessible. Marketing should be tailored to audiences using various formats (e.g. card in wallet vs. detailed implementation guide). Members also said the role of publishers in this process should be explored.
As a final overarching theme, the Work Group members agreed that developing standards for effective practices in special education would be an important statement about "what we have learned" from IDEA-Part D investments and "what we can offer" the field of education. Developing such standards would also highlight the role of OSEP and other Federal agencies in supporting scaling up activities. The resulting issues about the appropriate Federal role in facilitating the scaling up process are discussed next.
Following the large group discussion where members identified issues that need to be considered, and how they can be addressed, the Work Group members broke into small groups to focus on the question, "What is the appropriate Federal role in scaling up?" The small groups used the issues and strategies identified in the previous session as a guide for their discussions. Each small group reported back recommendations for the larger group to consider. Cutting across the various small group recommendations, three themes emerged, each of which is discussed below.
How can OSEP support the identification of standards of effective practice?
One of the first steps in scaling up is identification of practices, developed and validated with IDEA-Part D support, that help children learn and grow. The Work Group thought that there is a sense that OSEP could play a leadership role in identifying these standards (see Side Bar Ways in which OSEP can Support Identification of Standards of Practice). For example, there is a sense among the group members that districts, schools, teachers, and families in large numbers are adopting programs and practices that have not been validated by a solid research base. At the same time, a coherent and effective system is not in place for identifying what does work and communicating this to these audiences. Therefore, in thinking about appropriate and needed roles for OSEP in the scaling up process, finding ways to support identification of effective IDEA-Part D practices is an important overarching theme coming out of the small group discussions. This involves identification of what works in terms of specific practices, what does not work, and a continuing process of defining standards of effective practice that goes beyond a specific intervention.
Ways in which OSEP can Support Identification of Standards of Practice
One of the key recommendations is to develop promotion of a process for identifying core program elements, practices and parameters for scaling up. Members believe a process for identifying effective IDEA-Part D practices can be fostered through a JDRP-like process which reviews and evaluates promising practices. Such a process can further be used as a means for identifying standards of effective practices for the field, and could serve as a source for learning more about scaling up these practices.
Another way members believe OSEP can promote identification of effective IDEA-Part D practices is to sponsor a meta-analysis or thorough synthesis of the research literature in selected areas. The National Academy of Sciences and other organizations have produced syntheses of this nature in the past. This could be developed in a way that identifies and begins to build a coherent database on effective IDEA-Part D practices tailored primarily for students with disabilities, as well as those practices that are effective with all students, including those with disabilities. Members talked about how research syntheses updated on a regular basis and communicated through various formats could be effective in identifying what works and where gaps exist. They also emphasized how a valid synthesis will require outcomes for students with disabilities be reported in ways that accurately represent their progress.
A related small group recommendation is establishment of a directed research priority that focuses on implementation research. Members talked about how information gleaned from a synthesis of what works could be used to focus the research priority to increase understanding of how effective practices are implemented under different circumstances and conditions. OSEP is currently sponsoring implementation research, but a research priority that pulls this information together in a coherent way could be especially useful. This could be accomplished through a coordinated constellation of projects, or some other format.
The small groups reported additional ways in which OSEP can begin to identify areas where more research is needed. One group suggested OSEP use information about effective practices in their long term strategic planning. Members believe that as these activities are supported by a research synthesis, ongoing strategic discussions could work hand in hand with this information to focus attention on important research to practice needs related to effective practices. Another recommended approach is to choose three high demand areas and begin to identify and disseminate effective practices as a pilot project.
Members also talked about OSEP holding a "Wingspread" type conference to establish criteria for effectiveness. Such a meeting could tap into the rich knowledge existing in the field, create visibility for OSEP's scaling up policy goals, and encourage buy-in among top people in the field on common standards for effective practice.
Finally, members believe it is important to connect these efforts with general education. One small group recommendation is for OSEP to encourage general education to examine effective practices by coordinating with other Federal agencies in identifying and disseminating best practices.
What policies need to be developed by OSEP to facilitate scaling up effective IDEA-Part D practices?
Once standards of effective IDEA-Part D practice are identified, group members believe OSEP can foster implementation of these practices in the field through various policy levers that influence preservice training, monitoring and accountability systems, and technical assistance (see Side Bar: OSEP Policies Designed to Facilitate Scaling Up Effective IDEA-Part D Practices)
OSEP Policies Designed to Facilitate Scaling Up Effective IDEA-Part D Practices
Small group members identified preservice training as a critical source of professional values, accepted norms for practice, and knowledge of effective IDEA-Part D practices. Given this potential influence, it is important that programs expose prospective teachers, administrators and other professional educators to IDEA-Part D endorsed practices. Group members believe OSEP should provide incentives for higher education to imbed these practices into their preservice programs.
Another small group recommendation is that OSEP support and articulate standards of practice for the field of special education by consistently using them in monitoring, technical assistance, personnel preparation, credentialing, and other Federal activities. It is especially critical that a consistent message be sent to practitioners and families on what is effective and endorsed by OSEP. The messages communicated by monitoring, accountability, and technical assistance systems should be aligned with effective IDEA-Part D practices. One group believes OSEP should focus on ensuring that continuous improvement monitoring (CIM) and Federally-funded technical assistance promotes the use of these effective practices at the state level.
Another related recommendation is that standards of practice be developed for technical assistance providers in a manner that facilitates their capacity to help schools and teachers improve services for children. Members suggested that standards for technical assistance should include clear articulation of outcomes that can be expected from assistance provided to clients, expected deliverables, standards for relationships with clients, and documentation of service delivery and results.
Finally, as an additional lever for promoting standards of effective practice, one small group recommended that expectations for using these standards be embedded in priority requirements and new versions of model demonstration and outreach competitions.
How can OSEP facilitate dissemination of information about effective IDEA-Part D practices?
Identification of effective IDEA-Part D practices, and development of policies that encourage adoption of those practices in increasingly broader contexts, are important elements in an OSEP-coordinated process for scaling up promising programs, practices, and reforms. The small groups identified one additional area for Federal involvement: marketing information and educating various external stakeholders about effective IDEA-Part D practices that should be scaled up (see Side Bar: Ways in which OSEP can facilitate dissemination of information about best IDEA-Part D practices). These issues are discussed next.
Ways in which OSEP can facilitate dissemination of information about best IDEA-Part D practices
One group recommendation related to the establishment of a formal review and evaluation process is for OSEP to consider developing a logo for IDEA-Part D practices endorsed by the review process. It would be important to market this logo and communicate what it means, though some members believe a well known logo could reduce the need for other types of dissemination about effective IDEA-Part D practices. Members believe that as schools, teachers, and parents encounter programs and practices that do not have the endorsement logo, they can automatically be placed on alert and should be equipped to ask questions probing into reasons why the practice is not approved.
Another proposed strategy is for OSEP to sponsor a public relations campaign on the importance of using effective practices endorsed by IDEA-Part D. Members believe the campaign should target parents, educators, and professional organizations. This is consistent with the research to practice approach. Members also believe it is important for OSEP to further develop the research to policy connection by helping Congress become more acquainted with information about effective IDEA-Part D practices. Small group members said that linking effective practices with issues of broad concern such as school violence, student achievement, and reading could be especially effective with policymakers. Group members expressed that it is important to demonstrate improvements that special education can contribute to those areas.
In reaching critical local audiences, such as parents, group members recommend using Parent Training Information Centers (PTIs), Regional Resource Centers (RRCs), and other established mechanisms to disseminate information about effective IDEA-Part D practices in "family friendly" language. In part, this may involve building the capacity for linking researchers and PTI professionals. One related idea proposed by a group is to put conditions on Federal research and innovation funds to encourage a broader and different type of dissemination. They believe OSEP should consider alternative strategies for leveraging Part B funds.
Finally, one group recommended that OSEP explore the role of publishers in this process. They believe bringing publishers into the process could provide a broad range of venues and opportunities for connecting with several audiences: parents, teachers, administrators, policymakers, and other stakeholders.
Funding for NRCLD is provided by the U.S. Department of Education, Office of Special
Education Programs. Award No. H324U010004. Opinions expressed herein are those of the National
Research Center on Learning Disabilities and do not necessarily represent the position of the U.S.
Department of Education.
© 2007 National Research Center on Learning Disabilities
Last Updated: November 19, 2007
Contact: nrcld@ku.edu