2014 DaSy Framework [PDF]
Self-Assessment | Self-Assessment Comparison Tool
Introduction
The Center for IDEA Early Childhood Data Systems (DaSy Center), funded by the Office of Special Education Programs (OSEP), was charged with developing a data system framework. This framework was developed over 14 months with extensive input from Part C and Part B Section 619 staff from seven partner states.
The purpose of the DaSy framework is to assist Part C and Section 619 programs in developing and enhancing high-quality state data systems and in improving the quality of their IDEA data. The framework is intended to enhance the capacity of Part C and Section 619 state staff to
- Understand the characteristics and capabilities of a good state data system, so they can
- Lead or actively participate in state data system development efforts, including cross-agency work, so they can
- Use their state data systems to comply with IDEA federal reporting requirements and answer important program and policy questions, which will
- Enable states to build better systems of services and programs that will improve outcomes for young children with disabilities and families served under Part C and Section 619.
A high-quality data system provides data for multiple purposes. As reflected in the DaSy framework, these purposes are
- Accountability—federal and state reporting of data
- Program improvement—data to describe the provision of programs and services and the results achieved by young children with disabilities and their families
- Program operations—data that support the day-to-day management and implementation of programs and increase the effectiveness and efficiency of program activities.
High-quality data are fundamental to OSEP’s vision for Results-Driven Accountability (RDA), which focuses on using data to improve results for infants, toddlers, children, and youth with disabilities. OSEP recently introduced changes to its data reporting requirements and monitoring to shift from an accountability system concerned primarily with compliance to one that puts greater emphasis on improving educational results and functional outcomes for children with disabilities. Having high-quality data systems for Part C and Section 619 programs will improve states’ capacity to collect, analyze, and report high-quality data required under IDEA (the Individuals with Disabilities Education Act).
A fundamental assumption underlying the use of data for program improvement and one that is reflected throughout the DaSy framework is that Part C and Section 619 state staff will use data regularly to administer the program. Data-informed decision-making is at the heart of operating and improving programs and ultimately of improving results for children and families. Answering key policy and programmatic questions requires that states have high-quality data and the capacity to access*, analyze, and interpret these data.
Data system is conceptualized broadly in the DaSy framework. It refers to the hardware, software, and other applications that enable Part C and Section 619 programs to collect data about children, families, workforce, and/or program characteristics (e.g., program quality), as well as the analysis, reporting, and data use practices associated with those data. The framework was intentionally written to set a high bar for state data systems; a state that has addressed the entire contents of the framework will indeed have a very good data system. We recognize that few states have achieved the entire range of characteristics described in the framework, but the intent was to be aspirational. The framework also was designed to be comprehensive in that everything necessary for a high-quality data system is addressed. A question repeatedly asked during framework development was, “What does high quality look like?” This question was asked with regard to all facets of the framework’s broad conceptualization of data systems including the kinds of data collected, the uses of data, the process and structures for governing the data, and the processes for developing or enhancing the technology.
States vary significantly in their Part C and Section 619 service delivery systems as well as in their data systems, and the DaSy framework was developed to accommodate this variation.
Show Less
The Structure of the Framework
Subcomponents of the DaSy Framework
The DaSy framework is organized around six subcomponents: Purpose and Vision, Data Governance and Management, Stakeholder Engagement, System Design and Development, Data Use, and Sustainability. The subcomponents are interrelated. For example, the Purpose and Vision subcomponent addresses the mission, usage, and goals of the data system, which are fundamental to other subcomponents. The intended uses of data as addressed in the Data Use subcomponent must reflect the purpose and vision of the data system. Similarly, the data system must be designed to reflect its purposes; developing or enhancing a system is addressed in System Design and Development.
The phases of developing a new system or enhancing an existing one can be viewed as a life cycle, and the framework subcomponents reflect this. The cycle begins with planning and initiation; in the framework, having a process to identify the need for a change—for example, user needs for enhancing reporting capability or the need to address a new federal reporting requirement—is addressed in the Sustainability subcomponent. The Data Governance and Management subcomponent addresses the approval to move forward with the change to the data system, the first step in initiating a new system or an enhancement. The phases and processes related to the development of a new system or the enhancement are covered in the System Design and Development subcomponent. The ongoing operation and maintenance of the system is part of Data Governance and Management subcomponent, with the evaluation of how well the system is meeting user needs and recommendation for changes covered in the Sustainability subcomponent.
Framework Subcomponents and the
Life Cycle of a Data System
Each of the framework’s six subcomponents contains one or more quality indicators (QIs), and multiple elements of quality. Quality indicators are broad statements about actions or activities that state agency staff undertake or the policies, procedures, or documents that the state needs to have in place to support a high-quality data system. Each quality indicator has a corresponding set of elements of quality that operationalize the implementation of the quality indicator. The elements of quality describe various aspects of the quality indicator. In short, quality indicators describe what quality is in a data system, and the elements describe how quality is achieved.
Several critical themes are addressed in multiple subcomponents of the framework. Data quality is one such theme. Policies and procedures related to data quality are addressed in the Data Governance and Management subcomponent, technical features of the data system to promote data quality are addressed in the System Design and Development subcomponent, and the importance of using data to promote data quality is addressed in the Data Use subcomponent. Similarly, the need for various kinds of training and support materials is addressed in multiple subcomponents. An important and aspirational feature of the data system framework is the emphasis on the integration of the Part C and Section 619 data with data from other programs serving young children in the state through the linking of Part C and Section 619 data with the Early Childhood Integrated Data System (ECIDS). Elements across multiple subcomponents address the need for states to build early childhood integrated data systems and for Part C and Section 619 to be a part of these efforts.
Show Less
Development of the Framework
The framework content was developed through an iterative process of literature reviews, information gathering, and multiple rounds of feedback and revisions from state staff in seven partner states and external reviewers. In spring 2013, DaSy invited applications from state Part C and Section 619 programs interested in working on the development of the framework. The seven states selected as partners were Alaska, Arkansas, Connecticut, Georgia, Idaho, Massachusetts, and Pennsylvania. The individual staff members from each state were the Part C and 619 coordinators and the Part C and Part B/619 data managers, along with additional personnel from some of the states. The state staff participated in monthly individual state calls and monthly all-state calls. In addition, the state staff participated in four face-to-face meetings between summer 2013 and spring 2014.
A work group of DaSy staff members and consultants began developing the quality indicators and elements for each subcomponent with a review of the literature. The work group also developed a set of questions to gather information about the current status of that subcomponent in each partner state. Drawing on the literature and the information collected from the states, the work group drafted preliminary quality indicators and elements of quality. These were reviewed by other DaSy staff and revised, shared with the partner states during the all-state and individual state calls, and further revised on the basis of the additional input. The face-to-face meetings provided the DaSy staff and partners with the opportunity to engage in more in-depth discussion and refine the framework content. In the spring of 2014, DaSy staff conducted a series of conference calls with a group of external reviewers (see Acknowledgments) to further refine the content. Final revisions to the framework occurred during summer 2014 and included an overall review by OSEP.
Show Less
Coordination of the Data System Framework with Other Frameworks
ECTA System Framework
The DaSy framework was developed in coordination with two other efforts: the Early Childhood Technical Assistance (ECTA) Center’s System Framework and the Early Childhood Integrated Data System (ECIDS) Planning Guide and Self-Assessment. OSEP charged the ECTA Center with developing a framework for high-quality Part C and Section 619 systems. From the literature and extensive input from six partner states and a technical work group of national, regional, and state experts, ECTA developed a framework to answer the question, “What does a state need to put into place in order to encourage/support/require local implementation of effective practices that result in positive outcomes for children with disabilities and their families?” The purpose of the ECTA system framework is to guide state Part C and Section 619 coordinators, staff, and leadership in evaluating their current state Part C and Section 619 systems and identifying areas for improvement and to provide them with direction on how to develop a more effective, efficient system that supports implementation of effective practices. The ECTA framework consists of six interrelated components: Governance, Finance, Personnel/Workforce, Data Systems, Accountability and Quality Improvement, and Quality Standards.
Because DaSy was tasked with developing a framework for data systems, the two centers agreed that the DaSy’s data system framework would serve as the Data Systems component in the ECTA framework. Accordingly, the DaSy data system framework follows the same organizational structure as the ECTA framework (i.e., component, subcomponent, quality indicator, element) to facilitate use by Part C and Section 619 state staff. The two centers worked closely throughout the development of both frameworks to ensure that the data system framework was compatible with the other areas of the ECTA system framework. Because the data system framework was developed on its own in addition to being a component in the ECTA system framework, it has considerably more subcomponents, quality indicators, and elements than the other five components in the ECTA system framework.
DaSy also coordinated with the technical assistance project working with states on developing an early childhood integrated data system. Funded by the U.S. Department of Education, in 2012 the State Support Team worked with states to develop a planning guide and self-assessment for them to use in building an early childhood integrated data system. As DaSy was beginning the development of the data system framework, the State Support Team was undertaking a revision to the planning guide and self-assessment. Recognizing that state staff could be using the data system framework to improve their Part C or Section 619 data system while using the ECIDS materials as part of their work on an early childhood integrated data system, DaSy committed to making the DaSy framework compatible with the revised ECIDS materials. During the revision, the planning guide and the self-assessment were renamed the ECIDS toolkit.
Components of the ECIDS Toolkit
After discussion with the partner states, DaSy decided to use the same subcomponents (e.g., Purpose and Vision, Data Governance) as the ECIDS toolkit. As the work on the DaSy framework progressed, DaSy staff found that the ECIDS subcomponent Planning and Management was not a good fit with the data system framework; management-related topics became part of the Data Governance and Management subcomponent, and planning-related topics became part of the Sustainability subcomponent in the DaSy framework. DaSy and ECIDS compared the contents of each ECIDS component with the comparable DaSy subcomponent and identified many similarities and some differences that were deemed necessary given the different purposes of the two frameworks.
Show Less
Considerations for Understanding and Using the Framework
As states well know, developing a high-quality Part C or Section 619 data system is a complicated, multifaceted undertaking. The nature and scope of state data systems vary greatly by state. The considerations that follow are important for making the best use of the contents of the DaSy framework.
1. What is quality? The operating assumptions for the framework are that
- A state that has fully implemented all of a quality indicator’s elements has that quality indicator in place.
- A state that has all the quality indicators in the subcomponent in place has high quality in the subcomponent.
- A state that has all the subcomponents in place has a high-quality data system.
Fully implementing an element means that the element is (1) in place and (2) of high quality. For the sake of brevity and because of the extensive variation across states, the framework does not provide much detail on what constitutes quality implementation for each element. For instance, one of the elements speaks to the need for a state to develop data governance policies with input from stakeholders. The element does not describe what constitutes high- or low-quality stakeholder participation. DaSy will be compiling and developing additional resources for states to further clarify quality at the element level.
2. Part C/619 state staff or representative. To the extent possible, the quality indicators and elements were written to identify who is expected to carry out the action described in the statement while also being sensitive to the variations in Part C and Section 619 organization and administration across the states. Therefore, the actors identified in quality indicators and elements are somewhat open ended. When an indicator or element stipulates “Part C/619 state staff,” it refers to staff with knowledge of the program such as a state coordinator or other individuals in the state program office.
When the quality indicator or element uses the phrase “Part C/619 state staff or representatives,” it indicates that the individuals carrying out the action could be members of the state Part C or Section 619 program staff, but the action also could be carried out by someone who does not have direct connections to or knowledge of the program. An example of a “representative” would be a data analyst housed in another department within the agency or a vendor who supports the data system. When an action needs to be carried out by someone with program knowledge, the actor is identified as “Part C/619 state staff.”
3. States have multiple state data systems. States collect multiple types of data related to the implementation of IDEA, and much of the data reside in different data systems (Derrington, Spiker, Hebbeler, & Diefendorf, 2013). For example, in many states, data on children reside in one data system whereas information about the personnel who work with them is in another. The information in the framework applies to every state data system that contains data related to the implementation of IDEA. The phrase “data system” appears many times throughout the framework. In the interest of succinct communication, the framework uses “data system” rather than “data systems” or “data system(s).”
4. What are State Part C/619 data? State Part C/619 data are data related to the implementation of IDEA and encompass different types of data such as data about the children, their families, their services, the providers, and the programs that serve the children and their families. Included are all the data required for IDEA reporting and other data the state collects about the program. As noted above, some of these data reside in different data systems. Once data are included in the state Part C/619 data system, they are considered part of the Part C/619 state data set and within the oversight of the state Part C or Section 619 program, regardless of their origin (e.g., from the local programs or districts).
5. State and local. The framework was written to identify the components of a high-quality state data system. Many of the quality indicators and elements apply equally well at the local level, but they were not written or reviewed with local data systems in mind. Local programs are addressed explicitly in the Data Use subcomponent because quality data are an essential feature of a good data system and when data are used by those who provide the data, the quality of the data improves. For the data received by the state to be high quality, it needs to be used at both the local and state levels. In addition, the return on investment in collecting the data is maximized when data are used at multiple levels. As reflected in the Data Use subcomponent, the framework sees the state as having a responsibility to build the capacity of local programs and districts to use data.
Show Less
Use of the Framework and Next Steps
The DaSy Center has developed a self-assessment based on the framework. The self-assessment enables states to systematically review their status on the elements and generate a numerical profile across the quality indicators. The self-assessment is intended to provide states a current snapshot to help them prioritize improvement efforts, generate a set of scores for states to measure progress over multiple points in time, and serve as a mechanism to encourage state participants to engage in rich conversation about their data systems.
The results of the self-assessment will help a state identify the relative strengths and weaknesses of its data system, but the framework is not a road map for how to build a high-quality data system in that it does not tell a state where to start or what to do next. The state will need to determine where to focus improvement efforts based on its priorities and resources. A state might choose to focus entirely on one subcomponent or on elements from multiple subcomponents. A state might choose to complete the self-assessment for only one or two subcomponents. State staff and their stakeholders can use the self-assessment results to support a planning process that identifies the activities, timelines, resources, and outcomes needed to improve the system. The DaSy Center hopes that states will use the framework and self-assessment in this way, but states might find other ways to use them as well. The framework and self-assessment are designed to be tools to help states build high-quality data systems. There are no rules, only suggestions, for how the framework is to be used. Because the framework and self-assessment were designed to help the states, we encourage states to use these tools in whatever ways they find most helpful.
The DaSy Center will be compiling resources to support improvement activities in each of the subcomponents. Some of these resources will be specific to an element, whereas others will address a quality indicator or subcomponent more globally. The center will also be gathering examples of how states are implementing the elements; these resources are likely to include examples of policies, reports, procedures, data displays, planning documents, and other state-developed tools. The DaSy Center will post these resources to our web site with links to the related part of the framework. The center will be using the framework and the associated resources to guide intensive technical assistance to be provided to a small number of states over the next three years.
Show Less
DaSy Center Technical Assistance Related to the Framework
We hope that states find value in this framework and the additional supports for it that will be coming soon. We encourage states to contact the DaSy Center for technical assistance related to the framework. We can help with finding resources and with improvement activities. We hope the quality indicators and elements are clear, but we can provide clarification if needed. When the self-assessment is available, we can help, for example, with facilitating a stakeholder process to complete the self-assessment or a strategic planning process to make use of the results. We look forward to working together to improve the quality and use of data to improve programs for young children with disabilities and their families.
