Rigorous Evidence in Educational Practices

U.S. Department of Education - Research, Statistics and Publications

In reading studies, reports, and especially, journalists' impressions and advocacy articles, the paper entitled Identifying and Implementing Educational Practices Supported By Rigorous Evidence: A User Friendly Guide should be required reading.

It is a well-written 28-page summary of good research design and the problems that can and do occur and the inappropriate conclusions drawn from poorly-designed and implemented research.

It should certainly stop all of us from merely repeating opinions and articles as though they were true, even when they support our own prejudices.

I'm reminded of a quote (paraphrased), I believe from John Tukey: "You can lie with statistics, but you can't tell the truth without statistics."

Quoting from the Executive Summary of this report:

Purpose and Executive Summary
This Guide seeks to provide educational practitioners with user-friendly tools to distinguish practices supported by rigorous evidence from those that are not.

The field of K-12 education contains a vast array of educational interventions - such as reading and math curricula, schoolwide reform programs, after-school programs, and new educational technologies - that claim to be able to improve educational outcomes and, in many cases, to be supported by evidence. This evidence often consists of poorly-designed and/or advocacy-driven studies. State and local education officials and educators must sort through a myriad of such claims to decide which interventions merit consideration for their schools and classrooms. Many of these practitioners have seen interventions, introduced with great fanfare as being able to produce dramatic gains, come and go over the years, yielding little in the way of positive and lasting change - a perception confirmed by the flat achievement results over the past 30 years in the National Assessment of Educational Progress long-term trend.

The federal No Child Left Behind Act of 2001, and many federal K-12 grant programs, call on educational practitioners to use "scientifically-based research" to guide their decisions about which interventions to implement. As discussed below, we believe this approach can produce major advances in the effectiveness of American education. Yet many practitioners have not been given the tools to distinguish interventions supported by scientifically-rigorous evidence from those which are not. This Guide is intended to serve as a user-friendly resource that the education practitioner can use to identify and implement evidence-based interventions, so as to improve educational and life outcomes for the children they serve.

Table of Contents:
1. Title Page
2. Coalition Board of Advisors
3. Purpose and Executive Summary
4. Identifying and Implementing Educational Practices Supported By Rigorous Evidence: A User Friendly Guide
5. I. The randomized controlled trial: What it is, and why it is a critical factor in establishing "strong" evidence of an intervention's effectiveness.
6. II. How to evaluate whether an intervention is backed by "strong" evidence of effectiveness.
7. III. How to evaluate whether an intervention is backed by "possible" evidence of effectiveness.
8. IV. Important factors to consider when implementing an evidence-based intervention in your schools or classrooms.
9. Appendix A: Where to find evidence-based interventions
10. Appendix B: Checklist to use in evaluating whether an intervention is backed by rigorous evidence
11. References

Posted by Larry Winkler at December 19, 2006 8:40 PM
Subscribe to this site via RSS/Atom: Newsletter signup | Send us your ideas