I started out this week ready to share a fabulous article with you, excited that I had something very practical and user friendly, for those of you who are teaching reading, just like me.
I found the article in my favourite journal, Teaching Exceptional Children (TEC). This volume of the journal in particular, was a great find, as it shared the best articles recently published by TEC, combined into one “best of” compilation (TEC, Vol 47, No.2, 2014). The article is entitled “A Special Educator’s Guide to Successfully Implementing Evidence-Based Practices” (EBPs), by Torres, Farley & Cook (2014).
The authors provide a ten-step guide for special education teachers, that is easy to follow and practical to use. The steps are as follows:
” 10 Step EBP Implementation Process Checklist
- Determine student, environmental and instructor characteristics
- Search sources of EBPs
- Select an EBP
- Identify the essential components of the selected EBP
- Implement the practice within a cycle of effective instruction
- Monitor implementation fidelity
- Progress monitor student outcomes
- Adapt the practice if necessary
- Make instructional decisions based on progress monitoring data
- Become a leader and an advocate (Torres, Farley & Cook, 2014, p. 88)“
Sounds like a great plan, doesn’t it? With the reading interventions that I use each day at work in mind, I was set to take a closer look at my teaching, to see how it could be improved. I now had some clear steps as to how to go about this.
As a special educator, I know the value in choosing programs and practices that are supported by research, so I was happy to see the words “Evidence-Based” in the title. I want to see measurable improvement, as soon as possible, and I want to be sure that I select programs that are very likely to bring about results. I do not have time to waste, especially when trying to lessen the gap for students who are reading below grade level in the early- and middle-years.
One thing that I have found confusing in my work, however, is the language that is used to describe reading programs, which make it seem like everything has evidence or research backing it. Some of the claims I have come across include: backed by research, research-proven, research based, supported by research, empirically proven, science-based, supported by empirical research, research tested, and on and on. Do these terms sound familiar? It makes my head spin! These descriptions appear to all mean the same thing, but really, they do not.
“…It is rare to encounter an education program or practice that does not claim to be ‘research-based’. This means that the provider claims that research was considered in the design of the program. However, we must go beyond these claims and, in the spirit of healthy skepticism, demand the compelling evidence that a program works. This means we must ask whether the program is ‘research proven’ and not just ‘research based’ ” (Fleischman , 2014, p. 57).
Fleischman’s words caused me some discouragement, when I took a closer look the article I mentioned above. The title, “A Special Educator’s Guide to Successfully Implementing Evidence-Based Practices” specifically states “evidence-based”, not “evidence proven” which now caused a red flag to come up for me. I was disappointed, since when I had first read it, I did not notice the issue with the terminology.
Regardless, the points expressed, to do with properly implementing an educational program, do appear to be very helpful, and I decided to follow the process with the simple change of subbing in a research proven program. Easy enough, right?
I was still left with a problem. How would I know for sure whether a program was research proven? Is there a quick way to find out?
Unfortunately, it turns out, there really is no quick way. Even seeing the description “research proven” is not a guarantee that the program can be trusted more than others. Duke and Martin (2011) provide the following warning: “The terms research-tested, research-based, research-proven, and others are being used by many to elevate the status of their product or approach. However, use of these terms alone means very little. We have to ask questions that allow us to get underneath any individual’s, organization’s, or company’s use of these terms (p. 17).
There are a series of questions that can be asked, in an effort to look critically at the claims made. Please see the article “10 Things Every Literacy Educator Should Know About Research (Duke and Martin, 2011) for a detailed description of how to go about analyzing the claims made in research studies and articles (pp. 17-18)
Duke and Martin go on to say, “Notably, we should not assume that something that is research-tested is inherently more supported by research than something that is research-based. For example, a practice that has been tested and found to be effective in a single study (i.e., research-tested) is, in our view, generally less compelling than a practice that is similar to, but not exactly the same as, a number of practices that have been tested and shown to be effective in a larger number and wide range of studies (i.e., research-based).” (2011, p. 18).
Therefore, it is not simply a question of which terms were used to describe the research or program, and there is no simple way to check whether the results were well-founded or substantiated. No wonder we sometimes feel like throwing up our hands, when faced with so many options that are supposedly research based.
Over the next few months, I will use the process laid out by Torres, Farley & Cook (2014) to look more closely at the programs that I am currently using with students, to explore this issue further (2014, p. 87). I plan to answer the following questions:
Are the claims of “research-based or “research-proven” for this particular program well substantiated?
How was this program tested? Who was it tested on? Is there a match between the people who participated in the study, and the students I intend to use this intervention with? Are they of the same age group and do they have other important characteristics in common? (Duke and Martin, 2011, p. 18)
Does the study look at the same, specific goal that I am working on with my students? (Duke and Martin, 2011, p. 18)
Does the study meet the other criteria described by Duke and Martin, with regards to outcome measures / standards / quality / degree of impact, and so on? (pp. 18-19)
I plan to report on my progress with these programs in this blog! I will follow the steps outlined above, using the implementation checklist and recommendations by Torres, Farley and Cook (2014) to determine my next steps.
I also will strive to improve my success with using these interventions, through collaboration with you!
I would like to invite your feedback, questions an discussions on these topics, below.
References
Duke, N.K. & Martin, N.M. (2011). 10 Things Every Literacy Educator Should Know About Research. The Reading Teacher, 65, 9-22.
Steve Fleischman, “Before Choosing Ask Three Questions” in Proven Programs in Education: Classroom Management and Assessment, ed. Robert E. Slavin (Thousand Oaks: Corwin, 2014),55-59.
Torres, C., Farley, C. A., & Cook, B.G. (2014). A Special Educator’s Guide to Successfully Implementing Evidence-Based Practices. Teaching Exceptional Children, 47 (2), 85-93.
Recent Comments