by Lori-Anne Busch | Sep 8, 2020 | Dyslexia / Specific Disability in Reading, Teaching Reading to Struggling Readers, Understanding Research
Now, here is something to really sink your teeth into. Get yourself a cup of coffee or tea, and find a cozy, quiet spot to read. I have been learning about orthographic mapping over the past many months, and I feel that I am finally getting a grasp on it. This post covers recent reading research that is incredibly interesting, but also quite complex.
The big question we are concerned with today is, “How do students develop the ability to read words with ease?”
It is an important question to answer, because if we can help more students become fluent readers, we will be able to get them to the point where they can read for fun! This happens only when students are able to read words automatically, with ease. When words are instantly recognized, reading turns into something effortless and fun, and kids can dive into books and stories for enjoyment.
When first learning to read, students who have been taught to decode through phonics instruction are able to sound out words letter by letter. This happens very slowly, but eventually, most students are able to recognize words instantly. How does this happen?
Word learning, or developing a sight word vocabulary, involves a mental process called “orthographic mapping” (Kilpatrick, 2016, p. 27).
When we talk about orthography, we are talking about the symbols (letters) that are put together to form words. A word’s orthography is the precise sequence of individual letters, in a precise order (Kilpatrick, 2016, p. 39). Certain letter combinations that occur frequently in English become “unitized” once the child processes them a number of times. This means that the common combinations like “str” and “ing” become single units, with the letters attached together in a chunk (Kilpatrick, 2016, p. 33).
Orthographic Knowledge
Orthographic knowledge is the understanding that there are rules which determine the correct order of letters within words (Zakaluk, 1982 / 1886). This structure is predictable, since in the English language, certain rules govern how letters can be arranged in words (Zakaluk, 1982 / 1996).
“One of the most significant facts about language is that it is structured: orthographic symbols are arranged according to a fixed set of rules and cannot be put together in nonpermissable graphemic sequences. For example, it is appropriate to string together the letters s, t and r to form /str/, but inappropriate, in English to combine them to form the sequence /tsr/…The fact that letters cluster in repetitive patterns thus serves as a generalization that can be applied to decipher new words” (Zakaluk, 1882/1996, Section: Orthographic Mapping, para: 1, bolding mine)
Sight Word – Definition
Reading researchers use the term “sight word” to mean “a familiar word that is recognized instantly, automatically and effortlessly, without sounding it out or guessing. It does not matter if the word is phonetically regular or irregular. The point is that it is immediately recognized” (Kilpatrick, 2016, p. 27). This is important to clarify, since many people use the term “sight word” to mean a word that is spelled in a non-phonetic, irregular way. In orthographic mapping, any word can become a sight word. It happens when the word is instantly recognized by the reader.
When students read with ease, fluently, we know that they can’t be using their decoding skills to do it. It happens too quickly for that. We assume that the student has a picture of the full word stored in the brain, like a kind of visual picture of the word (Kilpatrick, 2016 / Dehaene, 2013). However, this assumption turns out not to be true. In fact, it has been shown by recent brain research that visual memory is not involved in reading, after all (Kilpatrick, 2016, p. 27).
Whole Language Versus Phonics
Dr. Stanislas Dehaene (2013), a cognitive neuroscientist, has explained how recent science proves that words are not stored visually, as whole words. Dehaene explains that even though it feels like we read words as whole words, in actuality, we process letters one at a time, or in familiar chunks, when reading (Dehaene, 2013). This knowledge of how we decipher words using our brains is important, because when teachers learn about this science, it can’t help but impact their teaching practices.
If you are an early-years teacher, this knowledge that words are not stored as “whole words” may cause you to think differently about the kinds of tasks you have your students do. Common practice in many classes, especially for teachers who have a “whole language” philosophy about reading, is to compile a list of irregularly spelled or high frequency words for students to memorize. Students are assigned a stack of word cards, and the words are flashed in front of the child, while the word is said aloud. The child repeats the word, and moves on to the next card.
In this task, typically the child’s attention is not drawn to the internal structure of the word (the letter combinations / the sounds), but rather they notice general way that it looks, the shape of the word. The idea is that the child will remember the full word, once they have seen it and heard it said aloud numerous times. However, we now know that this practice is not likely to be useful for many students. Dehaene (2013), explains that having students pay attention to the “global shape” of a word, attending to the way that the letters descend below or rise above the lines, is not an effective method to develop a student’s sight word vocabulary. Instead what is needed is for students to develop strong letter – sound knowledge. Letter sound correspondence and decoding words, or sounding words out letter by letter is taught through the phonics method. Dehaene’s research is important because it essentially puts the debate to rest about whole language versus phonics instruction (Dehaene, 2013, https://www.youtube.com/watch?v=25GI3-kiLdo).
Phonological Awareness and Letter Sound Knowledge Support the Development of Sight Words
Moats & Tolman (2009) concur that it is not visual memory that is involved in reading, but rather a different process involving our phonological and orthographic processors. If you recall, “phonological” refers to the sounds in spoken language. Phonological awareness is the ability to break words down into syllables, and then into individual sounds or phonemes. Phonemes are usually shown in print by isolating a letter between two slashes, like this: /a/. The slashes around the letter a show the sound that the letter “A” makes, in a word like “cat”.
The phonemes for the word sit are: /s/ /i/ /t/.
For the word stand, the phonemes are /s/ /t/ /a/ /n/ /d/.
Some words have fewer phonemes than letters. An example is that in the word fish, there are four letters, but only three phonemes. The phonemes for fish are /f/ /i/ /sh/.
/sh/ is made up of two letters, but since it makes one sound, it is one phoneme.
As mentioned above, orthography has to do with the symbols in written language, the letters themselves. Therefore, it is both the sounds within a word, and the letters, that are involved in orthographic mapping. An important skill that must be developed then, is the ability to associate letters and their corresponding sounds quickly and instantly.
Dr. Dehaene explains that teaching letter to sound correspondence is the fastest way for children to learn to read and to comprehend what they read. He says that while visual memory is not used in full word reading, it is used in learning the names of the letters of the alphabet. Associating letters and sounds is ” a matter of visual-phonological memorization” (Kilpatrick, 2016, p. 30).
So how does orthographic mapping actually occur? This is a very difficult question to answer; it is hard to conceptualize exactly what is taking place. The information that I have come across on the topic is very intricate, but I will do my best to explain it, as I understand it. I highly recommend the following video, from the Reading League, that explains the process in detail: https://www.youtube.com/watch?v=XfRHcUeGohc.
Phonological Awareness
Kilpatrick believes that all children should receive phonological awareness training all the way through to the end of second grade. At the end of Grade 2, students should be screened to see which skills they still need to focus on. No age is too old, for teaching phonological awareness. Kilpatrick provides lessons for older struggling readers, who, he believes, will continue to struggle with reading if their phonological awareness difficulties are not addressed (2016, p. 18).
Dr. Kilpatrick recommends that we develop our students’ phonological awareness skills to the advanced stages, in order to develop strong readers. As mentioned above, phonemes are the smallest units of sound in a word. Phonological awareness is an umbrella term that encompasses the component skills involved in hearing the sounds in spoken language. These skills, which progress from easiest to hardest, include:
- Hearing distinct words within sentences
- Hearing syllables within words
- Noticing alliteration, or that certain words begin with the same sound (example: sun, sat, sip)
- Hearing rhyming, and being able to produce rhyming words
- Phoneme awareness, or hearing the individual sounds within a word (cat is made up of /c/ /a/ /t/)
- Phoneme manipulation, or being able to swap sounds in and out of a word, to create a new word
When a student is first developing the understanding that sentences are made up of words, and words are made up of syllables, they are demonstrating the beginning stages of phonological awareness. Eventually, students acquire the ability to break individual words into their smallest sounds, or phonemes. Students are able to hear alliteration, and can distinguish the first sounds in words. Usually, they will be able to hear the final sounds next, the sounds at the end of words. Eventually students are able to hear and pick out the sounds in the middle of words.
Phoneme Manipulation
The most complex phonological awareness task, and the task that is most highly connected with successful readers, is phoneme manipulation (Kilpatrick, 2016). Awareness of phonemes comes naturally to some students, and some need to be taught how to do this, in a very specific, direct way. For the 30 – 40% of students who do not develop this awareness on their own, it is essential to teach it, as they will not become strong readers without this ability (Kilpatrick, 2016).
Furthermore, Kilpatrick states that “poor phonological awareness is the most common cause of poor reading. Reading problems can be prevented if all students are trained in letter sound skills and phonological awareness” (2016, p. 13)
Phoneme manipulation is the most difficult of all of the phonological awareness tasks, because it involves taking a word apart (or segmenting it) into its component sounds, and then adding, deleting or changing one of those sounds. Here are some examples of phoneme manipulation tasks, taken from Kilpatrick’s book, Equipped for Reading Success: A Comprehensive Step by Step Guide for Developing Phoneme Awareness and Fluent Word Recognition (2016, pp. 221-221):
Teacher: “Say cord”
Student: “cord”
Teacher: “Now say cord, but instead of /d/, say /k/
Student: “cork”
Here is another one, that is a bit harder:
Teacher: “Say hand”
Student: “hand”
Teacher: “Now say hand, but don’t say /n/”
Student: “had”
According to Kilpatrick, activities like this help students to “map” words, so that they become sight words. In order to do these more difficult tasks with phonemes, students need to have mastered the easier phonological awareness tasks, to the point that they segment a word into phonemes so automatically that they do not even realise that they are doing it (Kilpatrick, 2016). The act of separating and blending sounds has the effect of cementing certain sound sequences together, in the brain.
How does this happen?
- The child takes a known word, divides it into phonemes.
- The child understands that certain letters or groups of letters correspond with each sound in the word, and then bonds these together. The graphemes (letters), are bonded to the phonemes (sounds) in an exact sequence.
- The sequence is specific to a single word, that becomes connected together, so that the word then becomes instantly recognizable (The Reading League, Retrieved on Sept 6, 2020 from: https://www.youtube.com/watch?v=XfRHcUeGohc).
This belief in the importance of teaching phonological awareness and phonics is echoed by Reutzel and Cooter (2016): “A prime objective of exemplary phonics instruction is to develop independent word-recognition strategies, focusing attention on the internal structure of words” (p. 187).
Phonics Defined
Recall here that phonics has to do with both letters and sounds, whereas phonological awareness has to do with just the sounds. Phonological awareness training happens orally, with no visuals, no pencil or paper, or letter tiles. Teaching phonics is visual, involving written letters. A good way to remember this is that you can do phoneme awareness tasks with your eyes closed. Phonics requires your eyes to be open.
Additional support for this kind of teaching, comes from the National Reading Panel (NRP) Report: “Recent findings indicate that connections between phonemes (sounds) and graphemes (letters) must be taught explicitly and that children must practice applying their phonological awareness skills in their reading and writing. These connections must be explicitly taught through phonics instruction, because “deduction does not always occur” (NRP, 2000 as cited by Reutzel and Cooter, 2016, italicized words mine). Some students might learn to read without explicit phonics instruction, yet without it, 30% of students will never become strong readers (Kilpatrick, 2016).
The Orton Gillingham Approach
What methods are best suited to teaching students to orthographically map words? A structured literacy program, in which letters and sounds, letter combinations, and rules for English spelling, are taught explicitly and systematically, is best suited for this. The Orton-Gillingham approach is one example of structured literacy. In this approach, students are taught “jingles” to help remember spelling rules. These rules are taught explicitly, with many examples, pictures and rhymes. Students memorize the rules, and repeat them over and over again, to help with reading and spelling (Bjornson, Orton Gillingham Academy, Classroom Instructor Course, June – July 2020). As Zakaluk (1982/1996) explains, the English language is very structured and redundant, making knowledge of English spelling rules very useful to beginning readers (Section: Orthographic Mapping, para: 1).
Further Explanation of Orthographic Mapping
I found a clear and teacher friendly explanation of how orthographic mapping occurs at another teacher’s blog site, and I strongly recommend it to you: https://sarahsnippets.com/how-do-we-learn-new-words-orthographic/. Sarah’s blog posts include helpful pictures, as well. She too, is a fan of Dr. Kilpatrick.
Thank you, Sarah, for your simplified explanations, and awesome visuals, and also to the Reading League for the very precise and descriptive videos! Please see the links below!
References:
Bjornson, V. (2020) Orton Gillingham Academy, Classroom Instructor Course.
Dehaene, S. (2013). Retrieved from: https://www.youtube.com/watch?v=25GI3-kiLdo.
Kilpatrick, D. A. (2016). Equipped for reading success: A comprehensive, step by step program for developing phonemic awareness and fluent word recognition. Syracuse: Casey & Kirsch Publishers.
Moats, L, & Tolman, C (2009). Language Essentials for Teachers of Reading and Spelling (LETRS): The Challenge of Learning to Read (Module 1). Boston: Sopris West.
Reutzel, D. R & Cooter, R. B. Jr. (2016). Strategies for Reading Assessment and Instruction in an Era of Common Core Standards: Helping Every Child Succeed. Pearson.
The Reading League, Retrieved on Sept 6, 2020 from: https://www.youtube.com/watch?v=XfRHcUeGohc
https://sarahsnippets.com/how-do-we-learn-new-words-orthographic/
by Lori-Anne Busch | Apr 27, 2020 | Dyslexia / Specific Disability in Reading, Evidence-Based Practices Series, Understanding Research
I was first introduced to the Orton Gillingham approach by our divisional school psychologist years ago, when she recommended “skywriting”, for a child who was struggling to remember letter names and sounds. In skywriting, the student writes the letter in the air, while pointing and looking upwards, and visualises the letter as they write it. Using large body movements, the student forms the letter, and says the name, and the associated sound (Gieni, personal communication, 2012). This kind of activity, which involves multiple senses and is kinesthetic, is key to the Orton-Gillingham approach.
More specific directions were given to me just last year, by another school psychologist, who is also an advocate for multi-sensory learning. She suggested having the child use two fingers, the pointer finger and middle finger, while pointing to the sky. She further recommended that the child write the letter with their arm extended, elbow straight. The student was to keep her eyes closed, while pointing up, and to the right, as she visualized the letter, in order to engage the part of the brain that would store the information (Betker, personal communication, 2019).
The psychologist further suggested some other techniques that involved engaging multiple senses. The child was to place a piece of paper over top of sand paper, and then write letters with a pencil, pressing lightly onto the page. This was to provide the student with a “muscle memory” of the shape of the letter. Feedback from the pencil being dragged over sandpaper, would cause a sensation in her hand that would lead to better memory (Betker, personal communication, 2019).
Being a former Roots of Empathy teacher, I was familiar with the idea of developing the brain through engaging a child’s senses, so this made sense to me. I gave it a try, and began to look further into this idea of using multiple pathways in the brain, in order to support learning. I began to see the influence of the Orton-Gillingham Approach in various programs and activities that I came across in my work as a resource teacher.
For some struggling readers at my school, who were still learning the alphabetic principle in Grade 2, the Orton-Gillingham multi-sensory approach was what finally worked, when I tried it with these students. In my office I had found a kit, developed by the Institute for Multi-sensory Education (1998), that was my starting point. It included a DVD that showed how to implement the steps outlined in the manual. Also included were some crayons, Popsicle sticks, a flat, red, plastic mat with a grid pattern, and some “house paper”. This kit was designed for parents, and did not require specific training to use, other than viewing the DVD. Other OG based programs that I am familiar with, including the Wilson Reading System and the Barton Reading & Spelling System, require extensive training.
I have found this approach to be useful, and have seen positive results myself, with these programs, however I want to be certain that I advocate for approaches that have the highest likelihood of resulting in a measurable, positive impact on students in our school division. I am aware of the danger in using anecdotal comments or personal success stories to draw conclusions, even it is my own success story! Instead, I must learn whether the approach has been proven effective, through empirical data.
Here are some quick facts about the Orton-Gillingham (OG) Approach:
- The Orton-Gilligham approach is not new. It was developed by a child neurologist named Dr. Samuel Orton, in the early 20th century (Ritchey &Goeke, 2006, p. 171).
- Anna Gillingham was a “gifted educator and psychologist” who “compiled and published instructional materials as early as the 1930s which provided the foundation for student instruction and teacher training in what became known as the Orton-Gillingham Approach” (Academy of Orton-Gillingham Practitioners and Educators, 2018).
- A curriculum was developed, based on Dr. Orton’s ideas, by Anna Gillingham and Bessie Stillman, and “remains the backbone of Orton-Gillingham (OG) instructional programs and other instructional programs derived from the original curriculum” (Ritchey &Goeke, 2006, p. 171).
- The OG approach was initially designed for students with Dyslexia, to remediate reading deficits, and was taught through one to one tutoring (Ring, Avrit & Black, 2017, p. 384).
- OG is called an approach, not a method because “the latter implies more rigidity in the practice than was intended. The flexibility to meet the needs of their students has since inspired practitioners to modify and adapt the instruction, and is one of the reasons why Orton Gillingham is the basis of many current published curricula” (Uhry & Clark, as cited in Ring et al., 2017, p. 396).
- Many educators in the past actually worked with Orton, Gillingham and Stillman to make adaptations to the original methodology, in order to change it to meet the needs of whole classes of students, for adult learners, and for other educational needs (Ritchey &Goeke, 2006, p. 172). Some programs that are adaptations to the original methodology, include the Wilson Reading System, Alphabet Phonics and Barton Reading and Spelling System. Each of them involve the instructional principles common to the original OG methodology (p. 172).
- In the Orton-Gillingham approach, there is a focus on phonics. The rules of the English language are taught directly and systematically (Ring, Avrit & Black, 2017, p.384). OG lessons present the “units of language …introduced in a systematic sequence of increasing complexity from simple vowels and consonants through multiple syllable words” (Uhry & Clark, as cited in Ring et al., 2017, p. 384).
- “Explicit instruction is provided in phonology and phonological awareness, sound-symbol correspondence, syllables, morphology, syntax and semantics” (Ritchey & Goeke, 2006, 171).
- The learning is cumulative, and students are required to master or overlearn concepts, before advancing in the curriculum (Ritchey & Goeke, 2006, p. 171)
- “A key characteristic of OG reading instruction is that it is multisensory, involving visual, auditory, and kinesthetic/tactile learning pathways, often referred to as the Language Triangle” (Ritchey & Goeke, 2006, p. 171, italics in original).
- The instruction is “individualized to the needs of each student” (Uhry & Clark, as cited in Ritchey & Goeke, 2006, p. 172).
- “OG instruction is to be provided by trained and qualified teachers, tutors or other specialists” (Ritchey & Goeke, 2006, p. 172).
Is the Orton-Gillingham Approach Supported by High Quality, Scientific Research?
In reading articles on this topic, I did find two sources which concluded that the OG approach is supported by research. One of them is a report from the Best Evidence Encyclopedia (BEE) site. BEE is an especially reliable source. This source was recommended by the Council for Exceptional Children journal in the article A Special Educators Guide to Successfully Implementing Evidence Based Practices (Torres, Farley & Cook, 2014) . Bestevidence.org is “a free web site created by the Johns Hopkins University School of Education’s Center for Data-Driven Reform in Education (CDDRE)” Retrieved on April 26, 2020 from http://www.bestevidence.org/aboutbee.htm.
The second source was a 2017 study that tested the effectiveness of two programs, each based on the original Orton Gillingham approach. One program, called “Take Flight”, was found to produce significant overall effects (Ring, Avrit & Black, 2017). Significant gains were shown across many types of reading outcomes, including comprehension, word identification, spelling and word attack skills / decoding (pp. 392-394)
Looking more closely at the information provided by John Hopkins University’s Best Evidence Encyclopedia, this is what I found:
In a summary for educators, entitled Effective Beginning Reading Programs, the Orton-Gillingham Approach was recommended. The Orton-Gillingham Approach is listed in a chart, under the heading, “Limited Evidence of Effectiveness: Weak Evidence with Notable Effects“.
Now, it is not immediately apparent that a program listed in this category, with the words “limited evidence” as the heading, might be a reliable option! However, upon studying the chart more closely, it became apparent to me how few programs actually make the cut, to be included in the BEE Effective Beginning Readings Programs chart.
The chart rates programs under the following five categories:
- Strong Evidence of Effectiveness,
- Moderate Evidence of Effectiveness,
- Limited Evidence of Effectiveness: Strong Evidence of Modest Effects,
- Limited Evidence of Effectiveness: Weak Evidence with Notable Effects, and
- No Qualifying Studies
http://www.bestevidence.org/aboutbee.htm>
- Only three programs, out of almost 200 that were researched, were listed as having “Strong Evidence”.
- Not one made it into the “Moderate Evidence” category.
- Twelve were listed as having “Limited Evidence”.
- Under “No Qualifying Studies” there were over 160 programs listed.
Of the approximately 160 programs, for which there were no qualifying studies, I saw many programs well known to me, listed. These programs are well known to many of my colleagues, and likely to you, and are widely used. Some of the ones included in this category were those based on the OG approach, such as WRS and Barton:
- Hooked on Phonics,
- Wilson Reading System (WRS),
- Seeing Stars,
- Jolly Phonics,
- Fundations,
- Fountas Pinnell Units of Study,
- Barton Reading & Spelling System
This does not necessarily mean that these programs are not effective. It means that currently, there is a lack of sufficient evidence, or studies of high quality, to determine their effectiveness.
The Orton Gillingham (OG) approach is well-known and widely used in schools all over Canada and the USA, however, according to many of the articles I read in learning more about OG, there is limited evidence on its effectiveness. (Ring, Avrit & Black, 2017, p. 384 / Ritchey & Goeke, 2006, p.172 / What Works Clearinghouse, July 2010).
According to Ritchey & Goeke (2006), teachers’ reactions to the approach are “overwhelmingly positive”. However its use is “fuelled by anecdotal evidence and personal experience” (p. 172). Most of the research studies carried out so far on this approach have not met the definition of scientifically-based research (p. 172). This does not mean that the approach is ineffective. It only means that conclusions about its effectiveness cannot yet be made, based on the information that is currently available (Institute of Education Sciences, 2010).
As I mentioned above, I, too, am a teacher who has personally found the approach to be effective with the students I teach. However, my goal in this blog is to seek out programs and strategies that are supported by scientific evidence.
What can I conclude from this week’s foray into educational research?
When taking the course “Diagnostic and Remedial Techniques in ELA” last winter, I couldn’t really understand why my instructor, who is a reading clinician, would often steer us away from “packaged programs”. However, after looking closely into the BEE research summary of packaged literacy programs, I am beginning to see why the instructor was more inclined to choose certain strategies, as opposed to entire programs, for use with students. My instructor taught us that the real strength in teaching reading is in the close observation of individual students, and applying specific strategies to match the students strengths and needs, to move them forward.
It is not enough to purchase a program and follow the steps. Instead we must take a more diagnostic approach, making small adjustments, trying an approach, and noting the results. After each lesson, it is necessary to make minute changes, to help the student learn and advance. There is nothing like a good teacher, observing, adapting and applying a variety of approaches, in the act of teaching reading to a unique individual.
In working with particular students it may very well be appropriate and effective to use the OG approach. It is necessary to measure the students’ progress and to determine the effectiveness of the approach with specific individuals, using assessment data.
I will continue to read articles on the topic of the Orton-Gillingham approach, and will add to this summary over time. If you know of any especially good sources of information on this topic, please comment below!
References:
Academy of Orton-Gillingham Practitioners and Educators (2018). Retrieved April 25, 2020, from https://www.orton-gillingham.com/about-us/orton-gillm/.
Academy of Orton-Gillingham Practitioners and Educators. (2018). Retrieved on April 26, 2020 from https://www.ortonacademy.org/resources/what-is-the-orton-gillingham-approach.
Institute of Education Sciences, 2010. Retrieved on April 25, 2020 from https://ies.ed.gov/ncee/wwc/EvidenceSnapshot/528.
Johns Hopkins University School of Education’s Center for Data-Driven Reform in Education (CDDRE). Retrieved on April 26, 2020 from http://www.bestevidence.org/aboutbee.htm.
Ring, J. J., Avrit, K. J., Black, J. L. (2017). Take Flight: The evolution of an Orton-Gillingham based curriculum. The International Dyslexia Association, Ann. of Dyslexia,67, 383-400.
Ritchey, K. D., & Goeke, J. L. (2006). Orton-Gillingham and Orton-Gillingham-based reading instruction: A review of the literature. The Journal of Special Education, (40), 171-183.
Slavin, R.E., Lake, C., Chambers, B., Cheung, A., & Davis, S. (2009) Effective beginning reading programs: A best evidence synthesis. Baltimore, MD: Johns Hopkins University, Center for Data-Driven Reform in Education. Retrieved on April 25, 2020 from http://www.bestevidence.org/reading/begin_read/begin_read.htm.
by Lori-Anne Busch | Mar 5, 2020 | Understanding Research
I started out this week ready to share a fabulous article with you, excited that I had something very practical and user friendly, for those of you who are teaching reading, just like me.
I found the article in my favourite journal, Teaching Exceptional Children (TEC). This volume of the journal in particular, was a great find, as it shared the best articles recently published by TEC, combined into one “best of” compilation (TEC, Vol 47, No.2, 2014). The article is entitled “A Special Educator’s Guide to Successfully Implementing Evidence-Based Practices” (EBPs), by Torres, Farley & Cook (2014).
The authors provide a ten-step guide for special education teachers, that is easy to follow and practical to use. The steps are as follows:
” 10 Step EBP Implementation Process Checklist
- Determine student, environmental and instructor characteristics
- Search sources of EBPs
- Select an EBP
- Identify the essential components of the selected EBP
- Implement the practice within a cycle of effective instruction
- Monitor implementation fidelity
- Progress monitor student outcomes
- Adapt the practice if necessary
- Make instructional decisions based on progress monitoring data
- Become a leader and an advocate (Torres, Farley & Cook, 2014, p. 88)“
Sounds like a great plan, doesn’t it? With the reading interventions that I use each day at work in mind, I was set to take a closer look at my teaching, to see how it could be improved. I now had some clear steps as to how to go about this.
As a special educator, I know the value in choosing programs and practices that are supported by research, so I was happy to see the words “Evidence-Based” in the title. I want to see measurable improvement, as soon as possible, and I want to be sure that I select programs that are very likely to bring about results. I do not have time to waste, especially when trying to lessen the gap for students who are reading below grade level in the early- and middle-years.
One thing that I have found confusing in my work, however, is the language that is used to describe reading programs, which make it seem like everything has evidence or research backing it. Some of the claims I have come across include: backed by research, research-proven, research based, supported by research, empirically proven, science-based, supported by empirical research, research tested, and on and on. Do these terms sound familiar? It makes my head spin! These descriptions appear to all mean the same thing, but really, they do not.
“…It is rare to encounter an education program or practice that does not claim to be ‘research-based’. This means that the provider claims that research was considered in the design of the program. However, we must go beyond these claims and, in the spirit of healthy skepticism, demand the compelling evidence that a program works. This means we must ask whether the program is ‘research proven’ and not just ‘research based’ ” (Fleischman , 2014, p. 57).
Fleischman’s words caused me some discouragement, when I took a closer look the article I mentioned above. The title, “A Special Educator’s Guide to Successfully Implementing Evidence-Based Practices” specifically states “evidence-based”, not “evidence proven” which now caused a red flag to come up for me. I was disappointed, since when I had first read it, I did not notice the issue with the terminology.
Regardless, the points expressed, to do with properly implementing an educational program, do appear to be very helpful, and I decided to follow the process with the simple change of subbing in a research proven program. Easy enough, right?
I was still left with a problem. How would I know for sure whether a program was research proven? Is there a quick way to find out?
Unfortunately, it turns out, there really is no quick way. Even seeing the description “research proven” is not a guarantee that the program can be trusted more than others. Duke and Martin (2011) provide the following warning: “The terms research-tested, research-based, research-proven, and others are being used by many to elevate the status of their product or approach. However, use of these terms alone means very little. We have to ask questions that allow us to get underneath any individual’s, organization’s, or company’s use of these terms (p. 17).
There are a series of questions that can be asked, in an effort to look critically at the claims made. Please see the article “10 Things Every Literacy Educator Should Know About Research (Duke and Martin, 2011) for a detailed description of how to go about analyzing the claims made in research studies and articles (pp. 17-18)
Duke and Martin go on to say, “Notably, we should not assume that something that is research-tested is inherently more supported by research than something that is research-based. For example, a practice that has been tested and found to be effective in a single study (i.e., research-tested) is, in our view, generally less compelling than a practice that is similar to, but not exactly the same as, a number of practices that have been tested and shown to be effective in a larger number and wide range of studies (i.e., research-based).” (2011, p. 18).
Therefore, it is not simply a question of which terms were used to describe the research or program, and there is no simple way to check whether the results were well-founded or substantiated. No wonder we sometimes feel like throwing up our hands, when faced with so many options that are supposedly research based.
Over the next few months, I will use the process laid out by Torres, Farley & Cook (2014) to look more closely at the programs that I am currently using with students, to explore this issue further (2014, p. 87). I plan to answer the following questions:
Are the claims of “research-based or “research-proven” for this particular program well substantiated?
How was this program tested? Who was it tested on? Is there a match between the people who participated in the study, and the students I intend to use this intervention with? Are they of the same age group and do they have other important characteristics in common? (Duke and Martin, 2011, p. 18)
Does the study look at the same, specific goal that I am working on with my students? (Duke and Martin, 2011, p. 18)
Does the study meet the other criteria described by Duke and Martin, with regards to outcome measures / standards / quality / degree of impact, and so on? (pp. 18-19)
I plan to report on my progress with these programs in this blog! I will follow the steps outlined above, using the implementation checklist and recommendations by Torres, Farley and Cook (2014) to determine my next steps.
I also will strive to improve my success with using these interventions, through collaboration with you!
I would like to invite your feedback, questions an discussions on these topics, below.
References
Duke, N.K. & Martin, N.M. (2011). 10 Things Every Literacy Educator Should Know About Research. The Reading Teacher, 65, 9-22.
Steve Fleischman, “Before Choosing Ask Three Questions” in Proven Programs in Education: Classroom Management and Assessment, ed. Robert E. Slavin (Thousand Oaks: Corwin, 2014),55-59.
Torres, C., Farley, C. A., & Cook, B.G. (2014). A Special Educator’s Guide to Successfully Implementing Evidence-Based Practices. Teaching Exceptional Children, 47 (2), 85-93.
by Lori-Anne Busch | Mar 1, 2020 | Understanding Research
I recently began working toward my master’s degree, and have been struggling to make sense of some of the research studies that I have been reading. Can you relate?
If, like me, you did not take a statistics course in your undergraduate program, and you have not had the privilege of taking a course on interpreting research yet, it is likely that you, too, are having difficulty with some of the vocabulary, as well as the mathematical system, used in the “Results” section of research articles.
Or, perhaps you are you a teacher who is interested in learning more from educational research? Maybe you would like research to guide your instruction, but are unsure of how to tell whether the research you read is correctly and appropriately recommending a given practice?
In any case, it is essential to have a basic understanding of the vocabulary used in research studies, and the mathematical terms that are used. Without this, it is hard to determine what the impact of a given approach might be.
In this post, I will share some of the essential ideas that are common to research articles. I have simplified the definitions, in the interest of clarity, for use by those who are beginners in interpreting research data.
I am hopeful that this post may lead to some discussion and clarification of terms. If you have questions yourself about these ideas, or if you can provide some illuminating examples, please do so, in the comment section below.
How can you tell whether the intervention or practice that is described in a research study has made a positive difference? What do you need to know, in order to look at the results critically? According to Fisher, Frey and Hattie (2016), it is important to ask whether the story, that the research article tells, is convincing (p. 7). To answer this question, we first need to be able to make sense of the data.
John Hattie, whose mission is to inform teachers of the practices that work best in education, has come to his conclusions through an incredibly large review of educational research (Fisher, Frey & Hattie, 2016). In fact, his work is based on over 800 meta-analyses, conducted by researchers all over the world, and includes over 50 000 individual studies, and over 250 million students! His review of the research is considered to be “the most comprehensive review of literature ever conducted” (Fisher, Frey & Hattie, 2016, p. 4)
This brings us to our first term: Meta-Analysis. A Meta-Analysis is when the findings from many, many studies on the same topic are analyzed, to determine if there are any patterns or trends across the studies. They use this information to change and “inform practice” (Fisher, Frey & Hattie, 2016, p. 5). If you find a research article that is a meta-analysis, this indicates a broad swath of information has been summarized, and as such, the conclusions arrived at, would be even more reliable than the individual, associated research studies, that were analyzed in the study.
You likely have heard it said that research can prove anything. Why does it seem so easy for people to claim that their practice is supported by research? If everything works, why bother to look closely at the research that exists (Shanahan, 2019)? It turns out that the answer lies in understanding our next research term: Effect Size.
Effect Size refers to how we quantify the difference that was seen, as a result of the intervention or practice. For example, in studying a practice to see whether it improves reading abilities in students, researchers measure the students’ performance before the study begins, and then after a number of lessons have been taught. They then compare the difference between the scores, at the beginning and at the end of the study (Fisher, Frey & Hattie, 2016, p. 7).
Effect size can also compare two different groups of students, one which received the intervention, and one that did not. The abilities of the students at the end of the intervention, is then compared to see what the difference was, in terms of growth, between the two groups (Fisher, Frey & Hattie, 2016, p. 7). Researchers may also draw conclusions as to any underlying or apparent reasons for these differences, in addition to the specific intervention.
Effect size is shown as a decimal. A very clear explanation of effect size can be found in the book Visible Learning for Literacy, by Fisher, Frey and Hattie (2016). I have simplified the information from this book even further, to assist those who are quite new to reading educational research.
Effect size tells us whether the difference between the students in both groups, or the difference in growth across the study, was large or small. It is shown numerically, as follows:
- d= difference
- d= 0.0 This means that there was no change or growth
- d= 0.2 This means that there was a small improvement
- d= 0.4 Medium improvement
- d= 0.6 Large improvement (Hattie, as cited in Fisher, Frey and Hattie, 2016, p. 7)
Fisher, Frey and Hattie explain that if the number is above 0.0, the practice is said to have had a Positive Effect (p. 8). When research studies tell us that there was a “Positive Effect“, what they are saying is only that the effect was above zero. However, that does not mean that we should decide to invest money and time in the practice, solely on that fact. It is necessary to find out how large the effect was . “It turns out that 95%+ of the influences that we use in schools have a positive effect; that is, the effect size of nearly everything we do is above zero…If you set the bar at showing any growth above zero, it is indeed hard to find programs and practices that don’t work” (Fisher, Frey and Hattie, 2016, p. 9).
Alright then, the next question to ask is, “What is an acceptable effect size?”
John Hattie sets the “bar of acceptability” at 0.4, and calls this the “hinge point” ((Fisher, Frey and Hattie, 2016, pp. 8-9). This is the number we need to look for when reading research. A number of 0.4, or higher, is good. A number of 1.0 is a very, very good. In fact, an effect size of 1.0 would indicate a very noticeable improvement, or an advancement that is “large, blatantly obvious and grossly perceptible” (Cohen as cited in Fisher, Frey and Hattie, 2016, p. 7).
As I’ve said above, the definitions I am providing here are very simplified. There is much more complexity involved, that is outside the scope of this blog post, and at this point in time, beyond my own familiarity and conception. To give us an idea of the complexity involved, Fisher, Frey and Hattie (2016) point out that one must look closely at a particular situation, when interpreting the effect size (p. 7). If a practice has a lower effect size (0.2 or 0.3), there are times when it might still be a good idea to try it with students. Factors that one might consider when determining whether or not to try a practice that has low effect, would be the financial cost of such a practice, and the difficulty involved in its use. If the cost of using a specific practice is so low, and the act of putting it into place with students would be especially easy and quick to do, it might make good sense to try it.
When reading about effect size in Visible Learning for Literacy, (Fisher, Frey and Hattie, 2016), I was happy to have some clarification on a point that I brought up in an earlier post. In “A Sobering Reality” I had mentioned that a previous professor of mine, Dr. Freeze, had explained, that when determining whether or not to continue an intervention, we must check to see how much growth has occurred over the course of a year (Busch, February 24th, 2020). If the students involved have made less than a year’s progress over the course of a year, it is good practice to end the intervention, and to try something else (Rick Freeze, personal communication to author, February 23, 2020).
Why is this mention of “one year’s growth” so important, and where does it come from? Fisher, Frey and Hattie (2016) explain that “students naturally mature and develop over the course of a year, and thus actions, activities, and interventions that teachers use should extend learning beyond what a student can achieve by simply attending school for a year” (italics in original, p. 8). What we need to look for when determining next steps, is whether the results show improvement above the effect of the natural growth that occurs.
After looking at the effect size, the next step is to determine whether the results were Statistically Significant. Effect size and the size of the study (or how many participants there were) are combined to determine whether the practice would have a strong impact on student learning. Hattie and colleagues (2016) even provide a formula to determine this: “Significance = Effect size x Study size” (p. 7). If you are even a little familiar with research studies, you have likely heard that the larger the study (that is, the higher the number of participants in the study), the more it can be trusted. Alternatively, however, it is possible for a study to have a very large number of participants, but a tiny effect size. Therefore, these two factors must be considered together (p. 7).
For more information on the nuances or finer points of the terms defined here, I recommend Chapter 1, in Visible Learning for Literacy: Implementing Practices That Work Best to Accelerate Student Learning (Fisher, Frey and Hattie, 2016)
I would like to invite you to please share any clarifications of the terms I have introduced in this post, in the comments section below. I would especially like to invite you to contribute examples that explain the interplay of effect size and study size on statistical significance, if you happen to be well versed in educational research, yourself.
Also, I invite you to comment below if you find that I have misinterpreted or mis-communicated the concepts described above, keeping in mind that the goal here is to present as clear and simple an explanation as possible. We all become stronger through collaboration and discussion, and I thank you in advance for your ideas!
References
Fisher, D., Frey, N, & Hattie, J. (2016). Visible learning for literacy: Implementing the practices that work best to accelerate student learning. Corwin.
Shanahan, T. (2019). I’m a Terrific Reading Teacher, Why Should I Follow the Research? Reading Rockets. Retrieved March 1, 2020 from https://www.readingrockets.org/blogs/shanahan-literacy/im-terrific-reading-teacher-why-should-i-follow-research
Recent Comments