Reading Aloud to Children – Evidence-Based Practices Series

Reading Aloud to Children – Evidence-Based Practices Series

by | Mar 23, 2020 | Evidence-Based Practices Series

Quote of the Day:
“It has been contended that one of the most important things a parent can do, beyond keeping children healthy and safe is to read to them” (Joyce, as cited in Ledger and Merga, 2018, p. 135).

With today being the first official day, here in Manitoba, of school closures due to COVID-19, I am considering how to best support families who are at home with their children over the next few weeks.  I am especially interested, as I am sure other teachers and parents are, in maintaining the growth in reading skills that students have developed over the course of the school year, thus far.

Over the next few weeks, my blog posts will highlight various evidence-based practices (EBPs) to support children in developing literacy skills.  In this section of my blog, I will highlight the research in support of these EBPs, and then will provide a user-friendly synopsis in the Parent Resources section of my site.  These brief summaries will list the steps parents can follow, to implement the various practices at home.

The first topic in my “Evidence-Based Practices Series” is reading aloud to children.  Reading aloud with kids is something that many parents already do, and perhaps may decide to do more often, during the time they spend at home with their kids this month.

Parents can feel confident that they are making a difference for their kids, when reading aloud with them, especially considering that “the single most important activity for building the knowledge required for eventual success in reading, is reading aloud to children,” according to a report commissioned by the US Department of Education (Anderson et al., as cited in Ledger & Merga, 2018).

Ledger & Merga (2018) carried out a research study in which students’ perceptions to being read aloud to, were highlighted.  They defined Read-Alouds as “the shared reading experience between a child and a parent/guardian or teacher” (p. 125).  A Read-Aloud can be a parent reading the whole book to the child, or child and adult taking turns reading. 

I was very surprised to learn how infrequently children are read aloud to, by both their teachers and their parents (or at least how infrequently it occurs, according to the students who were interviewed!)   Although, I have to say, considering that many researchers are beginning to place more value on students’ perspectives, when looking at ways to improve learning, I found this study to be very progressive (Cook-Sather, 2006)!

74% – 79% of students who took part in the study, which included students up to 12 years of age, stated that they enjoyed being read to.  In fact, Ledger and Merga (2018) found that most kids wish that their parents and teachers did so more often (p. 131).

The list of skills that are developed as a result of teachers and parents reading aloud to kids include:  increased vocabulary, reading comprehension, cognitive skills, syntactic development, encouragement of children to read books themselves, including children who are less able to read (Ledger & Merga, 2018, p. 125).

Additionally, questioning skills, engagement in dialogue with another person, and even inquiry-based learning, are all developed in children, when they are being read aloud to, according to Trelease (as cited in Ledger & Merga, 2018, p. 125).

Most importantly, the development of a positive attitude toward reading, and the decision to read for fun, can be developed through reading aloud to children, and can actually prevent difficulties with literacy later on in life (Ledger & Merger, 2018, pp. 125 – 126).  The positive impact of taking part in Read-Alouds continues for kids, even after they have become independent readers.

There are also ways to make Read-Alouds more effective at improving literacy skills.  When parents use story time to talk about the meaning of the story, to explain vocabulary and to connect the story to the child’s own life in a more interactive style, it has been shown to have the highest impact (p. 555).  Story time can even be used to directly teach pre-reading skills, in a natural and fun way.  Duursma et al. (2008) provide an explanation for how reading aloud can help students to develop phonological awareness, when the adult directs students’ attention to the initial sounds in words (p. 554):

“Many alphabet books…contain the letter name accompanied by pictures of objects whose names begin with the critical sound, such as D, for example a dog, deer or doctor.  When parents stress the initial sounds in these words while reading with their children, they are teaching awareness of phonemes or shared phonemes across words.  Since children who have difficulty with phonological awareness can develop reading difficulties, parents might help prevent these difficulties by exposing children to a wide variety of literacy materials and helping them become aware of the relationship between letters and sounds” (Duursma et al., 2008, p. 554).

It is important to note that, as Ledger and Merga (2018) found in their interviews with students, some things caused kids to dislike being read aloud to.  For example, students did not like it when the adult over did it, in asking them questions about the book, or when they put too much of a focus on words (p. 130).  At school, some students disliked the frequent interruptions of the story that occurred, when the teacher needed to pause to remind other student about their behavior (p. 131).

It is imperative to consider the attitude that the child is developing about reading, as you read with them.  If the experience is relaxed and fun, the child will be more likely to approach books in the future with positive feelings (Ledger & Merga, 2018, p. 126).  This may eventually lead to students choosing to read books just for the fun of it. Some ways to increase the positive emotions kids may have while reading include, allowing the child to choose the book, using books that are funny, or ones that focus of topics of interest to the child, and developing a regular routine, such as reading books at bedtime.  Also, it is important to be careful not to put “undue weight …on skill development at the expense of enjoyment and entertainment (p. 134)

 In terms of interventions for struggling readers, which is the main focus of my blog, it is interesting to note the following: 

1)  Richard Allington (2012) recommends that when having children read, we ensure that the books we use are easy, or just right for the child:  “Good readers read with accuracy almost all the time. The last 60 years of research on optimal text difficulty—a body of research that began with Betts (1949)—consistently demonstrates the importance of having students read texts they can read accurately and understand. In fact, research shows that reading at 98 percent or higher accuracy is essential for reading acceleration. Anything less slows the rate of improvement, and anything below 90 percent accuracy doesn’t improve reading ability at all (Allington, 2012; Ehri, Dreyer, Flugman, & Gross, 2007, highlighting mine).

2) Parents who read aloud to their children prevent challenges with reading, once the child enters school. A recent analysis of 29 studies suggested that read-aloud interventions “provide children at risk of reading difficulties with higher literacy outcomes than those who did not participate in these interventions” (Swanson et al., as cited in Ledger & Merga, 2018, p. 125).

3) Read-Alouds were categorized under the heading “Literature-Based Program” on the Best Evidence Encyclopedia website.  See http://www.bestevidence.org/.  This site classifies programs according to the “overall strength of the evidence support in their effects on reading achievement” (Slavin et al., 2009).  A score of +0.21 was given, which would place it in John Hattie’s “likely to have a positive impact on student achievement” category (Fisher, Frey & Hattie, 2016).

If you are a parent, do you expect to spend more time reading aloud with your kids over the next few weeks or months, due to the school closures?

What strategies do you use to make story time fun or relaxing?

What books have you found your students enjoy the most?

 

 

References

Allington, R. L., & Rachael E. Gabriel.  (2012).  Every Child, Every Day.  Reading:  The Core Skill, 69 (6), 10-15.

Allington R. L., & Gabriel, R. E. retrieved on March 24, 2020 from http://www.ascd.org/publications/educational-leadership/mar12/vol69/num06/Every-Child,-Every-Day.aspx

Cook-Sather, A.  (2006).  Sound, presence and power:  “Student Voice” in educational research and reform.  The Ontario Institute for Studies in Education of the University of Ontario.  Curriculum Inquiry, 36 (4), 359-390.

Duursma, E., Augustyn, M., & Zuckerman, B. (2008).  Reading aloud to children:  The evidence.  Archives of Disease in Childhood, 93, (7), 554-557.

Fisher, D., Frey, N, & Hattie, J. (2016).  Visible learning for literacy:  Implementing the practices that work best to accelerate student learning.  Corwin.

Ledger, S., & Merga, M. K. (2018).  Reading Aloud:  Children’s attitudes toward being read to at home and school.  Australian Journal of Teacher Education, 43 (3), 124-139.

Slavin, R.E., Lake, C., Cheung, A., & Davis, S. (2009). Beyond the basics: Effective reading programs for the upper elementary grades. Baltimore, MD: Johns Hopkins University, Center for Data-Driven Reform in Education. Retrieved on March 23, 2020 from http://www.bestevidence.org/reading/upper_elem/upper_elem_read.htm.

 

The 10 Most Important Things to Know About Teaching Reading to Students with ID or SCD  

The 10 Most Important Things to Know About Teaching Reading to Students with ID or SCD  

by | Mar 8, 2020 | Reading Instruction for Students with SCD

Here it is!  My very first Top 10 List, as a blogger!

What follows is a brief summary of the research I have read up to this point in time, with regards to teaching students with Intellectual Disability (ID) or Significant Cognitive Disability (SCD) to read..

Please scroll down to see details to explain each point, and references.

  1. IQ scores are not as important as you might think, when it comes to choosing reading interventions.
  2.  Some practices in teaching reading are well accepted, due to the large amount of research that has amassed over time to support them.   
  3. Students with ID / SCD require achievable goals to be clearly identified by teachers.
  4. It may be necessary to use alternate forms of assessment when determining whether an intervention has been effective for your students with SCD.
  5. It is important to understand and accept that progress for students with Significant Cognitive Disability (SCD) or Intellectual Disability (ID) will likely be very slow.   
  6. When identifying practices and programs for use with students, it is necessary to be skeptical and to look at research studies and articles critically.
  7. It is essential to have a basic understanding of the vocabulary used in research studies to determine the impact of a given approach.
  8. Deciding whether to continue an intervention with students with SCD should be based on data. 
  9. Use the principles of educational psychology, when teaching struggling readers, and in particular, those with SCD.
  10. Provide many, many repetitions, to enable the student to learn. Build the skills to the point that the student recalls the information with automaticity.

 

Here is the list a second time, with points to elaborate.

 

  1. IQ scores are not as important as you might think, when it comes to choosing reading interventions.

 

    • We do not need to teach reading differently to students with low IQ, as compared to other struggling readers (Stuebing, K. K., Barth, A. E., Molfese, P. J., Weiss, B., & Fletcher, J. M., 2009).

 

    • In a meta-analysis that looked at how students with low IQ learn to read, as compared to struggling readers who have average IQ, it was found that there is no difference in how these two groups of students learn to read (Stuebing, K. K., Barth, A. E., Molfese, P. J., Weiss, B., & Fletcher, J. M. (2009).

 

    • IQ scores do not have a role in planning interventions, or in matching interventions to readers, since there is no important difference in how we should approach teaching reading, to the two types of students (Stuebing, et. Al., (2009).

 

    • There is no difference in “growth patterns” and “no significant differences between these two groups of readers on how they develop reading precursor skills” (Wristers, Francis, Foorman, Fletcher, & Swank, as cited in Joseph, 2002).

 

 

    • It is a good idea to use reading programs that were designed for struggling readers in general, with those who have low IQ. Students with ID “should be provided with evidence-based reading instruction” (Allor et al., 2014, p. 302).

 

  1. Some practices in teaching reading are well accepted, due to the large amount of research that has amassed over time to support them.   
  •  “…it makes the most sense to think about research as proceeding as a slow accumulation of knowledge over time and to read across many different studies on a particular question or topic.  Although breakthroughs or headline-making studies periodically appear, it is usually a mass of related studies over a period of years that lead to a well-accepted or durable conclusion” (Duke and Martin, 2011, p. 21).

 

  • These features (below) were described as essential in teaching students with Intellectual Disabilities (ID) by Allor, et al., (2014)
    • Use Direct Instruction
    • Use highly detailed lessons allowing for explicit instruction
    • A fast pace is often necessary, to maximize student engagement and motivation
    • It is necessary to model all skills repeatedly and clearly
    • The teacher must carry out frequent cumulative reviews, to ensure mastery and maintenance of skills (p. 293)
    • Use of very “consistent, explicit, and repetitive routines, focusing on key-skills” is important (p. 303)
    • Individual pacing and behavioral supports are often necessary
    • It is necessary that groups be kept small, with only one to four students to each teacher (p. 303).

 

 

  1. Students with ID / SCD require achievable goals to be clearly identified by teachers.
    • Just as is necessary when teaching students without disabilities, clear goals need to be determined at the outset.

 

    • Lemons, C.J., Zigmond, N, Kloo, A.M., Hill, D.R., Mrachko, A.A., Pattera, M.F., Bost, T.J. & Davis, S.M. (2013), reasoned that teachers would be better equipped to create realistic, appropriate goals if they had better data on the existing skills of this population of students, and could measure progress more accurately and frequently (p. 410).

 

  1. It may be necessary to use alternate forms of assessment when determining whether an intervention has been effective for your students with SCD.

 

    • One difficulty is that when reading tests, designed for use with the typical student population, are used with students with disabilities, it is not known which tests to use and for what grade-level (Lemons et al., 2013, p. 411).

 

    • “More sensitive measures are needed to determine when small amounts of progress are being made so teachers and students will recognize the results of their hard work and continue instruction that is effective” (Allor, Mathes, Roberts, Cheatham & Otaiba, 2014, p. 304).

 

  1. It is important to understand and accept that progress for students with Significant Cognitive Disability (SCD) or Intellectual Disability (ID) will likely be very slow.   

 

    • “Students with low IQ do benefit from comprehensive reading programs that were designed for struggling readers, and readers with LD, but progress is slower” (Allor, Mathes, Roberts, Cheatham & Otalia 2014, p. 303).

 

    • The progress one might expect to see could may take many years, as opposed to many months, to achieve. Allor et al. (2014) noted that it took between one and three years for students in Grades 3 to 8 with Intellectual Disability, to achieve reading scores in the average range on Grade 1 reading passages.

 

  1. When identifying practices and programs for use with students, it is necessary to be skeptical and to look at research studies and articles critically.

 

    • “…it is rare to encounter an education program or practice that does not claim to be ‘research-based’. We must go beyond these claims and, in the spirit of healthy skepticism, demand the compelling evidence that a program works” (p. 57).

 

    • Seeing the description “research proven” is not a guarantee that the program can be trusted more than others.  Use of these terms alone means very little. We have to ask questions that allow us to get underneath any individual’s, organization’s, or company’s use of these terms (Duke and Martin, 2011, p.  17).

 

    • There are a series of questions that can be asked, in an effort to look critically at the claims made.  Some questions one might ask are:
        • How was this program tested?
        • Who was it tested on?
        • Is there a match between the people who participated in the study, and the students I intend to use this intervention with? 
        • Are they of the same age group and do they have other important characteristics in common? (Duke and Martin, 2011, p. 18)
        • Does the study look at the same, specific goal that I am working on with my students? (Duke and Martin, 2011, p. 18)
        • Does the study meet the other criteria described by Duke and Martin, with regards to outcome measures / standards / quality / degree of impact, and so on? (pp. 18-19)
  1. It is essential to have a basic understanding of the vocabulary used in research studies to determine the impact of a given approach.

 

    • In terms of effect size, John Hattie sets the “bar of acceptability at 0.4, and calls this the “hinge point” ((Fisher, Frey and Hattie, 2016, pp. 8-9). This is the number we need to look for when reading research.

 

    • For more information on understanding research, please see my previous post, “Important Research Terms Defined for New Graduate Students and Interested Teachers” (Busch, Mar 1, 2020)

 

  1. Deciding whether to continue an intervention with students with SCD should be based on data.  

 

    • Take baseline data before you begin your intervention.

 

    • Select interventions that are proven by research to be effective for struggling readers

 

    • After a period of six months, take data again, to see whether growth in the specific skills you are targeting has occurred. Use this information to decide whether to continue with the same intervention, whether to make some important adjustments, or to try something else.

 

 

    • If the students involved have made less than a year’s progress over the course of a year, many people believe it is good practice to end the intervention, and to try something else (Rick Freeze, personal communication to author, February 23, 2020).

 

    • Fisher, Frey and Hattie (2016) explain that “students naturally mature and develop over the course of a year, and thus actions, activities, and interventions that teachers use should extend learning beyond what a student can achieve by simply attending school for a year” (italics in original, p. 8).

 

  1. Use the principles of educational psychology, when teaching struggling readers, and in particular, those with SCD.

 

    • “Delivering reinforcers for efforts made toward achieving a goal can be considered as ways of providing support to students. This cannot be stressed enough when working with children with reading difficulties” (Joseph, 2002).

 

    • Use immediate reinforcement of behaviors that you want to increase.  “…it is crucial that educators and parents shape reading behaviors through praise and rewards contingent upon efforts made at achieving reading skills” (Joseph, 2002)

 

    • Use scaffolding to provide necessary support to the child when introducing new information then, fade your support over time. Your prompts will be “gradually faded once the child approximates independent functioning while completing tasks” (Wood, Bruner, and Ross as cited in Joseph, 2002)

 

  1. Provide many, many repetitions, to enable the student to learn. Build the skills to the point that the student recalls the information with automaticity.
    • “Within the time allotted for literacy activities, students need opportunities to make frequent responses during oral and silent reading as well as writing lessons” (Joseph 2002).

 

    • Children with learning disabilities [and significant cognitive disabilities] need more opportunities to practice than their peers (McCormick, as cited in Joseph, 2002).

 

    • Build the skills to the point that the student recalls the information with automaticity. Once the student can recall the information with ease and immediately, they will be able to use their brain for other skills, like comprehension (Freeze, 2020)

 

 

Do you agree with these points?  Do you feel that something important was missed?  Please tell me what you think needs to be added.  I welcome your comments!

For more information on teaching struggling readers, I strongly recommend the following two articles:

https://www.readingrockets.org/article/best-practices-planning-interventions-students-reading-problems

https://clarekosnik.files.wordpress.com/2016/04/10_things_to_know_about_research_duke_trtr1002.pdf

 

 

 

 

 

 

References

Allor, J. H., Mathes, P. G., Roberts, J. K., Cheatham, J.P. & Otalia, S. A. (2014).  Is scientifically based reading instruction effective for students with below-average IQs?  Council for Exceptional Children, 80 (3), 287-306.

Duke, N.K. & Martin, N.M. (2011).  10 Things Every Literacy Educator Should Know About Research.  The Reading Teacher, 65, 9-22.

Fisher, D., Frey, N, & Hattie, J. (2016).  Visible learning for literacy:  Implementing the practices that work best to accelerate student learning.  Corwin.

Steve Fleischman, “Before Choosing Ask Three Questions” in Proven Programs in Education: Classroom Management and Assessment, ed.  Robert E. Slavin (Thousand Oaks:  Corwin, 2014), 55-59.

Freeze, R. (2020). Precision Reading: Instructors’ Handbook (3rd Edition). Winnipeg, MB: D. R. Freeze Educational Publications (www.precisionreading.com).

Joseph, L.  (2002).  Best practices in planning interventions for students with reading problems.  Reading Rockets.  Retrieved February 22, 2020 from https://www.readingrockets.org/article/best-practices-planning-interventions-students-reading-problems

Lemons, C.J., Zigmond, N., Kloo, A.M., Hill, D.R., Mrachko, A.A., Paterra, M.F., Bost, T.J., &  Davis, S.M. (2013).  Performance of students with significant cognitive disabilities on early-grade curriculum-based measures of word and passage fluency.  Council for Exceptional Children, 79 (4), 408-426.

Rosen, P.  (n.d.).  The discrepancy model:  What you need to know.  Retrieved February 22, 2020 from  https://www.understood.org/en/school-learning/evaluations/evaluation-basics/the-discrepancy-model-what-you-need-to-know

Stuebing, K. K., Barth, A. E., Molfese, P. J., Weiss, B., & Fletcher, J. M.  (2009).  IQ is not strongly related to response to reading instruction:  a meta-analytic interpretation.  Council for Exceptional Children,  76 (1), 31-51.

Torres, C., Farley, C. A., & Cook, B.G. (2014).  A Special Educator’s Guide to Successfully Implementing Evidence-Based Practices.  Teaching Exceptional Children, 47 (2), 85-93.

 

How to Ensure That the Reading Programs you are Using are Research Proven

How to Ensure That the Reading Programs you are Using are Research Proven

by | Mar 5, 2020 | Understanding Research

I started out this week ready to share a fabulous article with you, excited that I had something very practical and user friendly, for those of you who are teaching reading, just like me.

I found the article in my favourite journal, Teaching Exceptional Children (TEC).  This volume of the journal in particular, was a great find, as it shared the best articles recently published by TEC, combined into one “best of” compilation (TEC, Vol 47, No.2, 2014). The article is entitled “A Special Educator’s Guide to Successfully Implementing Evidence-Based Practices” (EBPs), by Torres, Farley & Cook (2014).

The authors provide a ten-step guide for special education teachers, that is easy to follow and practical to use.  The steps are as follows:

” 10 Step EBP Implementation Process Checklist

  1.  Determine student, environmental and instructor characteristics
  2. Search sources of EBPs
  3. Select an EBP
  4. Identify the essential components of the selected EBP
  5. Implement the practice within a cycle of effective instruction
  6. Monitor implementation fidelity
  7. Progress monitor student outcomes
  8. Adapt the practice if necessary
  9. Make instructional decisions based on progress monitoring data
  10. Become a leader and an advocate (Torres, Farley & Cook, 2014, p. 88)“

Sounds like a great plan, doesn’t it?  With the reading interventions that I use each day at work in mind, I was set to take a closer look at my teaching, to see how it could be improved.  I now had some clear steps as to how to go about this.

As a special educator, I know the value in choosing programs and practices that are supported by research, so I was happy to see the words “Evidence-Based” in the title.  I want to see measurable improvement, as soon as possible, and I want to be sure that I select programs that are  very likely to bring about results.  I do not have time to waste, especially when trying to lessen the gap for students who are reading below grade level in the early- and middle-years.

One thing that I have found confusing in my work, however, is the language that is used to describe reading programs, which make it seem like everything has evidence or research backing it.  Some of the claims I have come across include: backed by research, research-proven, research based, supported by research, empirically proven, science-based, supported by empirical research, research tested, and on and on.  Do these terms sound familiar?   It makes my head spin!  These descriptions appear to all mean the same thing, but really, they do not.

“…It is rare to encounter an education program or practice that does not claim to be ‘research-based’.  This means that the provider claims that research was considered in the design of the program.  However, we must go beyond these claims and, in the spirit of healthy skepticism, demand the compelling evidence that a program works.  This means we must ask whether the program is ‘research proven’ and not just ‘research based’ ” (Fleischman , 2014, p. 57).

Fleischman’s words caused me some discouragement, when I took a closer look the article I mentioned above.  The title, “A Special Educator’s Guide to Successfully Implementing Evidence-Based Practices” specifically states “evidence-based”, not “evidence proven” which now caused a red flag to come up for me.  I was disappointed, since when I had first read it, I did not notice the issue with the terminology.

Regardless, the points expressed, to do with properly implementing an educational program, do appear to be very helpful, and I decided to follow the process with the simple change of subbing in a research proven program.  Easy enough, right?

I was still left with a problem.  How would I know for sure whether a program was research proven?  Is there a quick way to find out?

Unfortunately, it turns out, there really is no quick way.  Even seeing the description “research proven” is not a guarantee that the program can be trusted more than others.  Duke and Martin (2011) provide the following warning:  “The terms research-tested, research-based, research-proven, and others are being used by many to elevate the status of their product or approach. However, use of these terms alone means very little. We have to ask questions that allow us to get underneath any individual’s, organization’s, or company’s use of these terms (p.  17).

There are a series of questions that can be asked, in an effort to look critically at the claims made.  Please see the article “10 Things Every Literacy Educator Should Know About Research (Duke and Martin, 2011) for a detailed description of how to go about analyzing the claims made in research studies and articles (pp. 17-18)

Duke and Martin go on to say, “Notably, we should not assume that something that is research-tested is inherently more supported by research than something that is research-based.  For example, a practice that has been tested and found to be effective in a single study (i.e., research-tested) is, in our view, generally less compelling than a practice that is similar to, but not exactly the same as, a number of practices that have been tested and shown to be effective in a larger number and wide range of studies (i.e., research-based).” (2011, p. 18).

Therefore, it is not simply a question of which terms were used to describe the research or program, and there is no simple way to check whether the results were well-founded or substantiated.  No wonder we sometimes feel like throwing up our hands, when faced with so many options that are supposedly research based.

Over the next few months, I will use the process laid out by Torres, Farley & Cook (2014) to look more closely at the programs that I am currently using with students, to explore this issue further (2014, p. 87). I plan to answer the following questions:

Are the claims of “research-based or “research-proven” for this particular program well substantiated?

How was this program tested? Who was it tested on? Is there a match between the people who participated in the study, and the students I intend to use this intervention with?  Are they of the same age group and do they have other important characteristics in common? (Duke and Martin, 2011, p. 18)

Does the study look at the same, specific goal that I am working on with my students? (Duke and Martin, 2011, p. 18)

Does the study meet the other criteria described by Duke and Martin, with regards to outcome measures / standards / quality / degree of impact, and so on? (pp. 18-19)

I plan to report on my progress with these programs in this blog!  I will follow the steps outlined above, using the implementation checklist and recommendations by Torres, Farley and Cook (2014) to determine my next steps.

I also will strive to improve my success with using these interventions, through collaboration with you!

I would like to invite your feedback, questions an discussions on these topics, below.

 

 

References

Duke, N.K. & Martin, N.M. (2011).  10 Things Every Literacy Educator Should Know About Research.  The Reading Teacher, 65, 9-22.

Steve Fleischman, “Before Choosing Ask Three Questions” in Proven Programs in Education: Classroom Management and Assessment, ed.  Robert E. Slavin (Thousand Oaks:  Corwin, 2014),55-59.

Torres, C., Farley, C. A., & Cook, B.G. (2014).  A Special Educator’s Guide to Successfully Implementing Evidence-Based Practices.  Teaching Exceptional Children, 47 (2), 85-93.

Important Research Terms Defined for New Graduate Students and Interested Teachers

Important Research Terms Defined for New Graduate Students and Interested Teachers

by | Mar 1, 2020 | Understanding Research

I recently began working toward my master’s degree, and have been struggling to make sense of some of the research studies that I have been reading.  Can you relate?

If, like me, you did not take a statistics course in your undergraduate program, and you have not had the privilege of taking a course on interpreting research yet, it is likely that you, too, are having difficulty with some of the vocabulary, as well as the mathematical system, used in the “Results” section of research articles.

Or, perhaps you are you a teacher who is interested in learning more from educational research?  Maybe you would like research to guide your instruction, but are unsure of how to tell whether the research you read is correctly and appropriately recommending a given practice?

In any case, it is essential to have a basic understanding of the vocabulary used in research studies, and the mathematical terms that are used.  Without this, it is hard to determine what the impact of a given approach might be.

In this post, I will share some of the essential ideas that are common to research articles.   I have simplified the definitions, in the interest of clarity, for use by those who are beginners in interpreting research data.

I am hopeful that this post may lead to some discussion and clarification of terms.  If you have questions yourself about these ideas, or if you can provide some illuminating examples, please do so, in the comment section below.

How can you tell whether the intervention or practice that is described in a research study has made a positive difference?  What do you need to know, in order to look at the results critically?  According to Fisher, Frey and Hattie (2016), it is important to ask whether the story, that the research article tells, is convincing (p. 7).  To answer this question, we first need to be able to make sense of the data.

John Hattie, whose mission is to inform teachers of the practices that work best in education, has come to his conclusions through an incredibly large review of educational research (Fisher, Frey & Hattie, 2016).  In fact, his work is based on over 800 meta-analyses, conducted by researchers all over the world, and includes over 50 000 individual studies, and over 250 million students!  His review of the research is considered to be “the most comprehensive review of literature ever conducted” (Fisher, Frey & Hattie, 2016, p. 4)

This brings us to our first term: Meta-Analysis.  A Meta-Analysis is when the findings from many, many studies on the same topic are analyzed, to determine if there are any patterns or trends across the studies. They use this information to change and “inform practice” (Fisher, Frey & Hattie, 2016, p. 5).  If you find a research article that is a meta-analysis, this indicates a broad swath of information has been summarized, and as such, the conclusions arrived at, would be even more reliable than the individual, associated research studies, that were analyzed in the study.

You likely have heard it said that research can prove anything.  Why does it seem so easy for people to claim that their practice is supported by research?  If everything works, why bother to look closely at the research that exists (Shanahan, 2019)?  It turns out that the answer lies in understanding our next research term:  Effect Size.

Effect Size refers to how we quantify the difference that was seen, as a result of the intervention or practice.  For example, in studying a practice to see whether it improves reading abilities in students, researchers measure the students’ performance before the study begins, and then after a number of lessons have been taught.  They then compare the difference between the scores, at the beginning and at the end of the study (Fisher, Frey & Hattie, 2016, p. 7).

Effect size can also compare two different groups of students, one which received the intervention, and one that did not.  The abilities of the students at the end of the intervention, is then compared to see what the difference was, in terms of growth, between the two groups (Fisher, Frey & Hattie, 2016, p. 7).  Researchers may also draw conclusions as to any underlying or apparent reasons for these differences, in addition to the specific intervention.

Effect size is shown as a decimal.  A very clear explanation of effect size can be found in the book Visible Learning for Literacy, by Fisher, Frey and Hattie (2016).  I have simplified the information from this book even further, to assist those who are quite new to reading educational research.

Effect size tells us whether the difference between the students in both groups, or the difference in growth across the study, was large or small.  It is shown numerically, as follows:

  • d= difference
  • d= 0.0  This means that there was no change or growth
  • d= 0.2  This means that there was a small improvement
  • d= 0.4  Medium improvement
  • d= 0.6 Large improvement  (Hattie, as cited in Fisher, Frey and Hattie, 2016, p. 7)

Fisher, Frey and Hattie explain that if the number is above 0.0, the practice is said to have had a Positive Effect  (p. 8).  When research studies tell us that there was a “Positive Effect“, what they are saying is only that the effect was above zero.  However, that does not mean that we should decide to invest money and time in the practice, solely on that fact.   It is necessary to find out how large the effect was .  “It turns out that 95%+ of the influences that we use in schools have a positive effect; that is, the effect size of nearly everything we do is above zero…If you set the bar at showing any growth above zero, it is indeed hard to find programs and practices that don’t work” (Fisher, Frey and Hattie, 2016, p. 9).

Alright then, the next question to ask is, “What is an acceptable effect size?”

John Hattie sets the “bar of acceptability” at 0.4, and calls this the “hinge point” ((Fisher, Frey and Hattie, 2016, pp. 8-9).  This is the number we need to look for when reading research.  A number of 0.4, or higher, is good.  A number of 1.0 is a very, very good.  In fact, an effect size of 1.0 would indicate a very noticeable improvement, or an advancement that is “large, blatantly obvious and grossly perceptible” (Cohen as cited in Fisher, Frey and Hattie, 2016, p. 7).

As I’ve said above, the definitions I am providing here are very simplified.  There is much more complexity involved, that is outside the scope of this blog post, and at this point in time, beyond my own familiarity and conception.  To give us an idea of the complexity involved, Fisher, Frey and Hattie (2016) point out that one must look closely at a particular situation, when interpreting the effect size (p.  7).  If a practice has a lower effect size (0.2 or 0.3), there are times when it might still be a good idea to try it with students.  Factors that one might consider when determining whether or not to try a practice that has low effect, would be the financial cost of such a practice, and the difficulty involved in its use.  If the cost of using a specific practice is so low, and the act of putting it into place with students would be especially easy and quick to do, it might make good sense to try it.

When reading about effect size in Visible Learning for Literacy,  (Fisher, Frey and Hattie, 2016), I was happy to have some clarification on a point that I brought up in an earlier post.  In “A Sobering Reality” I had mentioned that a previous professor of mine, Dr. Freeze, had explained, that when determining whether or not to continue an intervention, we must check to see how much growth has occurred over the course of a year (Busch, February 24th, 2020).  If the students involved have made less than a year’s progress over the course of a year, it is good practice to end the intervention, and to try something else (Rick Freeze, personal communication to author, February 23, 2020).

Why is this mention of “one year’s growth” so important, and where does it come from?  Fisher, Frey and Hattie (2016) explain that “students naturally mature and develop over the course of a year, and thus actions, activities, and interventions that teachers use should extend learning beyond what a student can achieve by simply attending school for a year” (italics in original, p. 8).  What we need to look for when determining next steps, is whether the results show improvement above the effect of the natural growth that occurs.

After looking at the effect size, the next step is to determine whether the results were Statistically Significant.  Effect size and the size of the study (or how many participants there were) are combined to determine whether the practice would have a strong impact on student learning.  Hattie and colleagues (2016) even provide a formula to determine this:  “Significance = Effect size x Study size” (p. 7).  If you are even a little familiar with research studies, you have likely heard that the larger the study (that is, the higher the number of participants in the study), the more it can be trusted.  Alternatively, however, it is possible for a study to have a very large number of participants, but a tiny effect size.  Therefore, these two factors must be considered together (p. 7).

For more information on the nuances or finer points of the terms defined here, I recommend Chapter 1, in Visible Learning for Literacy:  Implementing Practices That Work Best to Accelerate Student Learning (Fisher, Frey and Hattie, 2016)

I would like to invite you to please share any clarifications of the terms I have introduced in this post, in the comments section below.  I would especially like to invite you to contribute examples that explain the interplay of effect size and study size on statistical significance, if you happen to be well versed in educational research, yourself.

Also, I invite you to comment below if you find that I have misinterpreted or mis-communicated the concepts described above, keeping in mind that the goal here is to present as clear and simple an explanation as possible.  We all become stronger through collaboration and discussion, and I thank you in advance for your ideas!

 

References

Fisher, D., Frey, N, & Hattie, J. (2016).  Visible learning for literacy:  Implementing the practices that work best to accelerate student learning.  Corwin.

Shanahan, T.  (2019).  I’m a Terrific Reading Teacher, Why Should I Follow the Research?  Reading Rockets. Retrieved March 1, 2020 from https://www.readingrockets.org/blogs/shanahan-literacy/im-terrific-reading-teacher-why-should-i-follow-research