Not unlike most people, I feel pretty strongly about certain things that are important to me or my life. I'm also pretty active online, mostly through Twitter. A few years ago, I wouldn't have thought twice about clicking the un-follow button if someone said something I found offensive or vehemently disagreed with.
Even now, the urge still strikes when I read something that makes me turn up my nose, but I'm making a conscious effort to recognize just how easy it is to surround yourself with information and people who support your beliefs...creating a comfortable little bubble with no nudges to think critically about novel points of view.
One definition of confirmation bias is the tendency to search for or interpret information in a way that confirms one's preconceptions, leading to statistical errors. While this phenomenon might be more prevalent on a
social or personal level, I'm not often statistically analyzing my Twitter activity, so I'm pretty sure that this refers to it occurring in academic research.
It will be important for me to remember to search for alternative and potentially disproving information as I move through this masters program towards the final research paper. I believe that researchers who acknowledge information that challenges their hypothesis and results is more valuable, especially considering the vast amount of research available in the literature.
I think I would consider avoiding confirmation bias an imperative skill when it comes to critical thinking. What do you think?
If you're interested in reading more about confirmation bias in academic research, you might find the following links useful.
Publication Predjudices: An Experimental Study of Confirmatory Bias in the Peer Review System
Confirmation Bias and the Watson Rule Discovery Test
Confirmation bias in science: how to avoid it
Showing posts with label Research. Show all posts
Showing posts with label Research. Show all posts
Sunday, August 24, 2014
Friday, August 22, 2014
Let's talk about replication
On Thursday evening, I came across an online article on the Washington Post that talked about one of the things I took issue with when I wrote a critique of an article that I handed in is this past weekend. If only I had seen it before now! It would have helped my argument immensely. But such is life.
The article (with it's Upworthy-esque title "A shocking statistic about the quality of education research") reports on a newly published academic article that looked into how little research replication is being done in the education sciences.
Complaints of click bait aside, the statistic actually is shocking.
Of those 0.13% that are replicated? Only 68% are successfully replicated.
And if you take a look at who is doing the replicating, only 54% of the 0.13% are replicated successfully without the original author on the research team.
That's not very good. 54% of 0.13% is....a really small percentage.
Replication is the norm in the "natural" sciences (as stated by education Professor Robert H. Bauernfeind in a magazine article published in 1968, available here if you have access to JSTOR through your library), and as an undergraduate student in zoology, I was constantly reminded that research methods needed to be clear and concise, because if someone couldn't repeat your experiment, well...it wasn't worth all that much.
Of course, replication in behavioural studies is more difficult than say, attempting to reaffirm the basal metabolic rate of Carcinus maenas. Unless of course your experiment goes wrong and your specimens end up eating each other and your results are consistent with the values of recently fed green crabs.
Bad memories. Anyway...
Replicating research in the education sciences can prove to be challenging because even the elements of research that should remain constant from study to study, may be difficult to keep that way. Teachers get new ideas for mixing up their instruction, Learning Management Systems get updated, section sizes in courses are constantly being inflated, not to mention that the demographics in the classroom from semester to semester, let alone year to year, can constantly fluctuate.
But still, shouldn't an effort be made? Wouldn't several studies or trials of your experiment confirming your results build a better case for your hypothesis? I imagine it would be especially beneficial to those in a position to stimulate change at their institution.
What do you think? Is replication important to education research or is the need for novel information too great?
The full report, titled Facts are more important than novelty: Replication in the education sciences, hasn't been published in print yet, but you can access it for free here.
The article (with it's Upworthy-esque title "A shocking statistic about the quality of education research") reports on a newly published academic article that looked into how little research replication is being done in the education sciences.
Complaints of click bait aside, the statistic actually is shocking.
Only 0.13% of education research experiments are replicated.
Of those 0.13% that are replicated? Only 68% are successfully replicated.
And if you take a look at who is doing the replicating, only 54% of the 0.13% are replicated successfully without the original author on the research team.
That's not very good. 54% of 0.13% is....a really small percentage.
Replication is the norm in the "natural" sciences (as stated by education Professor Robert H. Bauernfeind in a magazine article published in 1968, available here if you have access to JSTOR through your library), and as an undergraduate student in zoology, I was constantly reminded that research methods needed to be clear and concise, because if someone couldn't repeat your experiment, well...it wasn't worth all that much.
Of course, replication in behavioural studies is more difficult than say, attempting to reaffirm the basal metabolic rate of Carcinus maenas. Unless of course your experiment goes wrong and your specimens end up eating each other and your results are consistent with the values of recently fed green crabs.
Bad memories. Anyway...
Replicating research in the education sciences can prove to be challenging because even the elements of research that should remain constant from study to study, may be difficult to keep that way. Teachers get new ideas for mixing up their instruction, Learning Management Systems get updated, section sizes in courses are constantly being inflated, not to mention that the demographics in the classroom from semester to semester, let alone year to year, can constantly fluctuate.
But still, shouldn't an effort be made? Wouldn't several studies or trials of your experiment confirming your results build a better case for your hypothesis? I imagine it would be especially beneficial to those in a position to stimulate change at their institution.
What do you think? Is replication important to education research or is the need for novel information too great?
The full report, titled Facts are more important than novelty: Replication in the education sciences, hasn't been published in print yet, but you can access it for free here.
Subscribe to:
Posts (Atom)