Not unlike most people, I feel pretty strongly about certain things that are important to me or my life. I'm also pretty active online, mostly through Twitter. A few years ago, I wouldn't have thought twice about clicking the un-follow button if someone said something I found offensive or vehemently disagreed with.
Even now, the urge still strikes when I read something that makes me turn up my nose, but I'm making a conscious effort to recognize just how easy it is to surround yourself with information and people who support your beliefs...creating a comfortable little bubble with no nudges to think critically about novel points of view.
One definition of confirmation bias is the tendency to search for or interpret information in a way that confirms one's preconceptions, leading to statistical errors. While this phenomenon might be more prevalent on a
social or personal level, I'm not often statistically analyzing my Twitter activity, so I'm pretty sure that this refers to it occurring in academic research.
It will be important for me to remember to search for alternative and potentially disproving information as I move through this masters program towards the final research paper. I believe that researchers who acknowledge information that challenges their hypothesis and results is more valuable, especially considering the vast amount of research available in the literature.
I think I would consider avoiding confirmation bias an imperative skill when it comes to critical thinking. What do you think?
If you're interested in reading more about confirmation bias in academic research, you might find the following links useful.
Publication Predjudices: An Experimental Study of Confirmatory Bias in the Peer Review System
Confirmation Bias and the Watson Rule Discovery Test
Confirmation bias in science: how to avoid it
Sunday, August 24, 2014
Friday, August 22, 2014
Let's talk about replication
On Thursday evening, I came across an online article on the Washington Post that talked about one of the things I took issue with when I wrote a critique of an article that I handed in is this past weekend. If only I had seen it before now! It would have helped my argument immensely. But such is life.
The article (with it's Upworthy-esque title "A shocking statistic about the quality of education research") reports on a newly published academic article that looked into how little research replication is being done in the education sciences.
Complaints of click bait aside, the statistic actually is shocking.
Of those 0.13% that are replicated? Only 68% are successfully replicated.
And if you take a look at who is doing the replicating, only 54% of the 0.13% are replicated successfully without the original author on the research team.
That's not very good. 54% of 0.13% is....a really small percentage.
Replication is the norm in the "natural" sciences (as stated by education Professor Robert H. Bauernfeind in a magazine article published in 1968, available here if you have access to JSTOR through your library), and as an undergraduate student in zoology, I was constantly reminded that research methods needed to be clear and concise, because if someone couldn't repeat your experiment, well...it wasn't worth all that much.
Of course, replication in behavioural studies is more difficult than say, attempting to reaffirm the basal metabolic rate of Carcinus maenas. Unless of course your experiment goes wrong and your specimens end up eating each other and your results are consistent with the values of recently fed green crabs.
Bad memories. Anyway...
Replicating research in the education sciences can prove to be challenging because even the elements of research that should remain constant from study to study, may be difficult to keep that way. Teachers get new ideas for mixing up their instruction, Learning Management Systems get updated, section sizes in courses are constantly being inflated, not to mention that the demographics in the classroom from semester to semester, let alone year to year, can constantly fluctuate.
But still, shouldn't an effort be made? Wouldn't several studies or trials of your experiment confirming your results build a better case for your hypothesis? I imagine it would be especially beneficial to those in a position to stimulate change at their institution.
What do you think? Is replication important to education research or is the need for novel information too great?
The full report, titled Facts are more important than novelty: Replication in the education sciences, hasn't been published in print yet, but you can access it for free here.
The article (with it's Upworthy-esque title "A shocking statistic about the quality of education research") reports on a newly published academic article that looked into how little research replication is being done in the education sciences.
Complaints of click bait aside, the statistic actually is shocking.
Only 0.13% of education research experiments are replicated.
Of those 0.13% that are replicated? Only 68% are successfully replicated.
And if you take a look at who is doing the replicating, only 54% of the 0.13% are replicated successfully without the original author on the research team.
That's not very good. 54% of 0.13% is....a really small percentage.
Replication is the norm in the "natural" sciences (as stated by education Professor Robert H. Bauernfeind in a magazine article published in 1968, available here if you have access to JSTOR through your library), and as an undergraduate student in zoology, I was constantly reminded that research methods needed to be clear and concise, because if someone couldn't repeat your experiment, well...it wasn't worth all that much.
Of course, replication in behavioural studies is more difficult than say, attempting to reaffirm the basal metabolic rate of Carcinus maenas. Unless of course your experiment goes wrong and your specimens end up eating each other and your results are consistent with the values of recently fed green crabs.
Bad memories. Anyway...
Replicating research in the education sciences can prove to be challenging because even the elements of research that should remain constant from study to study, may be difficult to keep that way. Teachers get new ideas for mixing up their instruction, Learning Management Systems get updated, section sizes in courses are constantly being inflated, not to mention that the demographics in the classroom from semester to semester, let alone year to year, can constantly fluctuate.
But still, shouldn't an effort be made? Wouldn't several studies or trials of your experiment confirming your results build a better case for your hypothesis? I imagine it would be especially beneficial to those in a position to stimulate change at their institution.
What do you think? Is replication important to education research or is the need for novel information too great?
The full report, titled Facts are more important than novelty: Replication in the education sciences, hasn't been published in print yet, but you can access it for free here.
Thursday, August 21, 2014
Are we all just faking it?
This cheeky little comic popped up in my Facebook news feed the other day. I'm sure some will find it condescending, but I'm willing to bet there's a little truth in there...
Considering Open Access
Information is everywhere online. Anyone who desires to can create a website and fill it with words, and whether those words are accurate or not, well...
And if there's a catchy title? Misinformation can spread like wildfire.
Considering how information is so easily accessible, it's strange to me that a significant amount of academic work is hiding behind pay walls on the Internet. It may never be seen by the masses who just might need it most, despite organizations like The United Nations Educational, Scientific, and Cultural Organization supporting open access to scientific information.
There are some exceptions, of course. academia.edu is a self-described place to share and follow research, but it comes with it's share of ethical issues. While authors may have done the research and written the papers, the copyright belongs to the journal publishers. Menachem Wecker wrote a thought provoking piece on the subject over on Vitae.
In the United States, citizens have petitioned for open access to tax-payer funded research, and succeeded in having the federal government respond by issuing a memorandum to Federal agencies that receive a significant amount of funding from the government, which requires them to publish their results free of charge.
Not irrelevant to the conversation about open access is the more recent discussion about clinical research trial publishing practices. The AllTrials Campaign seeks to have all results from clinical research made available, not just those results that make it to publication. Scientific publishing giant Elsevier is supporting this initiative. Elsevier plays a role in supporting open access as well.
But what good are open source academic publications if the content is so formal and context specific that the public in general can't make much sense of it?
Some pioneering science loving individuals are using social media to their advantage to share science with the world. Instead of clogging up Facebook news feeds with more pictures of babies and cats, they're firing out news articles about new scientific discoveries, up and coming innovators, and a whole lot of awesome. Below are a few of my favourites...
I *bleeping* love science
***heads up - "bleeping" isn't the actual word used in the name of this group!
Science is a Verb
Not an Exact Science Show
It would be interesting to know how often and to what extent non-academics would make use of academic articles if access was easier. Could it lead to the creation of a more knowledgeable population? Or is it likely to be ignored in favour of articles of videos that are quick to read and move on from?
What are your thoughts on open access to scientific information? Do you have a favourite Facebook group or Twitter handle that you follow?
And if there's a catchy title? Misinformation can spread like wildfire.
Considering how information is so easily accessible, it's strange to me that a significant amount of academic work is hiding behind pay walls on the Internet. It may never be seen by the masses who just might need it most, despite organizations like The United Nations Educational, Scientific, and Cultural Organization supporting open access to scientific information.
There are some exceptions, of course. academia.edu is a self-described place to share and follow research, but it comes with it's share of ethical issues. While authors may have done the research and written the papers, the copyright belongs to the journal publishers. Menachem Wecker wrote a thought provoking piece on the subject over on Vitae.
In the United States, citizens have petitioned for open access to tax-payer funded research, and succeeded in having the federal government respond by issuing a memorandum to Federal agencies that receive a significant amount of funding from the government, which requires them to publish their results free of charge.
Not irrelevant to the conversation about open access is the more recent discussion about clinical research trial publishing practices. The AllTrials Campaign seeks to have all results from clinical research made available, not just those results that make it to publication. Scientific publishing giant Elsevier is supporting this initiative. Elsevier plays a role in supporting open access as well.
But what good are open source academic publications if the content is so formal and context specific that the public in general can't make much sense of it?
Some pioneering science loving individuals are using social media to their advantage to share science with the world. Instead of clogging up Facebook news feeds with more pictures of babies and cats, they're firing out news articles about new scientific discoveries, up and coming innovators, and a whole lot of awesome. Below are a few of my favourites...
I *bleeping* love science
***heads up - "bleeping" isn't the actual word used in the name of this group!
Science is a Verb
Not an Exact Science Show
It would be interesting to know how often and to what extent non-academics would make use of academic articles if access was easier. Could it lead to the creation of a more knowledgeable population? Or is it likely to be ignored in favour of articles of videos that are quick to read and move on from?
What are your thoughts on open access to scientific information? Do you have a favourite Facebook group or Twitter handle that you follow?
Tuesday, August 19, 2014
Whoops.
My current masters course and I aren't really friends. I don't get it, it doesn't get me. I haven't been giving it the full attention that it needs in order for me to do well, and now I have four days to jam out a handful of 400-600 word blog posts. Procrastination always seems like a good choice until that deadline comes looming and I realize how hard it is to actually write that many words about something I don't really care about.
Tuesday, July 8, 2014
Hey there, hi there.
Blogging isn't new to me...I've run a personal blog for the past few years, although finding time to update it lately is proving to be difficult. That said, having a professional, academically focused blog is completely foreign...so we'll see how this goes. At least here I'm not going to bother worrying about alt text and SEO and no follow links and Pinterest and and and and....
Subscribe to:
Posts (Atom)