After some discussion here on Forbes about the validity of “influence” in social media, news that Twitter has a significant impact on scientific citations is something of a surprise. But could it be?
That debate has been ongoing in the science community through January. The possibility hit a wider public when Alexi Madrigal raised it in The Atlantic a couple of days ago.
The bottom line is simple: articles that many people tweeted about were 11 times more likely to be highly cited than those who few people tweeted about. Its implications are even more interesting. It generally takes months and years for papers to be cited by other scientific publications. Thus, on the day an article comes out, it would seem to be difficult to tell whether it will have a real impact on a given field. However, because the majority of tweets about journal articles occur within the first two days of publication, we now have an early signal about which research is likely to be significant.
The relationship is not even marginal. 11x more likely is a huge influence. The research appeared in the Journal of Medical Internet Research, and was conducted by its editor Gunther Eysenbach.
However the data had already been challenged before The Atlantic article, both by a rebuttal – and by a suggestion that the research author had a vested interest in the results:
What’s more, many of these counted tweets were not sent out by humans,” says Phil Davis at The Scholarly Kitchen after reviewing the data. “The Journal of Medical Internet Research sends out an automatic tweet when a paper first appears and then sends out monthly tweets to promote the journal’s most tweeted papers. Tweets promoting the journal’s most viewed, most purchased, and most cited articles (from Scopus and Google Scholar) are also sent out automatically, many of which are then retweeted by other tweet bots (and human bots) to the blogosphere.”
Davis also points out that the work was not peer reviewed and that the journal editor had registered a number of domains (twimpact.org, twimpactfactor.org and twimpactfactor.com) that suggested he could make commercial advantage of this research, though Eysenbach had fully disclosed this possibility.
Eysenbach then corrected the paper: but stood by the findings, claiming the correction had no impact on the study findings.
By this time the prestigious BMJ Group, publishers of the British Medical Journal, had taken it up – but in its blogs section, concluding:
Whether or not you agree with the validity of Eysenbach’s study, the very fact that it has been published and discussed so widely is surely a testament to the increasing importance of social metrics in evaluating article impact.
The BMJ Group was interested because the Eysenbach paper had caused a stir in the Altmetrics community, a project set up to discuss the post-peer review environment.
Peer-review has served scholarship well, but is beginning to show its age. It is slow, encourages conventionality, and fails to hold reviewers accountable. Moreover, given that most papers are eventually published somewhere, peer-review fails to limit the volume of research.
As scholars migrate their publication to the web, and publish earlier, the web offers a better way to filter science or as Altmetrics puts it: “Instead of waiting months for two opinions, an article’s impact might be assessed by thousands of conversations and bookmarks in a week.”
# 1. We are creating knowledge in new ways but have a philosophy of science modeled on a pre-web way of working; we still tend to think of science and any rigorous thinking as an object that we collectively cultivate and grow. I wonder if this is a useful analogy any longer.
# 2. Eysenbach’s research may be a useful early indicator of how social is changing science publishing but also a lesson for the wider community of opinion formers that opinion forming is itself changing and we need to understand its more fluid nature
# 3. What we know will change. For decades it has mattered where you publish and peer review has been a brake on some innovative perspectives. It has tended to defend established viewpoints. The possibility is that new interpretations of experience can evolve and evolve rapidly. It needs a new philosophy of knowledge.
Follow me on Twitter @haydn1701