Last Friday Gunther Eysenbach published a very interesting paper in the Journal of Medical Internet Research (JMIR):
Can Tweets Predict Citations? Metrics of Social Impact Based on Twitter and Correlation with Traditional Metrics of Scientific Impact
Gunther Eysenbach analyzed a total of 4208 tweets citing 286 distinct JMIR articles. 60% of the tweets were sent the day the paper was published, or the day after:
There was a correlation between the number of tweets about a JMIR paper, and the number of citations in Google Scholar or Scopus (analyzed 17-29 months later). Highly tweeted papers were more likely to become highly cited, but the numbers were to small for any firm conclusions (12 out of 286 papers were highly tweeted).
This is a great study because it shows empirically what many of us felt already: Twitter is one of the fastest tools to discover newly published scholarly papers, and the number of tweets is an important measure of scholarly impact. This is an important paper for the altmetrics movement, even though Gunther Eysenbach in the paper says that he doesn’t like the term. In the paper he coins the term tweetations for tweets citing a paper – I personally prefer the term citation for all content linking to and discussing a scholarly work. Not surprisingly the paper has been tweeted more than 250 times in the first few days after publication, and will certainly become highly cited.
Are altmetrics only about counting citations?
A few weeks ago Euan Adie from altmetric.com suggested to me that we do a little research project for the altmetrics session we moderate together at the ScienceOnline2012 conference in January. We both felt that altmetrics could be improved by taking a closer look at the content of social media discussing scholarly works, and that using the Citation Typing Ontology (CiTO) would be a good start. CiTO is an ontology for the characterization of citations, and we agreed that crowdsourcing would be the only way we could do this with a large enough number of social media contributions, and with only a few weeks left before ScienceOnline2012. The project launched today at http://crowdometer.org.
To get this project started in time we had to make two other simplifications: a) we use only a subset of 7 CiTO relationships (see below, cito:usesConclusionsFrom is new in CiTO 2.1, and was added for us by David Shotton), and b) we focus on short content (Twitter, Facebook Walls, Google+), as it would be much more difficult to crowdsource the semantic meaning of a blog post and similar longer content.
The social media data for this project were all collected by Euan Adie at his altmetric.com site, and consist of 500 tweets, Facebook Wall posts and Google+ comments about scholarly papers from the month of October. In the end we used only tweets, but they made up more than 90% of the posts anyway.
Users can classify these tweets in two ways, either by browsing or searching the list of nearly 500 tweets, or by getting presented random tweets (hint: search for “I’m feeling lucky” in CrowdoMeter). You have to register via your Twitter account to classify tweets.
CrowdoMeter is doing a basic analysis of the crowdsourcing effort in real-time, allowing everyone to follow along:
We are still learning how to attract as many users as possible to help with this effort, but a long list of tweets about hopefully interesting papers, an uncluttered user interface, the real-time analysis and a high score list seem to be good start. We hope to have enough classifications together when we discuss this and other altmetrics projects at ScienceOnline2012 next month.
References
- Eysenbach, G. (2011). Can Tweets Predict Citations? Metrics of Social Impact Based on Twitter and Correlation with Traditional Metrics of Scientific Impact. Journal of Medical Internet Research, 13(4), e123. https://doi.org/10.2196/jmir.2012
