Oscillatory Thoughts: The role of Facebook and Twitter in scientific citations and impact factors

Source: http://blog.ketyov.com/2011/07/role-of-facebook-and-twitter-in.html

The role of Facebook and Twitter in scientific citations and impact factors 

This past weekend I was messing around on the internets (when I should have been relaxing by a river) when I was struck by a question: is there a way to see what impact social media has had on science?

Turns out, social media such as Facebook and Twitter may have a big influence on journal impact factors and paper citations... but more on that in a second. (For a primer on impact factor and citations, read my old post: "Something ghoti with science citations".)




First: my rambling thoughts.

Many of us scientists have this perception that "social media" is "important" for science, but I don't yet have a grasp on what that means, and I don't think anyone else does, either.

In fact, just yesterday over on Wired, @Sheril_ asked a bunch of science twitterers why they use Twitter. The responses are all over the place! People use it for self promotion, for sharing and learning new ideas, for staying in touch, professional networking, etc.

I've talked about the role of various social networking/Web2.0 sites in the scientific process ad nauseam. I've espoused my interest in Quora in my post "Quora for Scientists":

...with Quora, anyone can connect with potential experts. The question is how can Quora engage these experts? Why would anyone contribute their expert knowledge for free? I mean, I do it because... well I really enjoy talking with the public about science. Sometimes it's the really simple questions (e.g., What is the neurological basis of curiosity?) that really make me stop and think about "easy" questions I normally wouldn't worry about. That's exciting to me. I like to solve hard problems (especially ones that seem easy), and I think most scientists feel the same way.

Again, that's just another opinion. Another rationale added to many others.

But what do the data suggest?

I turned to the ISI Web of Knowledge, which provides excellent journal-specific metrics such as the total number of citations a journal has received, the journal's impact factor, and so on. (Sadly these metrics are closed to the general public; you need a license to view them.)

For each journal I got its 2010 impact factor as well as the total number of citations the journal has received. The former measures the average number of times a paper published in that journal was cited between 2008 and 2010. The latter is more of a "popularity" measure weighted by the number of publications in the journal.

I wanted to see what impact social media has on these two common metrics. To do that I looked to Facebook and Twitter to find which journals have their own Facebook pages and Twitter accounts. In the end I was left with 17 of the "top" peer-review general science and medical/biomedical publications that had both a Facebook page and a Twitter feed:

Nature, PNAS, Science, The New England Journal of Medicine, Cell, The Lancet, Journal of the American Medical Association, British Medical Journal, PLoS Biology, PLoS Medicine, Genes & Development, Nature Medicine, Genome Research, The Journal of Experimental Biology, and Cell: Stem Cell

For each of these journals I then noted the number of "likes" on its Facebook page, which is a measure of how many people will see an announcement made by that journal. I also took note of how many Twitter followers each journal has, as well as how many people the journal follows and how many tweets per day the account makes, on average.

First, some simple metrics:
* Nature has received the most total cites with 511145
* The New England Journal of Medicine had the highest 2008-2010 impact factor at 53.5
* They have--by far--the most Facebook likes with 232385
* They also have the most Twitter followers with 31863
* The British Medical Journal has made the most tweets with 5405
* They follow the most twitterers with 1471
* And they are the most chatty with 5.4 tweets per day

First-pass metrics show that there is a strong correlation between number of tweets and number of followers (r = 0.63, p = 0.012) as well as the number of followers and tweets per day (r = 0.54, p = 0.037).

Tweet more, you get more followers.


(It's important to note that all metrics are on a log scale!)

Interestingly, total citations and impact factor are only weakly correlated (r = 0.49, p = 0.063).

The question is, do social media metrics explain any of the variance in citations or impact factor?

Again, looking at simple correlations, we see that number of Facebook page likes for a journal and the number of twitter followers it has are correlated (r = 0.53, p = 0.044). This probably represents measure of social networking engagement.

But what's crazy is that number of Facebook page likes is strongly correlated with the total number of citations a journal has received (r = 0.78, p = 0.001)!


Both Facebook page likes and number of Twitter followers correlate (equally well!) with impact factor (r = 0.59, p = 0.021; r = 0.59, p = 0.021 respectively).

For those of you who like a little more scientific rigor than a bunch of uncorrected correlations, I also ran a multiple (log-transformed) linear regression for total citations and impact factor separately. What's interesting is that, in both cases, number of Facebook page likes seems to be the dominant significant factor in determining citations and impact factor.

However, for the impact factor analysis, both the number of people the journal follows on twitter, as well as the number of tweets per day, are inversely correlated with impact factor (partial correlations of r = -0.44 and r = -0.66, respectively)!

So what does all of this mean?

Well, it seems like having a social media presence is probably a reflection of the journal's popularity in general, but that, on average, the journals that do the most social media engagement (amongst the top-tier journals!) show the lowest impact factors.

That's a little disheartening.
Oscillatory Thoughts: The role of Facebook and Twitter in scientific citations and impact factors

Tweet