Publish or Perish

I know its been a while since I've written, but I decided that I shouldn't keep complaining about something without taking action. Venting is only useful if it inspires action, and while I'm still not complacent or content, I'm being productive and am more centered. Perhaps it's not amazing that my inspiration dried up with my inability to bitch.

Today's inspiration came from something I read on the Scholarly Kitchen. This article, and the study it cites, discusses the role of citation (and publication) in science. It suggests that the current Publish or Perish environment not only discourages innovation, which I've discussed previously, but also results in false support/reproduction of inaccurate (at best) hypotheses.

What exactly does this mean? As a scientist at a research university (and despite my hiatus into editing, I still remember how it works) , there is enormous pressure to publish in as 'High Impact' Journal as possible. Where did this pressure come from, and what does it mean?

Historically, scientific publication began as handwritten letters sent to other scientists and to scientific societies describing new findings. These letters would be read aloud, or printed and distributed to other scientists in the field, who would then either applaud or deride the author, depending on whether the new data supported their current working hypothesis or not. Publication was not only a way to freely disseminate scientific information (as the current open source idealists would have it), but a way to build a reputation. And a reputation could be built as much on your response to an original letter as on the letter itself.

Jump ahead to present day. Science is expensive. Gone is the day when you can drop a bowling ball out of a tower and measure the acceleration due to gravity with a stopwatch, where the lone wealthy scientist can work in a lab in his/her attic. High tech immunology labs require hundreds of thousands of dollars a year just to remain in operation. In order to afford expensive equipment, such as a half a million dollar multi-photon/confocal microscopes, individual scientists need to pool resources. Thus the academic research institution. (I won't talk about industrial research - that's a whole different can of worms.)

What are the driving forces of current academic researchers? 1) Funding. 2) Space. 3) Tenure. The first two at least (as is the third, but in a different way) are limited resources, and there are incredible levels of competition to achieve them. Tenure committees, university administrators, and funding agencies needed 'objective' criteria by which to measure scientific success to distribute these resources. And they somewhat randomly chose publication (and later citation).

On first blush, publication/citation seems a valid means of judging scientific success. The sheer quantity of the papers that you publish can be used to measure your scientific output, and the number of people who cite your paper indicates the value of your work. Most tenure committees and funding agencies use some combination of these values to determine success (as well as the more ambiguous "scientific reputation" metric).

However, judging a scientist on publication/citation has had some unwanted side effects:
1) Scientists, especially early career scientists, are more likely to take on "sure thing" projects so that they can produce their requisite publications, vastly diluting the quality of production of, arguably, their most creative years.
2) Scientists are loathe to publish data that contradicts currently accepted "dogma", as it not only tarnishes their scientific reputation but also results in publications (if the data get published at all) with few citations in Low Impact Journals. After all, most scientists only support work that supports their own; they are encouraged to focus more on telling a coherent 'story' than on an objective evaluation of all of the available data.
3) Negative data (experiments that didn't work) don't get published, resulting in repetition by multiple groups and a waste of resources.
Taken together, these side effects result in incremental scientific advances that inch forward, rather than shift or even malign, the current paradigm.

I realize that much of my argument relies on my firm belief that while individuals have free will, populations of individuals do not. A loan scientist may throw off the system, take great risks for a different motivation (curing cancer, winning the Nobel?); however, the group that is scientists are currently rewarded for publication, not breakthroughs in scientific knowledge. And the group will follow the status quo and try to be successful in the current reward structure. I certainly did.

Describing a problem is only the beginning, however. Finding a solution is more difficult. It requires out-of-the-box thinking from visionary individuals who have an unwavering belief in their solution and the courage and committment to institute change. Such individuals are rare (note the pathetic Congressional attempt to modify our current dilapidated health care system), and if there's one thing I've learned over the last year, I am not that visionary. I see the need for change but lack that critical spark, the untouchable self-confidence, that I've seen in others that drive change.

I do have some suggestions, however, that maybe when vetted may form a basis for an action plan.

1) The current funding system should be restructured.
(A) Promising young career scientists (based on graduate and postdoctoral work as well as innovative grant writing) should be given special money and rewards to focus at least a part of their time on potential high impact research - the more daring the idea, the more likely to be funded (ie - the current "young innovator" grants should be expanded to include a broader population).
(B) Innovative senior scientists and scientists that would like to switch fields should be offered incredibly competitive but high reward (ie high monetary value) funding. Again, this is currently done, but should occur on a much grander scale, providing at least 30% off all available funding.
(C) Incentives should be changed from publication to results. I guess I'm envisioning something akin to the X prize - a high level of funding dollars in response to achieving specific goals, something to motivate not only innovation, but direct competition - a biomedical sciences space race of sorts. Major goals, some "attainable" and others "unattainable", could be decided by a multi-disciplinary committee of scientists, policy advocates, and community members.

2) The current publication paradigm should be reconfigured.

I agree with the open choice idealists; government funded work should be freely available to the public (not a popular view in the publishing industry, let me tell you). I believe, however, that publishers do provide a strong value to manuscripts and to readers. Peer review is critical, but the most important service that different Impact Factor level journals provide is quality screening. I know that when I read a paper in Science or Nature, it will be of strictly higher quality (and potentially higher impact) than if I read a paper in a specialized society Journal, at least the majority of the time. Considering the high number of articles that come out every day in a given field, scientists need an external vetting process to differentiate the articles they read as compared to the articles they scan. And given that I can't think of another way to objectively evaluate science external from peer review (money and marketability of course being the choice for industrial science), that is exactly what I'm suggesting.

I would like to see the paper publication process be streamlined into one workflow per field. Rather than submitted to Science, being rejected, then submitting to a more specialized Journal, etc, etc etc and modifying the paper at every subsequent stage until it settles into its proper "impact density", I think that all manuscripts should go through the same peer review process. Reviewers could suggest the appropriate rank/level for the publication. (Note in this view, print Journals are a thing of the past, as they should be currently). The most important thing about this system, however, is that a manuscript's rank is not static. In a wiki/utube-like manner, articles may rise or fall down the ranks by additional rankings from other scientists, who are the most able to evaluate the potential impact of a body of work. In addition, each article can have an associated comment stream, moderated to ensure comments discuss the scientific value (or lack there of) of the current work. These articles should be freely available to the public (potentially through governmental funding for the service or maintenance by a not-for-profit). Production costs (formatting, editing, peer review and comment mediation, site maintenance and archiving) could be paid by the authors (which is currently the case anyway) or other currently-being-explored revenue streams.

If you believe in the equilibrium of a wiki, over time and with sufficient rankings, articles in this system will be a more accurate measure of scientific success/impact, as judged by the entire scientific community and not just 2 peer reviewers and an editor. Indeed, a less important paper could rise in the ranks over time as prevailing views shift, whereas an immediately exciting but ultimately unimportant paper could fade away in this dynamic system. True innovators, both short- and long-term, will be rewarded, and incremental papers will be discouraged. Plagarism and figure manipulation will be strongly selected against in such an interactive scientific community. Work quality and thoroughness will improve - no one wants to be dissed openly for sloppy work.

Concluding Thoughts
As I mentioned previously, while its easy to wax poetic about dreams of an idealized scientific world, implementation is not quite as simple. The building blocks are already in place for the funding changes I've outlined; however, shifting money from traditional funding avenues (RO1 grants, etc) to encourage innovation will certainly hurt many labs. I think the long-term benefits outweigh the short-term costs, but I don't have a lab that's going to close because of lack of funding.

The changes to the publishing industry that I've outlined are more revolutionary than evolutionary. As with any revolution, heads are lost. As a publisher/editor at an independent society Journal, I'm likely to be one of the first on the chopping block, which I don't relish. And yet I think it's worth it for the potential benefits to science, and the resultant benefits to humanity as a whole. The STM publishing industry is currently in crisis, but publishers are proud of their tradition and slow to change. At publishing meetings, every talk has an underlying current of panic, but also the feeling of inevitability. Change would have to occur at the more powerful publishing companies that publish multiple Journals. For example, NPG or Cell Press (which are already pulling small independent publishers under their umbrella in the name of economies of scale) could create a top-down publishing process such that there is one submission process for all of their Journals in the same field, and an article can be assigned to a particular level Journal within the system without having to be resubmitted. Some Journals already accept peer review from previous submissions to expedite the review process. My plan merely formalizes the process. As authors realized the simplicity of the system, submissions to independent presses would likely dry up, and they would join the larger publisher umbrellas. This seems possible, and even likely.

Less likely is the dynamic article ranking/comment process. Publishers like 'copyright' and 'official record', whatever that means in today's world of online interactivity. And yet, I think that STM publishers are especially prone to experimentation. I think it's possible, over time, that this systemic change could come to pass, and that it SHOULD come to pass if STM publishers want to remain viable and relevant to the changing needs of information consumers. As to open access, while I think that publishers can find other ways to provide revenue than by charging for information access, I doubt this will occur unless by government mandate. (Note that publishers seem to be flourishing even under the current open access government mandates).

I'll be interested to see how scientific motivation changes in the future, and I think that scientific publishers will play a key role in guiding that motivation. Without the proper innovations, as in conventional news media, they (we) will not survive. They truly must "Publish or Perish," and I hope that science doesn't perish with them.