Interesting series of articles in June 26, 2015, Science

[mstp1992]

A couple of articles in this issue of Science looks at the problems associated with reproducibility of results following publication in any number of scientific journals with a wide range of impact factors, including some upper tier journals. There have been some significant cases reported in the media of frankly fraudulent research. While these have been highlighted in the media, the vast majority of scientists remain committed to ethical scientific endeavors. Transparency is an issue that needs to improve in science. Published data needs to be made available when appropriately requested. Methods need to be crystal clear so that another researcher accurately replicate the experimental conditions.

Some points they bring up include the use of open-source platforms. But most of these have been created by 'volunteers' or the private market, each of which brings its own set of caveats. Concluding that some public research funds should be set aside and used for creation of such platforms makes a lot of sense to me.

Very interesting topic as far as I'm concerned.

Editorial -- Stuart Buck

Self-correction in science at work

Promoting an open research culture

 

[KitingHigh]

You could start the openness by sharing a link.

 

[mattymoknows]

It's a reality that many scientists are strictly in the profession to make a solid living and with that tend to be the ones doing research/projects that will inevitably be for the benefit of someone's, or some company's, bottom line. Yet, this isn't all bad if they're working towards things like technological/medical advancements. However no one should be fooled that all things done in the name of science are noble, sadly.

 

[mstp1992]

You could start the openness by sharing a link.

I thought I had. I keep forgetting we have institutional subscriptions.

Here's the text of the editorial by Stuart Buck:

The reproducibility problem in science is a familiar issue, not only within the scientific community, but with the general public as well. Recent developments in social psychology (such as fraudulent research by D. Stapel) and cell biology (the Amgen Inc. and Bayer AG reports on how rarely they could reproduce published results) have become widely known. Nearly every field is affected, from clinical trials and neuroimaging, to economics and computer science. Obvious solutions include more research on statistical and behavioral fixes for irreproducibility, activism for policy changes, and demanding more pre-registration and data sharing from grantees. Two Perspectives in this issue (pp. 1420 and 1422) describe how journals and academic institutions can foster a culture of reproducibility. Transparency is central to improving reproducibility, but it is expensive and time-consuming. What can be done to alleviate those obstacles?

Most scientists aspire to greater transparency, but if being transparent taps into scarce grant money and requires extra work, it is unlikely that scientists will be able to live up to their own cherished values. Thus, one of the most effective ways to promote high-quality science is to create free open-source tools that give scientists easier and cheaper ways to incorporate transparency into their daily workflow: from open lab notebooks, to software that tracks every version of a data set, to dynamic document generation. Moreover, scientists who use open-source software are not locked into proprietary software platforms with unclear monetization plans. If philanthropy or government funds new tools that the open-source community can iterate and improve on, the per-dollar return on investment can far exceed the costs.

Infrastructural tools are now available, or in development, that should help to catalyze a change in scientific transparency. One example is the Open Science Framework (OSF), a free and open-source software platform for managing scientific workflow (supported by the Laura and John Arnold Foundation in partnership with the Center for Open Science). Among its many features, this platform can enable scientists to easily track the history of all versions of every document or data set and the exact contributions made by each team member. All project materials can be given persistent identifiers, and the tracking of provenance allows any subsequent research project to give proper credit to the original. Projects using this platform include the Shared Access Research Ecosystem project of the Association of Research Libraries and its partners. This project endeavors to connect scholarly metadata and allow the identification of various elements of a research project, such as grant proposals, journal articles, and data repository information.

Open-source platform innovations are growing. Other examples include the iPython project (supported largely by the Alfred P. Sloan Foundation), which offers a web-based computing notebook for users to create documents such as code, computational results, and narrative explanations. Although originally developed around the Python language, the project has expanded to cover other languages (such as R) under the banner of Project Jupyter. The Galaxy Project (funded by the U.S. National Science Foundation and others) provides a web-based platform for “data-intensive” biomedical research; and for managing bioinformatics and phylogenetic research on plants, there is the iPlant Collaborative.

The scientific community cannot depend entirely on volunteers or the private market to develop free platforms that address specialized scientific needs and encourage greater reproducibility. Ultimately, the infrastructure supporting science is a public good, just like the knowledge it produces. By supporting such infrastructure, the public and philanthropic sectors can make it painless for researchers to live up to their own values of openness and rigor.

 

[KitingHigh]

I thought I had. I keep forgetting we have institutional subscriptions.

Here's the text of the editorial by Stuart Buck:

The reproducibility problem in science is a familiar issue, not only within the scientific community, but with the general public as well. Recent developments in social psychology (such as fraudulent research by D. Stapel) and cell biology (the Amgen Inc. and Bayer AG reports on how rarely they could reproduce published results) have become widely known. Nearly every field is affected, from clinical trials and neuroimaging, to economics and computer science. Obvious solutions include more research on statistical and behavioral fixes for irreproducibility, activism for policy changes, and demanding more pre-registration and data sharing from grantees. Two Perspectives in this issue (pp. 1420 and 1422) describe how journals and academic institutions can foster a culture of reproducibility. Transparency is central to improving reproducibility, but it is expensive and time-consuming. What can be done to alleviate those obstacles?

Most scientists aspire to greater transparency, but if being transparent taps into scarce grant money and requires extra work, it is unlikely that scientists will be able to live up to their own cherished values. Thus, one of the most effective ways to promote high-quality science is to create free open-source tools that give scientists easier and cheaper ways to incorporate transparency into their daily workflow: from open lab notebooks, to software that tracks every version of a data set, to dynamic document generation. Moreover, scientists who use open-source software are not locked into proprietary software platforms with unclear monetization plans. If philanthropy or government funds new tools that the open-source community can iterate and improve on, the per-dollar return on investment can far exceed the costs.

Infrastructural tools are now available, or in development, that should help to catalyze a change in scientific transparency. One example is the Open Science Framework (OSF), a free and open-source software platform for managing scientific workflow (supported by the Laura and John Arnold Foundation in partnership with the Center for Open Science). Among its many features, this platform can enable scientists to easily track the history of all versions of every document or data set and the exact contributions made by each team member. All project materials can be given persistent identifiers, and the tracking of provenance allows any subsequent research project to give proper credit to the original. Projects using this platform include the Shared Access Research Ecosystem project of the Association of Research Libraries and its partners. This project endeavors to connect scholarly metadata and allow the identification of various elements of a research project, such as grant proposals, journal articles, and data repository information.

Open-source platform innovations are growing. Other examples include the iPython project (supported largely by the Alfred P. Sloan Foundation), which offers a web-based computing notebook for users to create documents such as code, computational results, and narrative explanations. Although originally developed around the Python language, the project has expanded to cover other languages (such as R) under the banner of Project Jupyter. The Galaxy Project (funded by the U.S. National Science Foundation and others) provides a web-based platform for “data-intensive” biomedical research; and for managing bioinformatics and phylogenetic research on plants, there is the iPlant Collaborative.

The scientific community cannot depend entirely on volunteers or the private market to develop free platforms that address specialized scientific needs and encourage greater reproducibility. Ultimately, the infrastructure supporting science is a public good, just like the knowledge it produces. By supporting such infrastructure, the public and philanthropic sectors can make it painless for researchers to live up to their own values of openness and rigor.

My bad, you did.

 

[mstp1992]

Sorry, I should have highlighted them better.

I think it's especially timely given the ex-ISU scientist and his federal sentencing today.