NSF Open Science Metrics Report

1 Data summary

We started with 1000 articles funded by the National Science Foundation; these articles were in xml format.

2 Data Generation & Sharing

We exclude articles that did not generate data. Articles sharing data as supplemental material were identified by locating the list of supplemental files in the full text and finding terms like ‘dataset’ or ‘data file’ in either the supplemental file name or description.

Articles sharing data online contained a) links to data repositories (e.g. Zenodo or Dryad) or other online locations (e.g. Github), b) DOIs or other Permanent Identifiers (PIDs) for data repositories, or c) accession numbers associated with data repositories. When an article was found to share data both in the supplemental material and online, we classified these as ‘online’ in the below table and figure.

Data Generation & Sharing
Data Sharing Status Total Percent
No data generated 29 3
No data shared 524 52
Supp mat 242 24
Online 204 20

3 Data repositories

The online repositories identified for ‘data sharing’ above can be identified through their DOIs or other PIDs, the site linked to by a dataset URL, or through the characteristic format of an accession number (e.g. PRJNA12345 is the format of a BioProject on the NCBI website).

As with data types above, the repositories for each article can be found in the raw data, and here we present a summary for the corpus as a whole. Repositories with less than 0.5% usage were combined into an ‘Other’ category.

Most commonly used data repositories
Data Repositories Total Percent
Github 124 44.4
NCBI Bioproject 66 23.7
Zenodo 17 6.1
figshare 15 5.4
NCBI Sequence Read Archive 14 5.0
Open Science Framework 9 3.2
Other 9 3.2
Gene Expression Omnibus 7 2.5
European Variation Archive 6 2.2
NCBI Biosample 6 2.2
Dryad Digital Repository 4 1.4
ENA or EBI Sequence Read Archive 2 0.7

4 Code Generation & Sharing

To identify articles producing shareable code objects the full text is searched for the names of command line software or programming environments. Articles mentioning one or more software or environments are classified as ‘generating code’.

To detect code sharing, we searched the full text for a) links or b) DOIs (or other PIDs) for repositories where code objects could be shared. We also check the supplemental file titles and descriptions for terms like ‘script’ or ‘code’ for code sharing in the supplemental material.

Articles found to share code both in the supplemental material and online are grouped under ‘online’ in the table and figure below.

Code Object Generation & Sharing
Code Sharing Status Total Percent
No code generated 353 35
No code shared 489 49
Supp. mat. 3 0
Online 155 16

5 Preprinting

To assess whether any articles were posted as a preprint, we performed a fuzzy match for the article title and authors against a listing of articles from all of the major preprint servers.

Preprint Repositories Total Percent
No preprint 759 76
bioRxiv 224 22
Other 10 1
Research Square 7 1