Open, Reliable and Transparent Data
2024-02-28
Sexual selection in plants
- Bateman gradients in angiosperms
- N = 2 (in 2021)
- Project goal
- Conduct a meta-analysis đ€
- Find datasets that could be re-analysed in this new context
- Combine into a meta-analysis to test predictions
Sexual selection in plants
- Initial search
- N=2167 đ
- After sorting
- N=30 đ„č
- After trying to source data
- N=9 đ
Exxon Valdez oil spill 1989
- 40.8 million litres of crude oil spilled
- Settlement funds from Exxon used for research and monitoring the impacts of the spill
- Between 1989 and 2010, 419 projects were funded
- In 2012, NCEAS tried to compile all historic datasets
- 70% were unrecoverable
Opaque research
- Publication bias
- Not all research is published
- Incomplete or insufficiently detailed methods
- Selective reporting in results
- Confirmation bias
- âHARKingâ
- âP-hackingâ
- Unaccessible underlying data
Opaque research limits science
- Harder to replicate or re-use methods
- Harder to build upon to progress the field
- Harder to interpret results
- Harder to trust the conclusions
Reproducible and reliable results
- Promotes accountability and trust
- Mistakes can be corrected1
- Analytical decisions can be justified
- Scientific misconduct canât hide
New questions & new methods
- Built upon more effectively
- Deeper understanding of data & analysis
- Used to develop new tools/methods/protocols
- E.g. Bumpus 1899
- Viewed in a new light
- Beyond the original paper
- Paradigm shifts
- Analysed using the latest methods
- Meta analysis
More accurate meta-analysis
- Easy extraction of accurate data
- No need to extract from figure
- Reduces ambiguity and error
- Go beyond the results section
- Helps reduce bias from selective reporting
- Capture the full picture of the study
- Extends the life the dataset
- Can always be accessed
Learning and teaching
- Teaching students using example datasets
- Real biological âquirksâ
- Real scenarios
- Can teach good practises from the start
- Learning and understanding new methods
- Complexity can be broken down
- Walkthrough when code also available
Benefits for the data archiver
- Increased exposure, reach, and trustworthiness
- Citation advantage (+25%) 1
- Your own best collaborator
- Data is clean and ready to use
- Well annotated
- Cannot be lost
Reducing research loss & waste
- Removes need for duplicated data collection effort
- Time/location/event dependant data
- Research animal use
- Reduces cost of research
Transparency and Openness Promotion (TOP) guidelines
- âA set of standards applied to journals to measure their alignment with open scientific principlesâ
- Specific guidance on data transparency:
- Level 3: open data + peer review of dataset and analysis
- Level 2: open data in trusted repository
- Level 1: mandatory data statement
- Specific guidance on data transparency:
- >5000 journals are signatories
- Field specific advice for ecology and evolution
How well are we doing?
How well are we doing?
Published without sufficient data to replicate:
- 89% (N=18) of micro-array gene expression analyses (Ioannidis et al. 2009)
- 35% (N=19) of population genetic studies (Gilbert et al. 2012)
- 64% (N=100) of non-molecular eco/evo studies in journals that mandate data archiving (Roche et al. 2015)
How do we improve things?
- Why we donât share data?
- Knowledge barriers
- Re-use concerns
- Disincentives
- How to work towards data transparency
Whatâs the process?
- Do not know how to share data effectively
- Which online data repository to use?
- What format to share data in?
Whatâs the process?
- Online guides and primers
- British Ecological Society âGuides to Better Scienceâ
- UKRN Primers
- SORTEE (coming soon)
Whatâs the process?
- Institutional libraries
- Often under-utilised advice and guidance
- FAIR templates and guides
- Any data is better than no data!
- Learn by doing
Insecurities
- Early career researchers can feel especially vulnerable
- Fear, insecurity and embarressment are powerful emotions
Insecurities
- Share before publication
- Lab meetings or data review sessions
- Pre-print (private or open)
- Data being hard to understand is bigger issue
- Culture that prioritises learning over citisism
Donât see value in their data
- Too niche
- Too small
- Why would someone be interested?
Donât see value in their data
- Highly subjective
- Hard to predict future use
- + all other benefits
Sensitive information
- Dual use problem
- Weigh up benefits and costs
- Ethical (and legal) implications
- Sharing limited subset
- Species example guidelines: Chapman 2020
Scooping
Fear of:
- A researcher performs an analysis on publicly shared data that the original data collected had not done yet
- Being âscoopedâ
- Reduced collaborations
- Loss of future publications
- Metric used to assess performance
Scooping
Less likely than you would imagine:
- Ideas are plentiful
- Original collectors in best position to act
- Most analyses by original authors on published data happen within 2 years1
- Most analysis by other researchers peak at 5 years^1
Scooping
- Pre-print to âclaimâ
- If major concern:
- restrictions on use of data can be made
- embargo periods
- Change in mindset to see data as a valuable contribution
2. Identify an appropriate repository
- Field specific
- Data type specific
- Journal preferences
- Good starting place: re3data.org
4. Pre-peer-review peer-review
- Ask a colleague to look through your README and dataset
- Data/code review sessions
- Can they make sense of it?
