To boldly go where no science…

Karen E. Adolph, David Millman, and Rick O. Gilmore

Acknowledgments

TL; DR

  • Video is essential for reproducible behavioral science
    • Even where video is not the primary raw data
  • Databrary specializes in storing and sharing research video
  • Video is identifiable, but it can be shared
  • All behavioral scientists should collect and share videos of their research

Is behavioral science reproducible?

Why we're unsure

  • Behavior rich, complex
  • Numeric, text-based measures reduce that complexity
  • Video captures and preserves it

Why we're unsure

  • Replications can fail due to methodological differences
  • Methods sections can't possibly report essential details
  • Video captures and preserves it

A reproducible behavioral science must…

  • Video record all tasks, measures, and behaviors
  • Share the recordings
  • Share all questionnaires, tasks, displays
  • Share statistical, computational, data workflows
  • Prepare to share from the beginning
  • Seek permission to share data

Databrary.org makes this possible

  • Digital data library specialized for research video
  • Video/audio + participant/context metadata
  • Share displays, materials, text-based data files
  • Restricted access for research/educational use
  • Policy framework for sharing identifiable data
  • Developmental focus, but not exclusive

How Databrary overcomes barriers to sharing video

  • Policies for sharing identifiable video data
  • Tools for reproducibly coding video
  • Tools for "active curation" == during data collection

Policies

  • Restrict access to authorized researchers (& affiliates)
  • Seek permission to share data from participants

Standardized release levels

Tools for coding video

  • Raw research video must be coded
  • Datavyu a free, open source coding tool
  • Add codes, annotations time-locked to video segments
  • Turn behavior into quantifiable data
  • Ruby API for scripting reproducible workflows

Curating data as it is collected

  • After-the-fact curation burdensome
  • Databrary organizes, shares, standardized participant metadata
  • Sharing based on
    • user access level
    • participant permission

(Adolph 2013)

Take homes

  • Video uniquely captures complexity & richness of behavior
  • Video captures vital methodological details
  • Identifiable video (and other data) can be shared
  • Preparing to share reduces the burden
  • Video data sharing essential for a reproducible behavioral science

References

Adolph, Karen; 2013. “Infants Crawling and Walking over High and Low Bridges.” Databrary. doi:10.17910/B7MW2K.

Collaboration, Open Science. 2015. “Estimating the Reproducibility of Psychological.” Science 349 (6251): aac4716. doi:10.1126/science.aac4716.

Gilbert, Daniel T., Gary King, Stephen Pettigrew, and Timothy D. Wilson. 2016. “Comment on ‘Estimating the Reproducibility of Psychological Science’.” Science 351 (6277): 1037–7. doi:10.1126/science.aad7243.