- NSF BCS-1238599
- NICHD U01-HD-076595
- Society for Research in Child Development (SRCD)
- NYU Libraries, NYU IT, IRB, and OSP
Acknowledgments
TL; DR
- Video is essential for reproducible behavioral science
- Even where video is not the primary raw data
- Databrary specializes in storing and sharing research video
- Video is identifiable, but it can be shared
- All behavioral scientists should collect and share videos of their research
Is behavioral science reproducible?
- Nosek et al. (Collaboration 2015)
- vs. Gilbert et al. (Gilbert et al. 2016)
Why we're unsure
- Behavior rich, complex
- Numeric, text-based measures reduce that complexity
- Video captures and preserves it
Why we're unsure
- Replications can fail due to methodological differences
- Methods sections can't possibly report essential details
- Video captures and preserves it
A reproducible behavioral science must…
- Video record all tasks, measures, and behaviors
- Share the recordings
- Share all questionnaires, tasks, displays
- Share statistical, computational, data workflows
- Prepare to share from the beginning
- Seek permission to share data
Databrary.org makes this possible
- Digital data library specialized for research video
- Video/audio + participant/context metadata
- Share displays, materials, text-based data files
- Restricted access for research/educational use
- Policy framework for sharing identifiable data
- Developmental focus, but not exclusive
How Databrary overcomes barriers to sharing video
- Policies for sharing identifiable video data
- Tools for reproducibly coding video
- Tools for "active curation" == during data collection
Policies
- Restrict access to authorized researchers (& affiliates)
- Institutional access agreement
- Seek permission to share data from participants
Standardized release levels
Tools for coding video
- Raw research video must be coded
- Datavyu a free, open source coding tool
- Add codes, annotations time-locked to video segments
- Turn behavior into quantifiable data
- Ruby API for scripting reproducible workflows
Curating data as it is collected
- After-the-fact curation burdensome
- Databrary organizes, shares, standardized participant metadata
- Sharing based on
- user access level
- participant permission
(Adolph 2013)
Take homes
- Video uniquely captures complexity & richness of behavior
- Video captures vital methodological details
- Identifiable video (and other data) can be shared
- Preparing to share reduces the burden
- Video data sharing essential for a reproducible behavioral science
References
Adolph, Karen; 2013. “Infants Crawling and Walking over High and Low Bridges.” Databrary. doi:10.17910/B7MW2K.
Collaboration, Open Science. 2015. “Estimating the Reproducibility of Psychological.” Science 349 (6251): aac4716. doi:10.1126/science.aac4716.
Gilbert, Daniel T., Gary King, Stephen Pettigrew, and Timothy D. Wilson. 2016. “Comment on ‘Estimating the Reproducibility of Psychological Science’.” Science 351 (6277): 1037–7. doi:10.1126/science.aad7243.