Swapnaa Jayaraman, Indiana University
Rick O. Gilmore, Penn State
Florian Raudies, HP Research
Supported by NSF BCE-1147440, NSF BCS-1238599, NICHD U01-HD-076595
2016-05-26 14:20:35
Changes in early optic flow experiences across development and culture
Background
- What is optic flow?
- How does sensitivity to optic flow develop?
- How do infants' natural experiences with optic flow develop?
- Do the statistics of infants' natural experiences align with changes in sensitivity?
What is Optic Flow?
Optic Flow Components
How Does Optic Flow Sensitivity Develop?
- Sensitivity at birth (Jouen et al. 2000)
- Infants more sensitive to fast, translational flow (Hou et al. 2009), (R. O. Gilmore et al. 2007)
- Adults more sensitive to slow, radial flow
Developmental Changes in Experiences of Optic Flow
- Infants experience faster flow than mothers even when geometry, speed of locomotion equated (F. Raudies et al. 2012)
- Motion "prior" for infants faster than for mothers (Florian Raudies and Gilmore 2014) and empirical motion priors not "slow" as predicted by theory (Stocker and Simoncelli 2006)
Developmental/Cultural Factors
- Posture: Walking vs. Crawling
- Distance to ground surface
- Head pitch relative to ground
- Geometry: Typical distance to surfaces, objects; number of people, objects
- Moving through space: self vs. object motion
Approach
- Simulating changes in optic flow due to changes in body size & posture, geometry
- Measuring flow experienced in natural contexts by head-mounted cameras
Optic Flow Equation
\(\begin{pmatrix}\dot{x} \\ \dot{y}\end{pmatrix}=\frac{1}{z} \begin{pmatrix}-f & 0 & x\\ 0 & -f & y \end{pmatrix} \begin{pmatrix}{v_x{}}\\ {v_y{}} \\{v_z{}}\end{pmatrix}+ \frac{1}{f} \begin{pmatrix} xy & -(f^2+x^2) & fy\\ f^2+y^2 & -xy & -fy \end{pmatrix} \begin{pmatrix} \omega_{x}\\ \omega_{y}\\ \omega_{z} \end{pmatrix}\)
Parameters For Simulation
| Parameter | Crawling Infant | Walking Infant |
|---|---|---|
| Eye height | 0.30 m | 0.60 m |
| Locomotor speed | 0.33 m/s | 0.61 m/s |
| Head tilt | 20 deg | 9 deg |
Parameters for Simulation
| Geometric Feature | Distance |
|---|---|
| Side wall | +/- 2 m |
| Side wall height | 2.5 m |
| Distance of ground plane | 32 m |
| Field of view width | 60 deg |
| Field of view height | 45 deg |
Simulating Flow Fields
Flow Direction Distributions by Geometry & Posture
Simulated Flow Speeds (m/s)
| Type of Locomotion | Ground Plane | Room | Side Wall | Two Walls |
|---|---|---|---|---|
| Crawling | 14.41 | 14.42 | 14.43 | 14.62 |
| Walking | 9.38 | 8.56 | 7.39 | 9.18 |
Measuring Optic Flow in Natural Contexts
- First-person videos from head-mounted cameras
- Parents asked to record 4-6 hrs of daily activities.
Sample characteristics
| Country | Females | Males | Age (wks) | Coded video Hrs |
|---|---|---|---|---|
| India | 17 | 13 | 3-63 | 3.1 (0.5-6.0) |
| U.S. | 15 | 19 | 4-62 | 4.6 (0.2-7.6) |
Data & excerpts: http://databrary.org/volume/81
Workflow
- Extract optic flow from video, (code repo)
- Human coders: is infant moving or stationary?
- How do flow speeds and patterns vary by
- Age
- Geography/Culture
- Moving/stationary status
Moving vs. Stationary Bouts
P(moving) by Age, Location
Illustrative Speed Histograms - 6 weeks
Illustrative Speed Histograms – 34 weeks
Illustrative Speed Histograms – 58 weeks
Empirical Pattern Distributions
- Correlation with 'canonical' flow patterns
- radial
- rotational
- translational
Pattern Correlation Results
Conclusions: Simulation
- Posture influences optic flow speeds & patterns
- Crawling: faster speeds, more translational flow
- Proximity to ground and pitch of head
- Geometry matters relatively little
Conclusions: Empirical Data
- Time stationary >> time in motion
- Time in motion increases, faster in U.S.
- Fast speeds, broad speed distributions
- Linear flow >> radial or rotational flow
Implications
- Infant perception, brain responses reflect statistics of natural visual environment
- Modest differences in experienced visual motion by culture
- Are fast speeds perceptually meaningful?
- Prolonged development of gaze stabilization, visual proprioception
Stack
- Databrary, https://nyu.databrary.org/volume/81
- Datayvu, http://datavyu.org
- Matlab-based optic flow routines, http://github.com/opticflow
- RStudio, https://www.rstudio.com/
- GitHub, http://github.com/gilmore-lab/jayaraman-gilmore-raudies-ICIS-2016/
References
Gilmore, R.O., F. Raudies, and S. Jayaraman. 2015. “What Accounts for Developmental Shifts in Optic Flow Sensitivity?” In 2015 Joint IEEE International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob), 19–25. doi:10.1109/DEVLRN.2015.7345450.
Gilmore, Rick O., C. Hou, M.W. Pettet, and A.M. Norcia. 2007. “Development of Cortical Responses to Optic Flow.” Visual Neuroscience 24 (06): 845–56. doi:10.1017/S0952523807070769.
Gilmore, Rick O., Florian Raudies, Swapnaa Jayaraman, and Linda B. Smith. 2014. “Natural Scene Statistics of Visual Experience Across Development and Culture.” Databrary. doi:10.17910/B7988V.
Hou, C., R. O. Gilmore, M. W. Pettet, and A. M. Norcia. 2009. “Spatio-Temporal Tuning of Coherent Motion Evoked Responses in 4–6 Month Old Infants and Adults.” Vision Research 49 (20): 2509–17. doi:10.1016/j.visres.2009.08.007.
Jouen, François, Jean-Claude Lepecq, Olivier Gapenne, and Bennett I Bertenthal. 2000. “Optic Flow Sensitivity in Neonates.” Infant Behavior and Development 23 (3–4): 271–84. doi:10.1016/S0163-6383(01)00044-3.
Kretch, Kari S., John M. Franchak, and Karen E. Adolph. 2014. “Crawling and Walking Infants See the World Differently.” Child Development 85 (4): 1503–18. doi:10.1111/cdev.12206.
Raudies, F., R.O. Gilmore, K.S. Kretch, J.M. Franchak, and K.E. Adolph. 2012. “Understanding the Development of Motion Processing by Characterizing Optic Flow Experienced by Infants and Their Mothers.” In 2012 IEEE International Conference on Development and Learning and Epigenetic Robotics (ICDL), 1–6. doi:10.1109/DevLrn.2012.6400584.
Raudies, Florian, and Rick O. Gilmore. 2014. “Visual Motion Priors Differ for Infants and Mothers.” Neural Computation 26 (11): 2652–68. doi:10.1162/NECO_a_00645.
Stocker, Alan A., and Eero P. Simoncelli. 2006. “Noise Characteristics and Prior Expectations in Human Visual Speed Perception.” Nature Neuroscience 9 (4): 578–85. doi:10.1038/nn1669.