dev-prosem-2015-09-16
Rick Gilmore
2015-09-16 10:25:04
Experience matters
Measuring the microstructure of visual experience to better understand perceptual development
Overview
- Development of Optic Flow Processing
- What is optic flow?
- How does optic flow sensitivity develop?
- How experience shapes the patterns of brain and behavioral development
What is Optic Flow?
- Structured pattern of visual motion generated by observer movement
Examples of Optic Flow
Examples of Optic Flow
Examples of Optic Flow
What is Optic Flow?
Types of Optic Flow
- Radial (expansion/contraction), rotation, linear/laminar/translational, shear
Different Actions, Different Flows
| Action Type | Flow Type |
|---|---|
| Forward/backward translation | Expansion/contraction |
| Rotation around visual axis | Rotation |
| Horizontal/vertical head rotation | Translation |
| Horizontal/vertical eye movements | Translation |
Functions linked to optic flow processing
- Environmental geometry
- Egocentric position, trajectory
- Forward models/sensorimotor stability
- Self vs. world/object motion
- Object detection, identification
How Does Optic Flow Sensitivity Develop?
- Sensitivity at birth, (Jouen et al. 2000)
- Infants
- Brain responses stronger to fast, translational flow, (Hou et al. 2009; R. O. Gilmore et al. 2007)
- Behavioral responses stronger to fast translational flow, (Kiorpes and Movshon 2004)
- Primate universal pattern? Not adult-like until late adolescence?
- Adults
- Brain responses stronger to radial flow, (R. O. Gilmore et al. 2007; Fesi, Thomas, and Gilmore 2014).
Gaps
- What influences developmental shifts?
- Why fast speeds and linear flows?
Infants Spend Lots of Time Being Carried
What Types of Flow Occur During Carrying?
- (Florian Raudies and Gilmore 2014; F. Raudies et al. 2012)
- (K. S. Kretch, Franchak, and Adolph 2014; K. S. Kretch and Adolph 2014)
- https://databrary.org/volume/123
Infant View
Adult View
Methods
- Head-mounted eye tracking videos
- n=6 8.5-9.5 mo-old infants carried by mothers
- 30 s video segments walking down the same indoor hallway
- Frame-by-frame analysis of optic flow patterns, speeds.
- (Florian Raudies 2013)
Reminiscent of (Held and Hein 1963)
Flow Pattern Data
Flow Speed Data
Gaze Stability
Gaze Stability
Looking Where They're Going?
What is the Motion 'Prior'?
- (Grzywacz and Yuille 1991,Weiss, Simoncelli, and Adelson (2002))
- Visual system 'assumes' motion distributions are slow
- Resolves ambiguities in direction under occlusion
- Based on lab-based psychophysical results
What is the Empirical Motion 'Prior'?
What is the Empirical Motion 'prior'?
Summary
- Infants' visual experiences ≠ mothers'
- Faster speeds, more vertical motion
- Laminar/translational motion common
- Empirical speed prior not slow
Concerns
- How general?
- Geometry of environment?
- Carrying vs. independent locomotion?
- Cultural differences in home environment, relatives, carrying practices?
- Changes across developmental milestones?
Methods
- Simulation
- How does "maturation" change optic flow?
- Does environmental geometry change optic flow?
- Measure natural scene statistics of optic flow
- Videos from head-mounted cameras
- Infants from India, Indiana
Simulating Optic Flow
(˙x˙y)=1z(−f0x0−fy)(vxvyvz)+1f(xy−(f2+x2)fyf2+y2−xy−fy)(ωxωyωz)
Parameters For Simulation
| Parameter | Crawling Infant | Walking Infant |
|---|---|---|
| Eye height | 0.30 m | 0.60 m |
| Locomotor speed | 0.33 m/s | 0.61 m/s |
| Head tilt | 20 deg | 9 deg |
Parameters for Simulation
| Geometric Feature | Distance |
|---|---|
| Side wall | +/- 2 m |
| Side wall height | 2.5 m |
| Distance of ground plane | 32 m |
| Field of view width | 60 deg |
| Field of view height | 45 deg |
Simulating Flow Fields
Flow Direction Distributions by Geometry & Posture
Flow Speeds By Geometry and Posture
Mean Simulated Flow Speeds By Posture and Geometry
| Type of Locomotion | Ground Plane | Room | Side Wall | Two Walls |
|---|---|---|---|---|
| Crawling | 14.41 | 14.42 | 14.43 | 14.62 |
| Walking | 9.38 | 8.56 | 7.39 | 9.18 |
Empirical Measurements of Optic Flow
- First-person videos from head-mounted cameras
- 20 infants, 41 days to 13.2 mos
- Chennai, India & Bloomington, Indiana
- Data: http://databrary.org/volume/81
6 weeks
36 weeks
Cultural Differences in Segment Durations
Segment Durations
Normalized Durations as p(total-time)
Natural Scene Statistics for Optic Flow
- Selected 10 5 s segments/participant, both moving and stationary
- Estimated frame by frame flow fields
- Details in Raudies & Gilmore 2014
Speed Distributions
Comparing Shapes of (Trimmed) Speed Distributions
- Fit γ distribution to trimmed (0,100) speed histograms
f(x;k,θ)=αxk−1e−xθθkΓ(k)
α, amplitude; κ, shape; and θ, scale parameters.Illustrative Speed Histograms – 6 weeks
Illustrative Speed Histograms – 34 weeks
Illustrative Speed Histograms – 58 weeks
Fitted κ Parameters
Fitted α Parameters
Fitted θ Parameters
Summary: Empirical Flow Speed Effects
- Fast speeds (> 100 deg/s) common
- Moving ≠ Stationary
- Broad distribution: κ, α(moving) > κ, α(stationary)
- U.S. ~ India
Empirical Pattern Distributions
- Correlation with 'canonical' flow patterns
- radial
- rotational
- translational
Pattern Correlation Results
Pattern Correlation Results by Country
Moving Laminar ≠ Stationary Laminar in 13/22 infants.
Conclusions: Simulation
- Posture influences optic flow speeds & patterns
- Crawling: faster speeds, more translational flow
- Proximity to ground and pitch of head
- Geometry matters relatively little
Conclusions: Empirical Data
- Time stationary >> time in motion
- Time stationary declines with age (India)
- Fast speeds, broad speed distributions
- Individual differences in moving vs. stationary speed distributions
- Laminar flow >> radial or rotational flow, especially when stationary
- Replicates and extends Raudies & Gilmore '12, '14
Summmary of Findings
- Infants commonly experience fast, laminar flows.
- Statistics of visual input may shape developmental transition from fast laminar to slow radial flow.
- Measuring the microstructure of experience hard, but informative.
- Broadly supported by EEG data with children
Thank you
Stack
- RStudio, https://www.rstudio.com/
- R Markdown, http://rmarkdown.rstudio.com/
- GitHub, http://github.com/gilmore-lab/, http://github.com/psu-psychology/developmental
- Databrary, https://databrary.org/volume/81
- Access to identifiable data (e.g. videos) restricted
- Access agreement + institutional authorization
- Datayvu, http://datavyu.org
- Matlab
References
Fesi, Jeremy D., Amanda L. Thomas, and Rick O. Gilmore. 2014. “Cortical Responses to Optic Flow and Motion Contrast Across Patterns and Speeds.” Vision Research 100 (July): 56–71. doi:10.1016/j.visres.2014.04.004.
Gilmore, Rick O., C. Hou, M.W. Pettet, and A.M. Norcia. 2007. “Development of Cortical Responses to Optic Flow.” Visual Neuroscience 24 (06): 845–56. doi:10.1017/S0952523807070769.
Grzywacz, NM, and A Yuille. 1991. “Theories for the Visual Perception of Local Velocity and Coherent Motion.” Computational Models of Visual Processing. MIT Press, Cambridge, MA, USA, 231–52.
Held, Richard, and Alan Hein. 1963. “Movement-Produced Stimulation in the Development of Visually Guided Behavior.” Journal of Comparative and Physiological Psychology 56 (5). American Psychological Association: 872.
Hou, C., R.O. Gilmore, M.W. Pettet, and A.M. Norcia. 2009. “Spatio-Temporal Tuning of Coherent Motion Evoked Responses in 4–6 Month Old Infants and Adults.” Vision Research 49 (20): 2509–17. doi:10.1016/j.visres.2009.08.007.
Jouen, François, Jean-Claude Lepecq, Olivier Gapenne, and Bennett I Bertenthal. 2000. “Optic Flow Sensitivity in Neonates.” Infant Behavior and Development 23 (3–4): 271–84. doi:10.1016/S0163-6383(01)00044-3.
Kiorpes, Lynne, and J. Anthony Movshon. 2004. “Development of Sensitivity to Visual Motion in Macaque Monkeys.” Visual Neuroscience 21 (6): 851–59. doi:10.1017/S0952523804216054.
Kretch, Kari S, and Karen E Adolph. 2014. “Active Vision in Passive Locomotion: Real-World Free Viewing in Infants and Adults.” Developmental Science. Wiley Online Library.
Kretch, Kari S, John M Franchak, and Karen E Adolph. 2014. “Crawling and Walking Infants See the World Differently.” Child Development 85 (4). Wiley Online Library: 1503–18.
Mathioudakis, E., D. Badaly, and K.E. Adolph. 2008. “One Child’s Day,” May.
Raudies, F., R.O. Gilmore, K.S. Kretch, J.M. Franchak, and K.E. Adolph. 2012. “Understanding the Development of Motion Processing by Characterizing Optic Flow Experienced by Infants and Their Mothers.” In 2012 IEEE International Conference on Development and Learning and Epigenetic Robotics (ICDL), 1–6. doi:10.1109/DevLrn.2012.6400584.
Raudies, Florian. 2013. “Optic Flow.” Scholarpedia 8 (7): 30724.
Raudies, Florian, and Rick O. Gilmore. 2014. “Visual Motion Priors Differ for Infants and Mothers.” Neural Computation 26 (11): 2652–68. doi:10.1162/NECO_a_00645.
Weiss, Yair, Eero P Simoncelli, and Edward H Adelson. 2002. “Motion Illusions as Optimal Percepts.” Nature Neuroscience 5 (6). Nature Publishing Group: 598–604.