- Physical Review Journals
All JournalsPhysics Magazine
Access Provided by Xinjiang University

Physical Review E

Export citation

Export citation

Choose format for download:

Download Citation
  • Editors' Suggestion
  • Access by Xinjiang University

The FlEye camera: Sampling the joint distribution of natural scenes and motion

Charles J. Edelson1,*,†, Paul Smith1, Sima Setayeshgar1, William Bialek2, and Rob R. de Ruyter van Steveninck1

  • *Contact author: ce3721@princeton.edu
  • †Present address: Princeton University, Princeton, New Jersey 08544, USA.
PDF

Phys. Rev. E 113, 044412 – Published 14 April, 2026

DOI: https://doi.org/10.1103/31k4-nq97

Abstract

Processing of sensory signals by the brain is subject to physical limitations, both external (statistical quality of sensory input) and internal (physiological limitations on neural signals). Efficient use of limited resources requires that the brain match its processing strategies to the statistical structure of input signals. We test this idea in the context of motion estimation in the fly visual system, where the optics of the compound eye and signals and noise in the retina are well understood and we can record from output neurons that encode velocity estimates. Still missing is a full characterization of the statistical relation between visual signals and motions relevant for flies in a natural context. Therefore, we develop a specialized camera mimicking fly eye optics, with inertial motion sensors providing ground truth about motions. We describe the design, construction, and performance characteristics of this FlEye camera. From camera data sampled in nature we construct optimal local motion estimators. These estimators show characteristic biases that are also observed in flies and other biological systems. Physical limitations of the camera data and computational estimator are negligible compared to biological systems. That we nevertheless observe similar biases suggests that biological performance is effectively limited by external statistics, not physiology.

View figure in article

Physics Subject Headings (PhySH)

Article Text

References (84)

  1. H. von Helmholtz, Treatise on Physiological Optics, trans. J. P. C. Southall from the 3rd German ed. (Optical Society of America, Washington, DC, 1925).
  2. C. E. Shannon, A mathematical theory of communication, Bell Syst. Tech. J. 27, 379 (1948).
  3. N. Wiener, Extrapolation, Interpolation, and Smoothing of Stationary Time Series (John Wiley and Sons, New York, 1949).
  4. J. L. Lawson and G. E. Uhlenbeck, Threshold Signals, Massachusetts Institute of Technology Radiation Laboratory Series Vol. 24 (McGraw-Hill, New York, 1950).
  5. P. Grassberger and J.-P. Nadal, editors, From Statistical Physics to Statistical Inference and Back (Kluwer, Dordrecht, 1994).
  6. H. B. Barlow, Sensory Mechanisms, the reduction of redundancy, and intelligence, in National Physical Laboratory Symposium on the Mechanization of Thought Processes (HM Stationery Office, London, 1959), Vol. 2, pp. 537–574.
  7. W. Bialek, Biophysics: Searching for principles (Princeton University Press, Princeton, NJ, 2012).
  8. W. Bialek, in Les Houches Summer School Lecture Notes: Theoretical Biological Physics 2023, SciPost Physics Lecture Notes, edited by A.-F. Bitbol, T. Mora, I. Nemenman, and A. M. Walczak (SciPost, Amsterdam, 2024).
  9. S. B. Laughlin, A simple coding procedure enhances a neuron's information capacity, Z. Naturforsch. 36, 910 (1981).
  10. J. H. van Hateren, Real and optimal neural images in early vision, Nature (London) 360, 68 (1992).
  11. F. Rieke, D. Warland, R. de Ruyter van Steveninck, and W. Bialek, Spikes: Exploring the Neural Code (MIT Press, Cambridge MA, 1997).
  12. N. Brenner, W. Bialek, and R. de Ruyter van Steveninck, Adaptive Rescaling Maximizes Information Transmission, Neuron 26, 695 (2000).
  13. D. A. Clark and J. E. Fitzgerald, Optimization in visual motion estimation, Annu. Rev. Vision Sci. 10, 23 (2024).
  14. R. Zbikowski, Sensor-rich feedback control, IEEE Instrum. Meas. Mag. 7, 19 (2004).
  15. W. Dickson, A. Straw, C. Poelma, and M. Dickinson, An integrative model of insect flight control, in 44th AIAA Aerospace Sciences Meeting and Exhibit (American Institute of Aeronautics and Astronautics, Nevada,2006), p. 34.
  16. G. K. Taylor and H. G. Krapp, Sensory systems and flight stability: What do insects measure and why? Adv. Insect Physiol. 34, 231 (2007).
  17. J. C. Tuthill and R. I. Wilson, Mechanosensation and adaptive motor control in insects, Curr. Biol. 26, R1022 (2016).
  18. F.-O. Lehmann and J. Bartussek, Neural control and precision of flight muscle activation in Drosophila, J. Comp. Physiol. A 203, 1 (2017).
  19. B. H. Dickerson, Timing precision in fly flight control: Integrating mechanosensory input with muscle physiology, Proc. R. Soc. B 287, 20201774 (2020).
  20. W. B. Dickson, A. D. Straw, and M. H. Dickinson, Integrative model of drosophila flight, AIAA J. 46, 2150 (2008).
  21. S. C. Whitehead, S. Leone, T. Lindsay, M. R. Meiselman, N. J. Cowan, M. H. Dickinson, N. Yapici, D. L. Stern, T. Shirangi, and I. Cohen, Neuromuscular embodiment of feedback control elements in Drosophila flight, Sci. Adv. 8, eabo7461 (2022).
  22. R. R. de Ruyter van Steveninck and S. B. Laughlin, Light adaptation and reliability in blowfly photoreceptors, Int. J. Neural Syst. 07, 437 (1996).
  23. D. Cao, A. J. Zele, and J. Pokorny, Linking impulse response functions to reaction time: Rod and cone reaction time data and a computational model, Vision Res. 47, 1060 (2007).
  24. D. G. Stavenga, Angular and spectral sensitivity of fly photoreceptors, II. Dependence on facet lens F-number and rhabdomere type in Drosophila, J. Comp. Physiol. A 189, 189 (2003).
  25. H. B. Barlow, The size of ommatidia in apposition eyes, J. Exp. Biol. 29, 667 (1952).
  26. D. G. Stavenga, Angular and spectral sensitivity of fly photoreceptors. I. Integrated facet lens and rhabdomere optics, J. Comp. Physiol. A 189, 1 (2003).
  27. D. G. Stavenga, Angular and spectral sensitivity of fly photoreceptors. III. Dependence on the pupil mechanism in the blowfly Calliphora, J. Comp. Physiol. A 190, 115 (2004).
  28. P. G. Lillywhite, Single photon signals and transduction in an insect eye, J. Comp. Physiol. A 122, 189 (1977).
  29. R. R. de Ruyter van Steveninck and S. B. Laughlin, The rate of information transfer at graded-potential synapses, Nature (London) 379, 642 (1996).
  30. R. Pierantoni, A look into the cock-pit of the fly, Cell Tissue Res. 171, 101 (1976).
  31. K. Hausen, in Photoreception and Vision in Invertebrates, edited by M. A. Ali (Springer, Boston, MA, 1984), pp. 523–559.
  32. H. G. Krapp, Estimation of self-motion for gaze and flight stabilization in flying insects, Navigation 55, 147 (2008).
  33. S. J. Huston and H. G. Krapp, Visuomotor transformation in the fly gaze stabilization system, PLoS Biol. 6, e173 (2008).
  34. T. W. Ullrich, R. Kern, and M. Egelhaaf, Influence of environmental information in natural scenes and the effects of motion adaptation on a fly motion-sensitive neuron during simulated flight, Biol. Open 4, 13 (2015).
  35. R. D. Williams, T. L. Massey, and M. M. Maharbiz, Blowfly yaw control via electrical stimulation of the H1 lobula plate tangential cell, in 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (IEEE, Honolulu, HI, USA, 2018), pp. 1685–1688.
  36. H. Wei, H. Y. Kyung, P. J. Kim, and C. Desplan, The diversity of lobula plate tangential cells (LPTCs) in the Drosophila motion vision system, J. Comp. Physiol. A 206, 139 (2020).
  37. S. Dorkenwald, A. Matsliah, A. R. Sterling, P. Schlegel, S.-C. Yu, C. E. McKellar, A. Lin, M. Costa, K. Eichler, Y. Yin, W. Silversmith, C. Schneider-Mizell, C. S. Jordan, D. Brittain, A. Halageri, K. Kuehner, O. Ogedengbe, R. Morey, J. Gager, K. Kruk, et al., Neuronal wiring diagram of an adult brain, Nature (London) 634, 124 (2024).
  38. R. de Ruyter van Steveninck and W. Bialek, Timing and counting precision in the blowfly visual system, in Models of Neural Networks IV: Early Vision and Attention (Springer-Verlag, Berlin, 2002), pp. 313–365.
  39. I. Nemenman, G. D. Lewen, W. Bialek, and R. R. de Ruyter van Steveninck, Neural coding of natural stimuli: Information at sub-millisecond resolution, PLoS Comput. Biol. 4, e1000025 (2008).
  40. M. S. Drews, A. Leonhardt, N. Pirogova, F. G. Richter, A. Schuetzenberger, L. Braun, E. Serbe, and A. Borst, Dynamic signal compression for robust motion vision in flies, Curr. Biol. 30, 209 (2020).
  41. A. L. Fairhall, G. D. Lewen, W. Bialek, and R. de Ruyter van Steveninck, Efficiency and ambiguity in an adaptive neural code, Nature (London) 412, 787 (2001).
  42. W. Bialek, F. Rieke, R. R. de Ruyter van Steveninck, and D. Warland, Reading a neural code, Science 252, 1854 (1991).
  43. R. de Ruyter van Steveninck and W. Bialek, Reliability and statistical efficiency of a blowfly movement-sensitive neuron, Philos. Trans. R. Soc. London 348, 321 (1995).
  44. S. R. Sinha, W. Bialek, and R. R. de Ruyter van Steveninck, Optimal local estimates of visual motion in a natural environment, Phys. Rev. Lett. 126, 018101 (2021).
  45. J. V. Hateren and C. Schilstra, Blowfly flight and optic flow: II. Head movements during flight, J. Exp. Biol. 202, 1491 (1999).
  46. J. A. Bender and M. H. Dickinson, A comparison of visual and haltere-mediated feedback in the control of body saccades in Drosophila melanogaster, J. Exp. Biol. 209, 4597 (2006).
  47. A. J. Kim, J. K. Fitzgerald, and G. Maimon, Cellular evidence for efference copy in Drosophila visuomotor processing, Nat. Neurosci. 18, 1247 (2015).
  48. S. Merrifield and L. Wilkinson, SimpleSDHC, https://github.com/ibm2030/SimpleSDHC (2000).
  49. D. Beersma, D. Stavenga, and J. Kuiper, Retinal lattice, visual field and binocularities in flies, J. Comp. Physiol. 119, 207 (1977).
  50. J. G. J. Smakman, J. H. van Hateren, and D. G. Stavenga, Angular sensitivity of blowfly photoreceptors: Intracellular measurements and wave-optical predictions, J. Comp. Physiol. A 155, 239 (1984).
  51. C. J. Edelson, MCRayTracing, https://github.com/edelsonc/MCRayTracing (2024).
  52. J. Limb and J. Murphy, Estimating the velocity of moving images in television signals, Comput. Graphics Image Process. 4, 311 (1975).
  53. W. Reichardt, Autocorrelation, a principle for evaluation of sensory information by the central nervous system, in Symposium on Principles of Sensory Communication 1959 (MIT Press, Cambridge MA, 1961), pp. 303–317.
  54. E. H. Adelson and J. R. Bergen, Spatiotemporal energy models for the perception of motion, J. Opt. Soc. Am. A 2, 284 (1985).
  55. W. Reichardt and T. Poggio, Visual control of orientation behaviour in the fly: Part I. A quantitative analysis, Q. Rev. Biophys. 9, 311 (1976).
  56. M. Egelhaaf, A. Borst, and W. Reichardt, Computational structure of a biological motion-detection system as revealed by local detector analysis in the fly's nervous system, J. Opt. Soc. Am. A 6, 1070 (1989).
  57. R. de Ruyter van Steveninck, W. Bialek, M. Potters, and R. Carlson, Statistical adaptation and optimal estimation in movement computation by the blowfly visual system, in Proceedings of IEEE International Conference on Systems, Man and Cybernetics (IEEE, 1994), Vol. 1, pp. 302–307.
  58. J. P. Van Santen and G. Sperling, Temporal covariance model of human motion perception, J. Opt. Soc. Am. A 1, 451 (1984).
  59. M. Potters and W. Bialek, Statistical mechanics and visual signal processing, J. Phys. I 4, 1755 (1994).
  60. Y. Weiss, E. P. Simoncelli, and E. H. Adelson, Motion illusions as optimal percepts, Nat. Neurosci. 5, 598 (2002).
  61. A. A. Stocker and E. P. Simoncelli, Noise characteristics and prior expectations in human visual speed perception, Nat. Neurosci. 9, 578 (2006).
  62. D. C. Knill and W. Richards, Perception as Bayesian Inference (Cambridge University Press, Cambridge, 1996).
  63. C. J. Edelson, FlEye reader, https://github.com/edelsonc/FlEye_Reader (2024).
  64. R. R. de Ruyter van Steveninck, W. Bialek, M. Potters, R. H. Carlson, and G. D. Lewen, Adaptive movement computation by the blowfly visual system, in Natural and Artificial Parallel Computation, Proceedings of the 5th NEC Research Symposium, edited by D. L. Waltz (SIAM, 1996), pp. 21–41.
  65. S. Roy, S. R. Sinha, and R. R de Ruyter van Steveninck, Encoding of yaw in the presence of distractor motion: Studies in a fly motion sensitive neuron, J. Neurosci. 35, 6481 (2015).
  66. M. Wertheimer, On perceived motion and figural organization (MIT Press, 2012).
  67. B. Hassenstein and W. Reichardt, Systemtheoretische analyse der Zeit-, Reihenfolgen- und Vorzeichenauswertung bei der Bewegungsperzeption des Rüsselkäfers chlorophanus, Z. Naturforsch. B 11, 513 (1956).
  68. D. L. Ruderman and W. Bialek, Statistics of natural images: Scaling in the woods, Phys. Rev. Lett. 73, 814 (1994).
  69. D. L. Ruderman, Origins of scaling in natural images, Vision Res. 37, 3385 (1997).
  70. v. A. Van der Schaaf and J. v. van Hateren, Modelling the power spectra of natural images: Statistics and information, Vision Res. 36, 2759 (1996).
  71. N. Brady and D. J. Field, Local contrast in natural images: Normalisation and coding efficiency, Perception 29, 1041 (2000).
  72. R. M. Balboa, C. W. Tyler, and N. M. Grzywacz, Occlusions contribute to scaling in natural images, Vision Res. 41, 955 (2001).
  73. R. M. Balboa and N. M. Grzywacz, Power spectra and distribution of contrasts of natural images from different habitats, Vision Res. 43, 2527 (2003).
  74. J. M. Salisbury and S. E. Palmer, Optimal prediction in the retina and natural motion statistics, J. Stat. Phys. 162, 1309 (2016).
  75. T. Poggio and W. Reichardt, Considerations on models of movement detection, Kybernetik 13, 223 (1973).
  76. J. E. Fitzgerald and D. A. Clark, Nonlinear circuits for naturalistic visual motion estimation, eLife 4, e09123 (2015).
  77. Q. Hu and J. D. Victor, A set of high-order spatiotemporal stimuli that elicit motion and reverse-phi percepts, J. Vision 10, 9 (2010).
  78. D. A. Clark, J. E. Fitzgerald, J. M. Ales, D. M. Gohl, M. A. Silies, A. M. Norcia, and T. R. Clandinin, Flies and humans share a motion estimation strategy that exploits natural scene statistics, Nat. Neurosci. 17, 296 (2014).
  79. T. Yildizoglu, C. Riegler, J. E. Fitzgerald, and R. Portugues, A neural representation of naturalistic motion-guided behavior in the Zebrafish brain, Curr. Biol. 30, 2321 (2020).
  80. R. C. Hardie, Functional organization of the fly retina, in Progress in Sensory Physiology (Springer, Berlin Heidelberg, 1985), pp. 1–79.
  81. E. Salcedo, A. Huber, S. Henrich, L. V. Chadwell, W.-H. Chou, R. Paulsen, and S. G. Britt, Blue- and Green-Absorbing visual pigments of Drosophila: Ectopic expression and physiological characterization of the R8 photoreceptor cell-specific Rh5 and Rh6 rhodopsins, J. Neurosci. 19, 10716 (1999).
  82. C. R. Sharkey, J. Blanco, M. M. Leibowitz, D. Pinto-Benito, and T. J. Wardill, The spectral sensitivity of Drosophila photoreceptors, Sci. Rep. 10, 18242 (2020).
  83. A. D. Buffry, J. P. Currea, F. A. Franke-Gerth, R. Palavalli-Nettimi, A. J. Bodey, C. Rau, N. Samadi, S. J. Gstöhl, C. M. Schlepütz, A. P. McGregor, et al., Evolution of compound eye morphology underlies differences in vision between closely related Drosophila species, BMC Biol. 22, 67 (2024).
  84. J. C. Lagarias, J. A. Reeds, M. H. Wright, and P. E. Wright, Convergence properties of the nelder–mead simplex method in low dimensions, SIAM J. Optim. 9, 112 (1998).

Outline

Information

Phys. Rev. E 113, 044412– Published 14 April, 2026
  • Received 20 June 2025
  • Accepted 4 March 2026
Article part of CHORUS
Crossmark - Check for updates button

©2026 American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation