Seeing Small Things with a Few Pixels and a Small Brain

Dr. Luat Vuong, Assistant Professor, University of California Riverside

There are increasing demands for robust, rapid-response computer-vision systems; however, complex images are difficult to process in real-time, especially when subtle or small features are pertinent. The visual systems of flies may serve as a model for real-time computer vision systems, since flies are capable of filtering extraneous information amid variable background conditions. Here, we develop a reliable, high-speed image processing pipeline based on the fly visual response that involves optical preprocessing, sparse sampling, and feed-forward neural networks. We show that the optical encoding from corneal nanostructures could offer essential computing functions associated with enhanced visual acuity and polarization sensitivity. Until recently, there was virtually no work related to hypothetical optical preprocessing in fly eyes from corneal nanostructures. Instead, corneal nanostructures were associated with anti-glare and hydrophobic functions; head movements or microsaccades were associated with the optical acuity of insects. Experimentally, we develop a mostly-air corneal coating that enables imaging resolutions higher than the sampling spacing typically avails. Numerically, we connect the function of these coatings with simple machine-learning neural networks. Our work points to opportunities for drone cameras and the high-speed distillation of features for edge computing applications.