Dr. Shahriar Nirjon is part of a project to develop an intelligent wearable system to detect imminent danger when users are distracted by smartphone use. Columbia University (PIs Jiang and Kinget) will be developing the hardware. UNC Chapel Hill (PI Nirjon) will be developing the algorithms, apply A.I., and develop the smartphone application. Barnard College (PI New) will be doing human factors study.
Abstract (from the National Science Foundation)
Using smartphones while walking poses an increasingly common safety problem for people in urban environments. Whether listening to music, texting, or talking, pedestrians that are absorbed with their smartphones are considerably less likely to notice important auditory cues of danger, such as the honks and sounds of approaching vehicles, putting pedestrians at far greater risk of being hit. This project aims to develop an intelligent wearable system that uses miniature microphones - embedded in earphones or headsets - to detect and locate approaching vehicles and warn the wearer of imminent dangers from cars, buses, motorbikes, trucks, and trams.
The system comprises multiple microphones embedded in a wearable headset, an ultra-low-power feature extraction and data processing pipeline, and a set of machine-learning classifiers running on a smartphone. This project is organized in four research thrusts: (1) designing an architecture and data processing pipeline for a wearable system composed of heterogeneous embedded modules; (2) devising an ultra-low-power, analog, signal-processing application-specific integrated circuit (ASIC) for energy-efficient, on-board feature extraction; (3) modeling and optimizing machine-learning classifiers for acoustic event detection and localization; and (4) designing an interface and feedback mechanisms that are optimized for the users' perceptual, cognitive, and motor control abilities.
This research will help reduce pedestrian injuries and fatalities, and expand knowledge on designing wearable systems for enhancing safety in cities, workplaces, and the home. The underlying framework can be generalized to other systems - employing low-power signal processors and algorithms to solve real-time sensing and classification problems of many kinds. Research products will be made publicly available to anyone to apply these techniques in their own system designs. Course modules developed on embedded systems, mobile computing, and Internet-of-Things will be used at the three participating universities to train undergraduate and graduate students, and will be made available online.
