Researchers at New York University (NYU) are leading a research project to understand how noisy are outdoor environments in the city and train a computer algorithm to identify the sources of sound contributing to these environments. Each sensor you see is one of 50-100 sensors the Sounds Of New York City (SONYC) project has installed around NYC. The project involves researchers from NYU Center for Urban Science + Progress, NYU Steinhardt School of Culture, Education, and Human Development, NYU Tandon School of Engineering, and Ohio State University’s School of Engineering.

Why are we doing this research?

New York City is noisy. New Yorkers make more calls to the City’s non-emergency 311 line to complain about noise than they do for any other reason.

It has been estimated that 9 out of 10 adults in New York City are exposed to excessive noise levels, i.e. beyond the limit of what the U.S. EPA considers harmful. Such levels of exposure have proven effects on health including acute effects such as sleep disruption, increased stress, annoyance and distraction; and long-term effects such as hypertension, heart disease and hearing loss. In addition, there is evidence of impact on educational performance, with numerous studies showing that noise pollution produces learning and cognitive impairment in children, resulting in decreased memory, reading skills and lower test scores. The economic impact of noise-related health effects is also huge. Early studies in the US demonstrated the effect of environmental noise on real estate markets, with housing prices falling as much as 2% per dB of noise increase. Recent studies in Europe have identified similar trends.

In the near term, we believe the SONYC sensor network will help the city understand and control noise more effectively. In the long term, we believe the SONYC sensor network will improve our understanding of the adverse impacts that noise has on public health, the educational outcomes of school children and real-estate prices.

What are our sensors doing now?  

Our sensors rely upon a technique known as supervised machine learning. To train a computer to listen, we first need to have lots of sound recordings annotated by humans with all the classes of sound sources they hear (e.g., jackhammers, sirens, music, yelling, barking, etc.). We then use that database of annotated recordings to train our computers to recognize similar sounds. Our plan is to record brief snippets of sound at random intervals in time. The duration of a “snippet” will be approximately 10 seconds long, which is just long enough for people to correctly identify the sources of sound in it. Outside those snippets, the sensor’s sound recording capabilities will be disabled.

To train computers to perform at an acceptable level of accuracy, we aim to deploy 50-100 sensors and record at least four months of audio from each sensor. The recordings we annotate must be drawn from realistic auditory environments in which the sensor network will be deployed, otherwise the algorithm embedded in each sensor will not identify sources accurately in the operational phase. Our computer algorithms will not be able to identify sources that they have not been trained to recognize.

What will our sensors do in the future?

When fully developed and deployed in the operational phase, a SONYC sensor will listen continuously to ambient outdoor sounds, measure the total volume, and transmit a statistical description of the audio it hears, for example:

  • 55 decibels, footsteps, birds, wind;
  • 80 decibels, traffic;
  • 95 decibels, jackhammer;
  • 112 decibels, siren; and so on.

The mature sensor network will not store nor transmit the sounds it analyzes, and it will be impossible to reconstruct the original sounds from these statistics.

Will our sensors be able to record speech?  

Yes, our sensors may record speech, but SONYC is not interested in the content of that speech (words, phrases). Our only interest is training the sensor to identify a sound as “voice”, as opposed to other sources such as “car”, “dog” or “honk.”

In the interests of privacy, the SONYC project submitted sample recordings for review by independent acoustical consultants Cerami & Associates, who judged them to be unrecognizable as conversation. Nevertheless, the project will clearly post visible signs alerting passers-by of the recording device.

Will SONYC be able to identify individuals whose voices may be recorded?  

No, SONYC will not collect any additional information (e.g. photographs or video) that would permit the research team to identify particular individuals. With no additional correlative data to link recorded signals to particular individuals, any incidentally recorded speech will remain de-identified. SONYC will restrict access to the raw audio files to the SONYC research team. The mature sensor network will not record nor transmit the audio signal, and it will be impossible to reconstruct the original audio signal from its output.

What measures will SONYC take to prevent the dissemination of the audio recordings?

Access to the raw audio data captured during the training phase will be restricted. Access will be granted only to authorized members of the research team. Individuals annotating snippets will be given random snippets that are neither sequential in time nor in geographic location. Their only task will be to identify the class of sources (jackhammers, sirens, music, traffic, people, dogs, birds, etc.) they hear in each snippet. Annotators will never be asked to transcribe or further identify anything they hear. Archived raw audio files will be encrypted.

Has NYU’s Institutional Review Board approved this project?

Yes, NYU’s Institutional Review Board determined in February 2015 that SONYC’s proposed research with sensors mounted on the sides of NYU buildings is exempt from further human subjects’ protection scrutiny. NYU’s Institutional Review Board reviews all proposed research involving human subjects to ensure that the subjects’ rights and welfare are adequately protected in accordance with regulations issued by U.S. Department of Health and Human Services Office for Human Research Protections.

Who is funding this research?  

SONYC is supported by a seed grant by NYU’s Center for Urban Sound and Progress, followed by a grant of $4.6 million from the National Science Foundation this August. SONYC has not received any funding from the City of New York nor any of its agencies.

Who is leading this research? 

The Principal Investigators are NYU Professors Juan Pablo Bello, Luke DuBois, Oded Nov and Claudio Silva, and OSU Professor Anish Arora. More information on the research team is available at wp.nyu.edu/sonyc/people. If you have any questions, please contact us at: ask.sonyc@nyu.edu