As whales face increasing threats from ship strikes, fishing net entanglements, and changing prey distributions due to rising ocean temperatures, tracking their populations and locations has become crucial. In response, second-year Chinmay Govind and third-year Nihar Ballamudi have dedicated their summer to an impactful project through the Penn Undergraduate Research Mentoring Program (PURM). This research merges mathematics, signal processing, animal behavior, and machine learning to enhance whale conservation efforts globally.
The duo’s primary aim is to harness whale sound data and artificial intelligence to not only map the locations of whales but also estimate their populations in specific areas. By leveraging advanced technologies, they hope to contribute crucial data that can inform policymakers and conservationists about whale populations and distribution.
Utilizing data from the National Oceanic and Atmospheric Administration (NOAA) sound receivers situated north of Cape Cod Bay, their research holds applicability for whale populations in any region. “The findings of our research can extend to not just whales, but also other marine life,” explains Govind, who is majoring in both artificial intelligence and computer engineering at the School of Engineering and Applied Science. Originally from Mechanicsburg, Pennsylvania, he believes that better data can significantly influence environmental policies.
PURM, which is organized by the Center for Undergraduate Research & Fellowships, provides students in their early years at Penn with a 10-week immersive research experience under the guidance of an experienced faculty mentor. In this case, Govind and Ballamudi are mentored by John Spiesberger, a distinguished visiting scholar from the Department of Earth & Environmental Science, alongside his son, Ari Spiesberger, a recent Penn graduate skilled in machine learning models.
The project’s multidisciplinary nature—combining elements of mathematics and environmental science—drew Govind and Ballamudi in. “Math research isn’t often linked to real-world applications,” notes Ballamudi, a mathematics major in the College of Arts & Sciences, who also minors in computer science. “Being able to influence policy through our math findings is incredibly exciting.”
Leveraging Sound to Locate Whales
In their PURM project, each student has taken on distinct roles: Govind’s focus is on locating whales, while Ballamudi is dedicated to estimating their populations. Locating whales involves pinpointing individual whales’ position, whereas censusing refers to estimating the size and distribution of whale populations.
To track individual whales, Govind has utilized acoustic data from NOAA receivers, essentially underwater microphones designed to capture whale calls. By analyzing the recorded sounds, he employs a machine learning model that determines the “time difference of arrival” (TDOA) of the whale calls, a method akin to GPS location tracking.
“Time difference provides data on specific curves for sound,” explains Govind. “With multiple receivers—typically five—we can gather enough information to pinpoint the whale’s coordinates or create a confidence interval for its location.” However, various challenges can obscure this process, such as ocean noise, which often distorts audio signals and hampers clear detections. The phenomenon known as “multipath,” where sound bounces off underwater surfaces, adds further complexity to the analysis.
“This data can be quite messy, which is why we’ve integrated machine learning,” Govind elaborates. By refining the acoustic data, he achieves a precise estimate of whale call origin points within a “median error of 20 milliseconds”—a level of accuracy that is more than satisfactory for whale location estimations.
Building a Census of Whale Populations
Meanwhile, Ballamudi focuses on utilizing machine learning models combined with NOAA sound data to simulate ocean environments and estimate whale populations. This innovative approach proves particularly useful as it can more effectively contend with the challenges posed by background ocean noise than the physical sources alone.
“We sample real ocean noise and generate signals based on established research about typical whale calls,” Ballamudi states. This allows them to produce unlimited data and deepen their understanding of both individual and collective whale behaviors.
During their work, Ballamudi has accurately predicted whale numbers and distributions with a remarkable success rate of 90-95%. This high degree of accuracy is particularly significant given the inherent difficulties involved in estimating whale populations.
Deep Dives in Mentorship
Throughout their research journey, Spiesberger has been instrumental in guiding Govind and Ballamudi. He emphasizes the importance of creating realistic simulations that reflect true ocean conditions to generate applicable data. Additionally, he coaches them on communicating their findings effectively to non-expert audiences, focusing on simplifying scientific jargon.
“We practice summarizing our work for a general audience in a matter of minutes,” Spiesberger explains, highlighting the importance of clear communication in positioning their findings within broader environmental policy contexts.
Ari Spiesberger also played a critical role as a mentor, teaching the students how to optimize AI models and refine their simulations to predict unknown variables. This collaborative atmosphere enhances the depth and applicability of their work.
The machine learning models employed in the PURM project not only address current challenges but also show potential for continuous improvement in precision, paving the way for future research endeavors.
As part of their PURM experience, Chinmay Govind and Nihar Ballamudi contributed to a broader research initiative at Penn, led by visiting scholar John Spiesberger, that focuses on locating and censusing whales. [From left to right: Christian Stuit; Nihar Ballamudi; Katherine Zhang; Sydney Fitzgerald; Justin Duong; Chinmay Govind; John Spiesberger; Mason Liu]
(Image: Courtesy of John Spiesberger)
“It would be incredibly beneficial to develop a model capable of recognizing multiple whale calls simultaneously,” Govind contemplates, noting that current limitations only allow for tracking individual whales at a time. Once they achieve the ability to record precise numbers of whales, Ballamudi envisions leveraging that data to retroactively pinpoint each whale’s specific location.
“Our ultimate goal is to test whether this approach is robust across various conditions, not solely in controlled environments but also in the more unpredictable ocean settings,” Ballamudi emphasizes, showcasing their determination to broaden the applications of their research.
The Tangible Impact of PURM
As the summer research period drew to a close, Spiesberger urged Govind and Ballamudi to present their findings to U.S. Navy sponsors—highlighting the implications for marine policy regarding whale conservation and potential avenues for future research expansions.
“Understanding whale sounds is crucial for the Navy, and our work through PURM addresses that requirement,” Spiesberger remarked. He aims to secure grants to facilitate the continuation of their impactful research beyond the summer, further enhancing conservation efforts.