Seeking Collaboration: ESP32 + FastAPI for Real-Time Location & Audio Threat Detection Prototype

Projects Interfacing w/ Software on the Computer
1

Hello everyone,
I am looking for someone who can design the hardware part for a alert system based on GPS and Audio input.
Currently I have created the backend service using FastAPI and mobile app using flutter to track user location and detect audio threats. I have assembled esp32, i2s audio input and gps sim module and put them together on the breadboard to test it). You can check my project here GitHub - Kashi7/shastraa -> GitHub project link

Project Overview:

  • ESP32 Tasks:
    • Fetch GPS coordinates and send them to FastAPI
    • Capture audio and send .wav files for processing
    • Use API key authentication for secure communication
  • Backend (FastAPI):
    • Stores user paths and real-time locations
    • Processes audio using an ML model for scream detection
    • Transcribes and analyzes audio for threats using DeepSeek
    • Raises an SOS alert if:
      • Deviation > 500m from saved path
      • Audio threat score exceeds threshold

What I Need Help With:

  • Reliable GPS Updates: How to optimize ESP32’s GPS polling frequency to balance accuracy and power consumption?
  • Efficient Audio Transmission: Best method to send compressed audio without losing key details for threat detection?
  • Power Optimization: How to reduce power consumption while maintaining periodic location and audio updates? I am looking for someone who can collaborate with me on this project and design a proper working prototype! :rocket: (kashisethia23@gmail.com)

Note: This is a passion project, and I am looking for like-minded individuals who are interested in collaborating. This is not a paid opportunity, but a chance to work on an impactful real-world problem together.

Thanks in advance! :raised_hands:

2

How does the 'alert' from the remote tracker get received ?

Would this be a commercial product ?

3

how does the ESP connects to the internet to send the GPS coordinates and send the .wav files ?
How does the Arduino knows the path ?

I understand you are not looking for feedback and more for collaboration but may be my 2 cents worth of thoughts if I had to tackle this would be of interest?

This sounds to me more like an opportunity for a smartPhone development where people already have the device in their pocket, you have all the APIs for location change detection at cheap battery impact, 3G to 5G communication, access to microphone and audio streaming APIs, multi-tasking etc and even on device ML capabilities with a trained model. (Some Apple Watch could also be doing all this - maybe not the the on device ML)

Of course there are restrictions for running some tasks in the background that would need to be explored as they could run against privacy protection measures (esp. in iOS).

Even if later on this turns into a dedicated pocket device, I would develop the PoC on a smartphone.

1 Like
4

To summarise, you are asking someone to design a power efficient battery powered voice radio that also sends GPS coordinates.

You say collaboration, however collaboration usually implies some flexibility in design choices, but you have already decided lots of things right down to a specific microcontroller without giving any technical reasons, or even saying if this is for cellular, WiFi, or something else. As other have said, why not just use a cheap mobile phone and write a custom app?

5

So the project seems to require in summary;

To be a portable device.
Have a microphone.
Have a GPS.
Presumably a battery that can be charged.
Some form of almost universal and reliable comms in cities.
Fast comms for streaming the recorded audio.
Uploadable from 'somewhere' with a track or path.

Sounds like a mobile phone app maybe ?

6

First I am thinking of completing this project, I have build the alert threat detection system (scream detection using machine learning and audio intent detection using llm) and gps based alert tracking. You can checkout details here GitHub - Kashi7/shastraa

7

I haven't started on hardware part because I myself is from software background. Looking for people who can collaborate doing the hardware part for it. You can check my GitHub repo GitHub - Kashi7/shastraa

8

Yeah developed the mobile app. Looking for someone who can do PCB design and create hardware components for it. Here is my code GitHub - Kashi7/shastraa

9

So you have part of the backend and none of the front end… as you are a Software developer, you really should explore developing for a smartphone.

You are building some sort of safety system, it would be terrible if the client system would be crappy.

Developing from scratch something that has

  • internet access through cellular with good antennas
  • Rechargeable battery with good autonomy
  • Embedded gps With motion tracking
  • audio system for good sound recording
  • a suitable OS with multitasking and energy efficiency
  • full reliability and portable form factor

Is years of development for a solid team.

Cheap android phones offer all this and you can probably hack Android to reach & meet your needs at a fraction of the cost and time.

1 Like
10

Is your goal to fund the back end service by selling a propietary gadget?

Why not just sell a subscription service instead and let the users mobile phone do the job that the gadget would be doing?

11

Will you liability insurance policy also cover your collaborator?

12

Not only is there is no need for the ESP32 component, adding it would make the system more complex, more expensive, significantly less reliable and much more difficult to maintain.

13

since OP said

I think we've shared enough of our thoughts.

Whilst I don't think it makes any technical sense (and is doomed from then get go to consider an ESP32 for this) I'm all in favor of anyone with a passion to explore, invent and push boundaries and prove me wrong.

So Good luck with your project @kashi07