Sentinel AI is an assistive navigation system that uses multimodal AI to guide visually impaired users.
The application relies on a YOLO model to continuously process video frames from the edge hardware.
It calculates an object’s relative size and trajectory to determine if it is in the safe walking corridor.
When a hazard is detected, a state graph managed by LangGraph triggers the reasoning layer.
The system passes the specific image frame alongside the edge bounding box metadata to an LLM.
A Google Generative AI model then generates a strict two-to-four word spoken audio command.
The command explicitly indicates the severity of the hazard to ensure the user understands the risk.
This urgent spatial audio warning is delivered instantly through an offline text-to-speech engine.
To prevent API spam, the vision loop implements a temporary pause after triggering the reasoning agent.
The entire architecture efficiently balances fast local object perception with complex scene understanding.