Hacktoberfest Hack Day x AI for Vizag

A.E.G.I.S. - Autonomous Edge-Guided Interface Sentinel

A privacy-first, 100% offline standalone Android OS agent powered by on-device Google Gemma 4 and LiteRT-LM.

A.E.G.I.S.

Autonomous Edge-Guided Interface Sentinel

A Privacy-First, 100% Offline Standalone Android OS Agent powered by Google AI Edge Gemma 4 & LiteRT-LM.

🏛️ System Architecture

AEGIS transforms an Android smartphone into an autonomous, voice-driven personal sentinel operating entirely on-device without cloud dependence or external telemetry.

                    ┌──────────────────────────────┐
                    │          USER VOICE          │
                    │   "Gemma, open Settings..."  │
                    └──────────────┬───────────────┘
                                   │
                                   ▼
                    ┌──────────────────────────────┐
                    │      OFFLINE WAKE-WORD       │
                    │         (Vosk 16kHz)         │
                    └──────────────┬───────────────┘
                                   │
                                   ▼
                    ┌──────────────────────────────┐
                    │     ANDROID STT & INTENT     │
                    │       SpeechRecognizer       │
                    └──────────────┬───────────────┘
                                   │
                                   ▼
                    ┌──────────────────────────────┐
                    │     AGENT ORCHESTRATOR       │
                    │  Room DB Cache (SkillDao)    │
                    └───────┬──────────────┬───────┘
          [Cache Miss]      │              │  [Cache Hit]
                            ▼              └───────────────┐
             ┌─────────────────────────────┐               │
             │   GEMMA 4 LITERTLM ENGINE   │               │
             │  (On-Device GPU Inference)  │               │
             └──────────────┬──────────────┘               │
                            │                              │
                            ▼                              ▼
             ┌─────────────────────────────┐ ┌─────────────────────────────┐
             │       RISK ASSESSOR         │ │     ACCESSIBILITY SERVICE   │
             │   (Biometric HITL Gate)     │─┤   GestureDispatcher + Node  │
             └─────────────────────────────┘ └──────────────┬──────────────┘
                                                            │
                                                            ▼
                                             ┌─────────────────────────────┐
                                             │     TARGET APPLICATION      │
                                             │  (WhatsApp, Settings, etc.) │
                                             └─────────────────────────────┘

⚡ Key Capabilities

100% On-Device Local Brain: Powered by Google AI Edge LiteRT-LM (gemma-4-E2B-it.litertlm on GPU) running real-time ReAct loop and function calling.
Dual-Engine Voice Pipeline: Continuous offline 16kHz Vosk listener for “Gemma” wake-word with atomic hardware-level microphone handoff to Android native speech recognition.
Persistent Stark Floating HUD: Draggable cyan Arc-Reactor HUD (WindowManager.LayoutParams.TYPE_APPLICATION_OVERLAY) supported by a persistent foreground sentinel service.
Autonomous OS Navigation: Direct view hierarchy parsing (AccessibilityNodeInfo) and programmatic touch gesture dispatch (GestureDescription).
Episodic Skill Memory: Self-improving episodic memory layer powered by Room SQLite (SkillDao) that caches successful ReAct tool sequences for sub-second cached execution.
Biometric Human-In-The-Loop (HITL) Gate: Zero-trust risk classification (RiskAssessor) that transparently halts critical actions (deletions, payments, credential inputs) behind Android Biometric Prompt verification.
Active Communication & Call Engine: Native tools to send messages via WhatsApp or SMS (send_message), place direct calls or launch the dialer (call_handling), and inspect recent call logs.
Local Media & File Discovery: Local audio memo & call recording scanner (read_recordings) and fast filesystem indexer (search_local_files).
Hardware & Process Telemetry: Real-time memory, storage, and battery metrics (device_diagnostics).
Project S.T.A.R.K. / Obsidian Second Brain Synchronization: Exports all episodic facts, captured notifications, and learned ReAct skills into Obsidian-compliant Markdown with frontmatter and [[backlinks]] (stark_context_sync).
Passive Notification Context Capture: Background listener service buffering WhatsApp and Gmail notifications into local Room SQLite.
Robust Verification: 57 unit tests covering boundary and corner cases, empirical challenger scenarios, and prompt sliding window managers.

🛠️ Tech Stack

Platform: Android 14+ / 16 (Target SDK 35/36, ARM64-v8a)
LLM Engine: com.google.ai.edge.litertlm:litertlm-android:0.18.0 + Gemma 4
Voice Stack: Vosk 0.3.47 + Android SpeechRecognizer + TextToSpeech
UI: Jetpack Compose (Material3 Dark Strategist HUD) + WindowManager System Overlay
Local Persistence: Room SQLite + Flow
Security: AndroidX Biometric 1.2.0-alpha05
Work Scheduling: AndroidX WorkManager 2.9.1

Built with

gemmaandroidkotlinjetpack-composelitert-lmroomsqlitevoskbiometricworkmanager

AI tools

Coded withAntigravityGemini 3.8 Flash

Prize Categories Entered

Best Use of Gemma 4
Submitted
AEGIS uses Google's Gemma 4 on-device via LiteRT-LM (gemma-4-E2B-it.litertlm with GPU acceleration) as its central autonomous reasoning brain. The model drives an agentic ReAct loop that perceives Android UI hierarchy and dispatches accessibility touch actions and tools entirely offline with zero cloud latency and complete user privacy.
Best Open-Source AI Project
Submitted
AEGIS is an open-source, privacy-first standalone Android OS agent built entirely around open-weight models: Google's Gemma 4 running via LiteRT-LM on GPU and Vosk offline speech recognition for 16kHz wake-word detection. It features an open-source ReAct agent loop, Room DB episodic skill memory caching, and a biometric human-in-the-loop safety gate.