الجامعة التقنية الجنوبية

1. Problem Statement Manual industrial gas leak inspection exposes human operators to severe health risks in hazardous environments. Existing static sensor networks fail to pinpoint precise leak locations, while traditional robotic systems struggle with dynamic obstacles and high-latency telemetry. 2. Proposed Solution / Innovation An autonomous AI-driven robotic system that localizes gas leaks in real-time. Our core innovation is a "headless" execution mode that disables visual rendering to accelerate reinforcement learning (Q-Learning) training speeds by over 10x, ensuring rapid environmental adaptation. 3. Method / Implementation The agent processes sensory inputs through a persistent Q-table to determine optimal navigation paths. We utilize a decoupled architecture where an AI engine handles the logic, while a Flask-based backend asynchronously streams telemetry data to a live web dashboard. 4. Key Components: HW/SW/AIAI/SW: Python-based Q-Learning engine and Pygame physics simulation. Sensory Framework: 2-sensor array calculating live concentration gradients (∇C). Backend/UI: Flask micro-framework and HTML5/JS frontend for real-time visualization. System Utilities: Automated background modules for crash-resilient data logging. 5. Expected Results / Impact The system achieves precise gas source localization in complex layouts. It drastically minimizes human occupational exposure to toxic environments and provides a scalable, low-latency monitoring solution for industrial safety. 6. What the 60s video demonstrates The video demonstrates the agent’s autonomous navigation in a complex environment, showing how it successfully reaches the gas source while bypassing obstacles. It highlights the real-time synchronization between the AI simulation and the web dashboard, where performance metrics and concentration gradients are streamed and updated instantly without any latency.

1. Introduction and Project Overview "Zero" is an advanced multimodal autonomous robotic assistant designed to bridge the gap between conversational Artificial Intelligence and physical real-world interactions. The primary objective of this project is to develop a cognitive robotic system capable of understanding natural human language—including complex dialects such as Iraqi Arabic—recognizing targeted objects in a dynamic environment, and executing complex physical fetch-and-deliver tasks. By synergizing cloud-based Large Language Models (LLMs) with Edge AI processing, Zero transitions from a passive voice assistant to an active physical agent. 2. System Architecture and Hardware Topology To ensure system stability and real-time responsiveness, the hardware architecture follows a strict Master-Slave paradigm: * The Master Node (Cognition & Perception): A Raspberry Pi 4 operates as the central processing hub. It hosts the operating system, handles heavy AI computations (vision and auditory pipelines), and manages the high-level decision-making process. * The Slave Nodes (Actuation & Control): Real-time hardware actuation is offloaded to two discrete ESP32 microcontrollers connected via asynchronous Serial communication: * ESP32 #1 (Locomotion): Controls a 4-wheel drive system via high-power motor drivers, executing velocity and trajectory commands. * ESP32 #2 (Manipulation): Manages a 6-Degree-of-Freedom (6-DOF) robotic arm, computing Inverse Kinematics (IK) and smooth motor interpolations utilizing a PCA9685 PWM controller via I2C. > Technical Note: This decoupled architecture guarantees that the heavy processing of Computer Vision and NLP does not interrupt the microsecond-level precision required for smooth motor control. > 3. Human-Robot Interaction (HRI) and Auditory Pipeline The acoustic pipeline is engineered for continuous, hands-free operation through four integrated stages: * Wake-Word Detection: Continuous monitoring for the trigger phrase "Hi Zero" using Picovoice Porcupine, a lightweight on-device neural network. * Speech-to-Text (STT): High-fidelity transcription using OpenAI’s Whisper API, which is resilient to background noise and dialectic variations (including Iraqi Arabic). * Cognitive Reasoning (LLM): OpenAI’s GPT-4o acts as the reasoning engine. Through advanced prompt engineering, it is restricted to either conversational responses or structured JSON payloads (e.g., {"action": "fetch", "object": "item_name"}). * Text-to-Speech (TTS): Natural vocal feedback generated via Edge-TTS for low-latency acoustic responses. 4. Edge Computer Vision and Spatial Awareness Environmental perception is executed strictly at the Edge (on the Raspberry Pi) to minimize latency: * Vision Engine: Utilizes MediaPipe’s EfficientDet in a headless OpenCV environment to avoid thermal throttling. * Targeting: When a "fetch" command is issued, the Vision module scans the environment for the target object, translating pixel coordinates into navigation data for alignment and pathfinding. 5. Kinematics, Navigation, and Manipulation * Navigation: The robot employs Proportional Logic to center the detected object within the camera's field of view (X-axis) and calculates depth approximations (Y-axis) to autonomously approach the target. * Manipulation: The 6-DOF arm utilizes Inverse Kinematics (IK) firmware on the ESP32. It employs polynomial trajectory generation (easing functions) for human-like fluidity. An ultrasonic sensor is used for final collision avoidance and distance confirmation before initiating a grasp. 6. Control Strategy: Finite State Machine (FSM) Orchestrating asynchronous modules (Voice, Vision, Wheels, and Arm) is managed through a centralized Python-based asynchronous Finite State Machine (FSM). This prevents system deadlocks and ensures a smooth transition: IDLE → LISTENING → PROCESSING → SEARCHING → NAVIGATING → PICKING → RETURNING → DELIVERING The FSM covers edge cases, including timeouts and auto-recovery if an object is lost or a grasp fails. 7. Conclusion Project Zero successfully demonstrates a cutting-edge integration of Mechatronics, Edge AI, and generative Cloud Models. By implementing asynchronous programming paradigms and a robust micro-service hardware structure, the system stands as a reliable, fully functional prototype representing the next generation of assistive and service robotics in unstructured human environments.

تناول هذا المشروع مشكلةً جوهريةً في مجال الزراعة وإدارة الموارد المائية، حيث يواجه المزارعون صعوبة في تشخيص أمراض النباتات واكتشافها مبكراً نتيجة غياب المراقبة اللحظية للبيانات الحيوية، بالإضافة إلى أضرار الري العشوائي الذي يسقي جميع المحاصيل بكميات متساوية متجاهلاً عتبة الرطوبة الخاصة بكل نبتة، مما يؤدي إلى هدر المياه واختناق الجذور. ويستند الحل المقترح (منصة بستاني+) إلى تقنيات إنترنت الأشياء (IoT) والذكاء الاصطناعي، من خلال دمج متحكم دقيق (ESP32) وحساسات بيئية مع نظام ري آلي متصل سحابياً، ووحدة رؤية حاسوبية، بهدف توفير بيانات مراقبة شاملة ولحظية لجميع النباتات، وتقديم إرشاد زراعي رقمي متكامل يُمكّن المزارعين -حتى المبتدئين منهم- من العناية بمحاصيلهم باحترافية. وتشمل النتائج المتوقعة أتمتة عملية الري بالكامل وترشيد استهلاك المياه بناءً على الاحتياج الفعلي للنبات، والتشخيص الآلي للأمراض لاقتراح خطط العلاج المناسبة، بالإضافة إلى حماية معدات الضخ من التلف، مما يؤسس لبيئة زراعية ذكية ومستدامة.