SmolVLM Flutter App

Project Overview

SmolVLM Flutter App is a real-time, offline AI camera assistant built using Flutter and LLaMA.cpp. It captures live camera frames and sends them to a locally hosted LLaMA multimodal server running SmolVLM-500M, which responds with intelligent, natural language descriptions of the scene.

This project demonstrates how AI and computer vision can run directly on-device, making it useful for accessibility, education, smart agriculture, and robotics — all without relying on cloud services.

Key Features:

• 📷 Captures images using front/back camera with seamless switching
• 🔄 Sends frames to the AI server every few seconds
• 🧠 Generates real-time smart feedback using SmolVLM-500M model
• 🛠️ Fully offline setup — no internet required
• 🧼 Handles UI layout and preview stretching for clean UX

Technologies Used:

Flutter, Dart, Camera plugin, SmolVLM-500M-Instruct-f16.gguf (model), llama-server from LLaMA.cpp, Base64 communication

How It Works:

1. Flutter app captures camera frames every few seconds.
2. Encodes the frame as a Base64 image.
3. Sends the image to a locally running LLaMA server.
4. Server uses SmolVLM to generate a description.
5. App displays the AI-generated feedback in real-time.

Use Cases:

• 🔍 Accessibility for visually impaired users
• 📚 Educational tools for visual recognition
• 🛠️ Real-time debugging or documentation assistant
• 🌾 Smart farming applications (object/plant recognition)
• 🤖 Robotics vision and autonomy support

📸 Demo

🎥 Project Demo Video

🤝 Contributions

Open to contributions, feature suggestions, or bug reports! Feel free to fork the repo, open issues, or connect on LinkedIn.