🎙️🖼️ Multimodal AI Assistant

Voice, Vision, and Speech Reasoning with LLaVA-1.5-7B & Whisper

🚀 Model: LLaVA-1.5-7B (ZeroGPU) | 🎙️ Speech: Whisper (base)

📥 1. Upload Image

🖼️ Upload Image

📥 2. Select Question Method (Choose One)

Question Input Mode (Either Voice OR Text)

🎙️ Voice Input 💬 Text Prompt

🎙️ Record Voice or Upload Audio

💡 Tip for Mac users: If recording sounds silent, click the orange microphone icon in your Mac's top menu bar and change Mic Mode from Voice Isolation to Standard.

📤 Responses

📝 Whisper Speech-to-Text Transcription

🤖 LLaVA Visual Reasoning Response

🔊 Spoken Response (Text-to-Speech)

💡 How It Works

Whisper: High-accuracy voice transcription with silence detection.
LLaVA-1.5-7B: State-of-the-art vision-language model on ZeroGPU for visual reasoning.
gTTS: Converts responses into spoken audio with 1-click playback and download.