Project Overview
Objective
Built a Thai-capable multimodal RAG assistant for searching and asking source-grounded questions across documents, scanned pages, audio, and video.
Stack
FastAPINext.jsPyPDFLoaderSentenceTransformers (bge-m3)QdrantElasticsearchBLIPTyphoon OCRWhisperGPT-4o-miniGPT-4.1GPT-5
Delivery highlights
- Developed DocVisionAI as a full-stack multimodal knowledge assistant that accepts PDFs, scanned documents, audio, and video. Used PyMuPDF with Typhoon OCR fallback for document extraction, Whisper for timestamped transcription, and BLIP for embedded-image captioning. Indexed multilingual BAAI/bge-m3 embeddings in Qdrant and combined them with Elasticsearch keyword results for hybrid retrieval. Built background processing and persistent file metadata with FastAPI, plus a Next.js viewer that displays summaries, key points, and source citations linked to the relevant document page or media timestamp.
Back to Topic ProjectsBack to All ProjectsSystem workflow
How DocVisionAI processes and answers
From file ingestion and multimodal extraction to hybrid retrieval, grounded generation, and traceable citations.Rendering workflow diagram…
Scroll horizontally to explore the full workflow on smaller screens.
System workflow
Multimodal content extraction
Different extraction strategies for PDFs (PyMuPDF + OCR), audio (Whisper), video (Whisper), and images (BLIP captions) to create a unified text and metadata layer.Rendering workflow diagram…
Scroll horizontally to explore the full workflow on smaller screens.
System workflow
Hybrid retrieval system
Qdrant vector search combined with Elasticsearch keyword matching to retrieve relevant content from any media type.Rendering workflow diagram…
Scroll horizontally to explore the full workflow on smaller screens.
System workflow
Citation linking architecture
Metadata tracking ensures every answer is traceable back to the original document page or media timestamp for verification.Rendering workflow diagram…
Scroll horizontally to explore the full workflow on smaller screens.
System workflow
Source-reviewed processing sequence
A conservative end-to-end boundary taken from the source audit. Specific endpoints, storage writes, and external calls are only claimed in the project-specific diagrams when the implementation confirms them.Rendering workflow diagram…
Scroll horizontally to explore the full workflow on smaller screens.