Project Overview
Objective
Unified text-to-video and text-to-image search into one cross-modal retrieval platform.
Stack
CLIP (ViT-B/32)FAISSFastAPIReact.jsTailwind CSS
Delivery highlights
- Extended and integrated previous projects (Textto-Video Semantic Search and Text-to-Image Semantic Search) into a unified multimodal semantic retrieval platform capable of searching across both videos and images using natural language queries. Leveraged CLIP (ViT-B/32) developed by OpenAI to generate shared embeddings for text, video keyframes, and images within the same vector space, enabling cross-modal semantic similarity search with FAISS and accurate timestamp alignment for video playback. Integrated language translation preprocessing (Thai → English) to improve embedding alignment and retrieval accuracy, as CLIP performs more effectively with English text inputs. Developed RESTful APIs using FastAPI that return structured JSON responses containing media_id, timestamp (for videos), similarity score, and media URL, and built a responsive frontend using React.js and Tailwind CSS for real-time result visualization.
Back to Topic ProjectsBack to All ProjectsSystem workflow
Cross-modal embedding alignment
Text, video keyframes, and images are encoded into a unified CLIP embedding space, enabling semantic search across all three modalities.Rendering workflow diagram…
Scroll horizontally to explore the full workflow on smaller screens.
System workflow
Hybrid search pipeline
User query is translated to English, encoded with CLIP, and matched against indexed media with timestamp alignment for videos.Rendering workflow diagram…
Scroll horizontally to explore the full workflow on smaller screens.
System workflow
Video indexing and timestamp tracking
Videos are segmented into keyframes, each encoded and linked with precise timestamps for accurate playback seeking.Rendering workflow diagram…
Scroll horizontally to explore the full workflow on smaller screens.
System workflow
Source-reviewed processing sequence
A conservative end-to-end boundary taken from the source audit. Specific endpoints, storage writes, and external calls are only claimed in the project-specific diagrams when the implementation confirms them.Rendering workflow diagram…
Scroll horizontally to explore the full workflow on smaller screens.