Building a Production-Style AI System: Designing VibeFit from Image Upload to Intelligent Recommendations
Live Demo → Experience the application.
Architecture Walkthrough → Understand how the system works.
Source Code → Explore the implementation.
Project Deep Dive → Read the complete architecture and engineering decisions.
I thought building an AI stylist would be about prompts.
It wasn't.
When I started VibeFit, I assumed most of my time would go into prompt engineering.
Instead, I spent far more time designing workflows, validating AI responses, building retrieval pipelines and thinking about system reliability.
That completely changed the way I think about AI engineering.
Today, VibeFit is much more than an application that recommends outfits. It's an end-to-end AI system that understands a user's wardrobe, stores it as semantic knowledge and retrieves only the relevant clothing before asking an LLM to generate recommendations.
This article is a walkthrough of how I designed that system and the engineering decisions that shaped it.
What is VibeFit?
VibeFit is an AI-powered wardrobe assistant.
Users upload images of their clothing, and the system automatically understands each item using Gemini Vision. It extracts structured metadata such as category, color, fabric, season and occasion, generates semantic embeddings and stores everything in PostgreSQL using pgvector.
Later, when a user asks something like:
"What should I wear for a client meeting?"
the application doesn't send the entire wardrobe to Gemini.
Instead, it retrieves only the most relevant garments using semantic search and lets Gemini reason over that much smaller context.
That one architectural decision significantly improved recommendation quality while reducing hallucinations and token usage.
System Architecture
The application consists of two independent AI pipelines.
1. Ingestion Pipeline
Runs once for every uploaded clothing item.
The goal of this pipeline is simple.
Convert an uploaded image into structured, searchable knowledge.
2. Retrieval Pipeline
Runs whenever a recommendation is requested.
Instead of asking the LLM to search hundreds of clothing items, the retrieval system performs that work first.
Gemini only focuses on reasoning.
Engineering Decisions That Made the Biggest Difference Migrating from n8n to LangGraph
The first version of VibeFit was built in n8n.
It helped me validate the product quickly, but as the workflow became more complex, I started needing shared state, retries and conditional routing.
That's when I migrated the orchestration to LangGraph.
Instead of designing connected nodes, I started designing state transitions. Every node became responsible for a single task, while the graph handled execution.
It made the workflow much easier to maintain and extend.
Structured Outputs Are Only Half the Story
One lesson I learned early is that valid JSON doesn't always mean valid data.
Gemini generates structured metadata using a predefined schema, but that alone isn't enough.
After parsing the response, I validate it against my own business rules before storing anything in the database.
The AI generates data.
The application decides whether that data can be trusted.
Why I Didn't Generate Embeddings from Images
Most people assume image search starts with image embeddings.
In VibeFit, I take a different approach.
Gemini first converts the image into structured metadata. That metadata is transformed into a descriptive sentence, and only then do I generate embeddings.
Users search with intent, not pixels.
Representing the clothing semantically produced much better retrieval quality than relying only on visual similarity.
PostgreSQL + pgvector Instead of Pinecone
Introducing another database would have added complexity without solving a real problem.
Every clothing item already had relational data like category, image URL and ownership. Keeping metadata and embeddings together inside PostgreSQL made the architecture simpler while still providing fast semantic search.
For the current scale, pgvector was the right tradeoff.
What This Project Taught Me
Before building VibeFit, I thought AI engineering was mostly about choosing models and writing prompts.
Today, I think it's mostly software engineering.
The quality of the system depends on validation, retrieval, orchestration and reliability far more than it depends on clever prompts.
A good AI application isn't the one with the smartest model.
It's the one that gives the model exactly the information it needs and nothing more.
That shift in thinking changed the way I approach every AI project.
What's Next?
There are several improvements I'd like to make as the project evolves.
Introduce a message queue for asynchronous processing. Add hybrid search and reranking. Version prompts and evaluate retrieval quality. Improve observability with tracing and workflow metrics. Introduce confidence scoring and recommendation validation.
The current architecture solves today's problems, but it's designed in a way that can evolve as the application grows.
Resources
🚀 Live Demo: https://vibe-fit-blond.vercel.app/
💻 GitHub: https://github.com/Ankita-Nandkumar-Patil/vibefit
🎥 Loom Walkthrough: https://www.loom.com/share/552e8099b6594e9685ce440ce8194865
🌐 Detailed Architecture: https://ankitapatil.dev/project-listing/vibe-fit
Final Thoughts
Building VibeFit taught me something I didn't expect.
The LLM is probably the least interesting part of the application.
The interesting engineering happens everywhere around it.
Designing reliable workflows, validating probabilistic outputs, building semantic retrieval and making different components work together is what turns an AI demo into an AI system.
That's the part of AI engineering I enjoy the most.
