Building a RAG Pipeline with PyMuPDF and ChromaDB: Document Summarization and Q&A Without Frameworks

Scale AI has confirmed a significant investment from Meta, valuing the data-labeling startup at $29 billion. As part of the deal, Meta acquired a 49% stake in Scale AI for approximately $14.3 billion. The company also announced that its co-founder and CEO, Alexandr Wang, will step down to join Meta and lead its superintelligence initiatives. Jason Droege, currently Scale’s Chief Strategy Officer, will serve as interim CEO. Scale emphasized that it remains an independent company, with Wang continuing on the board as a director. The investment underscores Meta’s strategic push to strengthen its AI capabilities amid growing competition from OpenAI, Google, and Anthropic. Scale AI has become a critical partner in the AI supply chain, providing high-quality training data for large language models. Its role in labeling and processing both text and visual content has made it indispensable to frontier AI labs. In the SEAD-Agent implementation, Scale AI’s document processing pipeline begins with extracting text and images from PDFs using PyMuPDF. Each page is processed to extract raw text and identify embedded images. Text is split into chunks of up to 512 characters using a sentence-based splitting method to preserve meaning. Images are converted to base64-encoded strings and sent to a vision-language model (Pixtral 12B) to generate descriptive captions. Each text and image chunk is assigned a unique ID and stored as a DocumentChunk object. These chunks are then individually summarized using a structured prompt that guides the model to extract key points such as objectives, findings, methodology, implications, and limitations. The individual summaries are aggregated into a final document summary using either a brief or detailed prompt, depending on user preference. To enable retrieval-based generation, the system uses ChromaDB as a vector store. Text chunks are embedded locally using the SentenceTransformer all-MiniLM-L6-v2 model. The embeddings, along with metadata and IDs, are stored in a persistent ChromaDB collection. When a user asks a question, the query is embedded and searched against the vector store using semantic similarity. The top three most relevant chunks are retrieved based on distance scores and returned for context. Log outputs from the system show successful processing of 118 chunks from a research paper on urban sustainability. The pipeline accurately extracted text and image content, generated relevant captions, produced coherent chunk-level summaries, and delivered contextually accurate answers to queries about EcoSphere, a decision-support tool for carbon optimization in city planning. This end-to-end implementation demonstrates how RAG can be built without relying on high-level frameworks, using only PyMuPDF for document parsing, ChromaDB for vector storage, and a VLM for multimodal understanding. It highlights the importance of careful chunking, embedding, and prompt engineering in creating robust, scalable AI systems for document analysis and Q&A.