Big RAG Plugin for LM Studio
A powerful RAG (Retrieval-Augmented Generation) plugin for LM Studio that can index and search through gigabytes or even terabytes (not tested) of document data. Hosted here: github.com/ari99/lm_studio_big_rag_plugin
Big RAG Plugin for LM Studio
A powerful RAG (Retrieval-Augmented Generation) plugin for LM Studio that can index and search through gigabytes or even terabytes (not tested) of document data. Hosted here: github.com/ari99/lm_studio_big_rag_plugin
Big RAG Plugin for LM Studio
A powerful RAG (Retrieval-Augmented Generation) plugin for LM Studio that can index and search through gigabytes or even terabytes (not tested) of document data. Hosted here: github.com/ari99/lm_studio_big_rag_plugin
Big RAG Plugin for LM Studio
A powerful RAG (Retrieval-Augmented Generation) plugin for LM Studio that can index and search through gigabytes or even terabytes (not tested) of document data. Hosted here: github.com/ari99/lm_studio_big_rag_plugin
Big RAG is an LM Studio plugin for local retrieval-augmented generation over large document collections. It scans a configured directory, parses supported files, chunks text, embeds the chunks with an LM Studio embedding model, stores vectors in a sharded Vectra index, and injects relevant passages into chat prompts.
Features
Recursive directory scanning with optional filename exclude patterns.
Sharded Vectra vector storage for large local indexes.
Incremental indexing based on file hashes.
Batched parallel chunk embedding for faster indexing.
Character or sentence-boundary chunking.
README
Big RAG Plugin for LM Studio
Big RAG is an LM Studio plugin for local retrieval-augmented generation over large document collections. It scans a configured directory, parses supported files, chunks text, embeds the chunks with an LM Studio embedding model, stores vectors in a sharded Vectra index, and injects relevant passages into chat prompts.
Features
Recursive directory scanning with optional filename exclude patterns.
Sharded Vectra vector storage for large local indexes.
Incremental indexing based on file hashes.
Batched parallel chunk embedding for faster indexing.
Prompt preprocessor integration with custom prompt templates and citations.
Embedding model manifest checks to prevent retrieval from incompatible indexes.
Improvements (v1.1.0)
Chunker fixed: chunkSize and chunkOverlap are now correctly interpreted as tokens, not words. Chunks match the configured size.
Sentence-boundary chunking: new "Chunking Strategy" setting for higher-quality semantically coherent chunks (slower, better retrieval).
Parallel chunk embedding: chunks are embedded in batches of 8, significantly reducing indexing time for large documents.
VectorStore stats cache: getStats() and hasFile() now use cached or in-memory lookups after load, reducing redundant disk reads per chat turn.
Min chunk length filter: short/noise chunks are discarded before embedding (configurable, default: 20 tokens).
New file formats: .docx, .odt, .csv, .tsv, .json, .jsonl, .yaml, .yml, .rst, and .log are now indexed.
Config change detection: changing documentsDirectory, vectorStoreDirectory, or embeddingModel in settings resets the in-memory store and sanity-check cache.
Max query length: very long user messages are truncated before embedding to match embedding model input limits (configurable, default: 512 tokens).
cd big-rag-pluginnpm installnpm run buildnpm run dev
npm run build expects LM Studio's generated .lmstudio/entry.ts to exist. If you are working from a raw source checkout without that file, npx tsc still validates TypeScript compilation.
Configuration
Required Settings
Documents Directory: root directory containing files to index.
Vector Store Directory: directory where the sharded vector store and metadata are written.
Embedding
Embedding Model: LM Studio embedding model id. Use one spelling consistently for indexing and retrieval. If you change this value, run a full reindex because vectors from different models are not comparable.
Retrieval Settings
Retrieval Limit: maximum number of passages returned for each query. Default: 5.
Max Query Length: estimated-token limit for the user query before embedding. Longer queries are truncated. Default: 512.
Chunking Settings
Chunk Size: target chunk size in estimated tokens. Default: 512.
Chunk Overlap: overlap between consecutive chunks in estimated tokens. Default: 100.
Chunking Strategy: character uses fixed character windows and is fastest. sentence groups complete sentences for better semantic coherence. Default: character.
Min Chunk Length: chunks shorter than this estimated-token count are discarded before embedding. Set to 0 to keep all chunks. Default: 20.
Indexing Settings
Max Concurrent Files: number of files processed at once. Default: 1.
Parser Delay (ms): delay before parsing each file to reduce local service pressure. Default: 500.
Enable OCR: enables OCR for images and image-based PDFs. Default: true.
Exclude Filename Patterns: one glob per line, matched against paths relative to Documents Directory.
Reindexing Controls
Manual Reindex Trigger: turn on and send a chat message to run indexing.
Skip Previously Indexed Files: when manual reindex is on, skip unchanged files if enabled; rebuild all files if disabled.
Automatic First Run: if the vector store is empty, the first chat message triggers indexing automatically.
Prompt preprocessor integration with custom prompt templates and citations.
Embedding model manifest checks to prevent retrieval from incompatible indexes.
Improvements (v1.1.0)
Chunker fixed: chunkSize and chunkOverlap are now correctly interpreted as tokens, not words. Chunks match the configured size.
Sentence-boundary chunking: new "Chunking Strategy" setting for higher-quality semantically coherent chunks (slower, better retrieval).
Parallel chunk embedding: chunks are embedded in batches of 8, significantly reducing indexing time for large documents.
VectorStore stats cache: getStats() and hasFile() now use cached or in-memory lookups after load, reducing redundant disk reads per chat turn.
Min chunk length filter: short/noise chunks are discarded before embedding (configurable, default: 20 tokens).
New file formats: .docx, .odt, .csv, .tsv, .json, .jsonl, .yaml, .yml, .rst, and .log are now indexed.
Config change detection: changing documentsDirectory, vectorStoreDirectory, or embeddingModel in settings resets the in-memory store and sanity-check cache.
Max query length: very long user messages are truncated before embedding to match embedding model input limits (configurable, default: 512 tokens).
cd big-rag-pluginnpm installnpm run buildnpm run dev
npm run build expects LM Studio's generated .lmstudio/entry.ts to exist. If you are working from a raw source checkout without that file, npx tsc still validates TypeScript compilation.
Configuration
Required Settings
Documents Directory: root directory containing files to index.
Vector Store Directory: directory where the sharded vector store and metadata are written.
Embedding
Embedding Model: LM Studio embedding model id. Use one spelling consistently for indexing and retrieval. If you change this value, run a full reindex because vectors from different models are not comparable.
Retrieval Settings
Retrieval Limit: maximum number of passages returned for each query. Default: 5.
Max Query Length: estimated-token limit for the user query before embedding. Longer queries are truncated. Default: 512.
Chunking Settings
Chunk Size: target chunk size in estimated tokens. Default: 512.
Chunk Overlap: overlap between consecutive chunks in estimated tokens. Default: 100.
Chunking Strategy: character uses fixed character windows and is fastest. sentence groups complete sentences for better semantic coherence. Default: character.
Min Chunk Length: chunks shorter than this estimated-token count are discarded before embedding. Set to 0 to keep all chunks. Default: 20.
Indexing Settings
Max Concurrent Files: number of files processed at once. Default: 1.
Parser Delay (ms): delay before parsing each file to reduce local service pressure. Default: 500.
Enable OCR: enables OCR for images and image-based PDFs. Default: true.
Exclude Filename Patterns: one glob per line, matched against paths relative to Documents Directory.
Reindexing Controls
Manual Reindex Trigger: turn on and send a chat message to run indexing.
Skip Previously Indexed Files: when manual reindex is on, skip unchanged files if enabled; rebuild all files if disabled.
Automatic First Run: if the vector store is empty, the first chat message triggers indexing automatically.