DOCUMENTATION.md
DOCUMENTATION.md
Date: 2026-08-19
Version: v1.9.12
Status: ✅ Complete
pattern_scan tool + puppeteer connected property-read fix + dead-file removal; previous release v1.9.11 on 28.08, see CHANGELOG_v2.md)| Component | Status | Notes |
|---|---|---|
| Tool Count | ✅ 24 tool modules (131 unique tools, incl. v1.9.12 pattern_scan) | All registered via declarative pattern (v1.8.2+) |
| Context Management | ✅ Scoping + Heuristic Scoring + TTL Pruning | v1.9.1+ improvements active |
| Token Counting | ✅ Native History API × 0.24 ratio | Matches LM Studio sidebar <0.3% deviation |
| Graphify Intelligence Suite | ✅ Fully Implemented (v1.9.5) | Confidence tags, hub-exclusion clustering, project auto-detection, tier provenance, cluster-aware priority |
| Gateway Pattern | ⚠️ Abandoned (v1.8.0+) | Direct SDK registration + schema minification handles compatibility |
| Priority System | ❌ Removed (v1.6.4) | Replaced by toolsSchemaMinifier.ts description truncation |
The following tools ONLY accept HTTP/HTTPS URLs and will FAIL with "Only HTTP and HTTPS URLs are allowed" if given file:// paths:
searxng_batch_fetch(urls) — Batch fetch multiple REMOTE pages (HTTP/HTTPS only!)fetch_web_content(url) — Fetch single remote webpage (HTTP/HTTPS only!)searxng_search(query) — Web search ONLY, not local file access!read_file(file_name="CHANGELOG.md") — Read single local filefind_files(pattern=".md", max_depth=5) — Find files by name pattern, then read each result❌ WRONG (will fail): searxng_batch_fetch(urls=["file:///C:/path/file.md"])
✅ CORRECT: read_file(file_name="CHANGELOG.md", max_length=5000) for local files
See CRITICAL_TOOL_USAGE_RULES.md in project root for complete reference.
executedTool — post-v1.9.12 incident follow-up (2026-09-01)Every plain-object tool result now carries the ground truth of which registered implementation actually executed. Follow-up to the 2026-09-01 "silent tool substitution" incident: log forensics attributed it to model-side substitution + a transcript observability gap (NOT an ai_toolbox routing defect), and this fix closes that observability gap.
Hardened every web/RAG allocation path against host heap exhaustion and terminated the chunking loop that could spin forever on poison-length documents. Full test suite green (user-verified 25.08.2026 ~00:13); version stays at v1.9.10 — no bump.
node --max-old-space-size=2048 … --maxWorkers=1 cleannpm run build; bundles carry no version strings)AutoTracker now evaluates token thresholds against history count + running per-tool deltas during a tool loop (FIX #20), and [AutoTracker] [DELTA] log lines carry a live | chat used ≈ N tok estimate.
[TokenCheck] log Math.round fix (build#3) and en-US locale pins for model-facing strings (build#4, known-good install)Fixed the checkpoint warning that was generated but never surfaced in chat, and restored confirm-first working-directory switching with German JA/NEIN reply support.
Step 0.7 had been refactored to silently switch the working directory on project-keyword match — because every message mentions a registered-project keyword, it took an early-return path that buried the pending checkpoint warning (logs: "THRESHOLD PROMPT GENERATED", chat: nothing) and bypassed Step 0.6 reply handling. Reply detection accepted English YES/NO only; transitionTo() cleared pending warnings on any state change.
promptPreprocessor.ts): Step 0.7 injects a confirm-first "⚠️ REGISTERED PROJECT DETECTED" banner without changing CWD; the one-shot switch executes only after an explicit YES/JA reply in a later message, then resets.promptPreprocessor.ts): checkpoint reply detection accepts German JA/NEIN (normalized onto canonical YES/NO FSM inputs).autoTracker.ts): transitionTo() no longer clears pendingCheckpointWarning on unrelated state changes; the warning is injected into all preprocessor return paths while pending.Eliminated the "ai-toolbox not found" clarification loop by adding Step 0.7 project keyword detection in promptPreprocessor.ts and _syncFromSessionMemory() lazy registry sync.
When users mentioned a registered project name (e.g., "switch to ai-toolbox"), the AI would:
search_projects(query="ai-toolbox") → empty results (stale registry)Root Cause: The cross-project registry was never synced from session memory decisions. Projects detected via keyword matching in Step 0.7 were registered once but not auto-synced when search_projects was called later.
| Tool | Sync Trigger | Purpose |
|---|---|---|
search_projects | _syncFromSessionMemory() before query | Ensures registry includes projects from past decisions |
get_project_info | _syncFromSessionMemory() before lookup | Same — prevents stale registry entries |
register_project tool with explicitConfirmation=true still works as primary registration methodno-unsafe-assignment Hardening & Type-Safety Refinement — v1.9.3 (2026-08-09)Resolved unused eslint-disable directives and eliminated implicit any assignments in HTTP client tools through explicit type annotations.
Three architectural improvements to the memory system inspired by persistent-memory-v2 analysis — preserving ai-toolbox's performance advantages while adding context isolation and intelligent retrieval.
MemoryScope type)global/project/session scope types to all ContextEntry records'global' — session/project entries can now be filtered for isolation(RecencyDecay × 0.7) + (FrequencySaturation × 0.3)freq / (freq + 5))getRecentEntries() and searchContext() — results sorted by relevance score instead of insertion orderSESSION_TTL_MS = 86400000)pruneExpiredSessionEntries() runs automatically before every retrieval operationPerformance preserved: All improvements are deterministic (no AI inference, no WASM overhead) — retrieval latency remains <10ms.
Resolved critical token counting inaccuracy and missing checkpoint prompt injection issues.
Architectural overhaul of tool registration system.
TOOL_REGISTRIES) containing 20 entriesconfig, stateManager, and at definition time via arrow functionsFixed critical performance issue where grep_files searched ALL directories.
Resolved critical token undercounting bug.
ContextGuard.countTokens() now properly extracts text from arrays of content blocks.getText() method or .text property before JSON serialization@typescript-eslint/no-base-to-string error with explicit type checksThe priority system (maxToolsInSchema, tier-based filtering, toolPriorityOverrides) was removed in v1.6.4 because:
toolsSchemaMinifier.ts)Replacement: toolsSchemaMinifier.ts handles grammar parser compatibility via description truncation (~150 chars) and constraint capping. No manual limits needed.
The gateway pattern (src/tools/gatewayTools.ts) was introduced in v1.6.0, abandoned in favor of direct SDK registration (v1.8.0+), and the file itself has since been removed from the codebase (v1.9.10 session, 24.08 — no gateway definitions remain under src/).
The following corrections reflect the current v1.8.2 implementation:
| Category | Previous Count | Corrected Count | Changes |
|---|---|---|---|
| File System Tools | 21 → 22 | 22 tools | Added fuzzy_find_local_files (previously counted separately) |
| Web Research Tools | 4 | 4 tools | No change |
| Browser Automation Tools | 5 | 5 tools | No change |
| Git & GitHub Tools | 13 → 15 | 15 tools | Added git_stash, git_blame (v1.5.23) |
| Database Tools | 1 | 1 tool | No change |
| Document Parsing | 1 | 1 tool | No change |
| Background Commands | 3 | 3 tools | No change |
| Execution Tools | 4 → 5 | 5 tools |
The isSafeRegex() function in src/security.ts performs precise pattern analysis:
All dangerous tool categories are disabled by default:
| Category | Default State |
|---|---|
browserAutomation | false |
gitOperations | false |
databaseQueries | false |
executionJavaScript / executionPython | Enabled (sandboxed) |
executionTerminal / executionShell | false |
Debounced State Saves: _queueSave() in stateManager.ts coalesces rapid set/delete/clear calls within a 500ms window → single batched disk write instead of N individual writes (~90% I/O reduction during bulk ops).
| Cache | TTL / Window | Max Entries | Purpose |
|---|---|---|---|
State Key Cache (_keysCache) | 1s TTL + invalidate on mutation | N/A | O(1) getAllKeys() — eliminates disk reload during auto-tracker checks |
Size Estimation Cache (sizeValueCache) | Per-object, memoized JSON.stringify() | Unbounded | O(1) vs. O(n serialization) for repeated complex state values |
Project Path Cache (_projectPathCache) | 5s TTL with staleness check | N/A | Eliminates duplicate fs.stat() on getProjectMemoryFilePath() |
| Fuzzy Search Cache | 60s TTL + LRU eviction via Map order | 100 entries | File name similarity results; frequently queried paths stay cached |
AI_TOOLBOX_DEBUG unset): ~80% fewer console.warn() calls — threshold near-misses (~95%), state transitions, and buffer operations are suppressed.$env:AI_TOOLBOX_DEBUG="true" on Windows / export AI_TOOLBOX_DEBUG=true on Linux/macOS): Full diagnostic output for all auto-tracker checks, context guard token counting, compression steps, and file read operations.config.ts Zod schema exactlyindex.ts implementation| File | Changes Made |
|---|---|
README.md | Up-to-date (v1.9.x release history incl. v1.9.12; 131 unique tools across 24 modules) |
ARCHITECTURE.md | Gateway Pattern marked as ABANDONED; tool counts corrected to 20 modules |
TOOLS_REFERENCE.md | Up-to-date (~132 tools documented) |
DOCUMENTATION.md | Deprecated features clearly marked; tool count corrections applied |
CHANGELOG.md | Up-to-date (v1.8.0–v1.8.2 entries complete) |
CONTRIBUTING.md | Updated to show declarative registry pattern for adding new tools (v1.8.2+) |
SECURITY.md | Up-to-date (threat model, security controls) |
SUMMARY.md | Rebuilt with v1.8.2 status, deprecated features noted |
These documentation updates correspond to the following source code locations:
| Source File | Documentation Section | Verification Method |
|---|---|---|
src/config.ts | Configuration tables in README.md, ARCHITECTURE.md | Zod schema fields match documented settings exactly |
src/tools/*.ts (20 files) | Tool counts and descriptions in all MD files | Manual count of registered tools per category |
src/index.ts | Plugin lifecycle in ARCHITECTURE.md | Code flow matches documented initialization sequence |
src/security.ts | Security pipeline documentation | Validation functions match documented threat model |
src/toolsProvider.ts | Declarative registry pattern (v1.8.2+) | Closure-based registry with 20 entries, single for...of loop |
All changes verified with comprehensive test suite:
npx tsc --noEmit — 0 errors)npm run lint)npm run build)@/ → src/) in both tsconfig.json and docs: update documentation for v1.8.2 — declarative registry, deprecated features markednpm run testsrc/toolsProvider.ts.src/config.ts.src/security.ts and individual tool modules.Five major architectural improvements inspired by graphify repository analysis — confidence-tagged results, hub-exclusion clustering, project auto-detection, context tier provenance, and cluster-aware tool priority ranking.
src/types/confidenceTypes.ts)Typed confidence metadata for all tool execution outputs following graphify's confidence-tagging pattern.
Root Cause Addressed: Prior to this fix, LLMs had no way to distinguish between deterministic results and inferred outputs, leading to over-trusting of low-confidence semantic similarity scores or fallback-path results. This pattern follows graphify_integration_analysis.md Section 1 (Confidence-Tagged Results).
src/utils/hubExclusionClustering.ts)Louvain community detection with hub-exclusion for architectural transparency and refactoring guidance.
Root Cause Addressed: Prior to this feature, there was no systematic way to analyze module dependency structure. Hub-exclusion clustering enables architectural visibility with modularity scoring, cluster density metrics, and hub identification — all running synchronously under 10ms for typical plugin dependency graphs.
Verification: All 83 tests pass across clustering suites including graph construction from known edges (24 test cases), hub identification at various percentiles (10th, 50th, 80th, 95th), Louvain convergence on synthetic graphs (small/large/connected/disconnected), majority-vote reattachment correctness, cluster density and modularity calculations, edge case handling (empty graph, single node, isolated nodes).
src/projectAutoDetect.ts)Automatically detects and registers projects in the cross-project registry when searches return empty results.
searchWithAutoRegister() & initializeProjectDetection() (v1.9.8+)Both functions are deprecated as of v1.9.8+:
searchWithAutoRegister(): No longer called from any code path. Registration now requires explicit user confirmation via the register_project tool with confirmed path.initializeProjectDetection(): Removed from startup flow in index.ts. Added explanatory comment: "NO AUTO-REGISTRATION ON STARTUP". Projects must be registered explicitly.promptPreprocessor.ts) + Registry Sync (_syncFromSessionMemory())This two-layer approach eliminates the clarification loop:
_syncFromSessionMemory(): Lazy auto-sync from session memory when search_projects or get_project_info is called — ensures registered projects are always up-to-date without explicit user confirmationRoot Cause Addressed: Prior to this fix, the "ai-toolbox not found" issue occurred when cross-project registry searches returned empty results — no auto-discovery mechanism existed. User-mentioned project names were not used as registration signals, causing failed lookups even when the project was clearly present in CWD.
src/contextTiers.ts)Typed provenance markers for tier-scoped context replacement following graphify's build_merge pattern.
Root Cause Addressed: Prior to this feature, context updates replaced entire node sets without tracking data origin. This caused silent overwrites of unchanged tiers (e.g., AST nodes replaced even when only semantic insights changed). The tier-provenance system follows graphify_integration_analysis.md Section 2 (Context Tier Provenance) to enable incremental, lossless context updates.
src/tools/toolPriority.ts)Five-tier priority ranking with hub-exclusion clustering integration for intelligent tool filtering.
Root Cause Addressed: Prior to this feature, all enabled tools were sent to the LLM without priority ordering. When tool count exceeded grammar parser limits (llama.cpp EBNF recursion), there was no intelligent way to decide which tools to prune — alphabetical sorting was arbitrary and could exclude critical file system tools while keeping low-usage backup tools. The cluster-aware priority system ensures architecturally important modules (high centrality) are retained first.
Impact:
Eliminated all synchronous file writes; introduced shared crash-resilient atomic write utility with randomized temp filenames and rollback-on-failure protection.
atomicWrite Utility (src/utils/atomicWrite.ts)crypto.randomBytes(9) (72-bit entropy) — prevents collisions, survives process crashesatomicWriteBinaryFile() with raw buffer writesAll previously synchronous file-write tools converted to async with shared atomicWrite:
| Module | Tools Affected | Write Pattern |
|---|---|---|
lineOperations.ts | delete_lines, line_operations | async → atomicWrite |
refactorCodeTools.ts | rename_identifier, move_function, extract_function, unused_import_cleanup | async → atomicWrite + rollback-on-failure |
utilityTools.ts | ~25 tools (backup, chart, etc.) | All async → atomicWrite |
dataVisualizationTools.ts | generate_chart | async → atomicWriteBinaryFile |
imageProcessingTools.ts | describe_image, compare_images saves | async → atomicWriteBinaryFile |
markdownPreviewTools.ts | markdown_preview HTML save | async → atomicWrite |
browserAutomationTools.ts | screenshot_desktop PNG save | async → atomicWriteBinaryFile |
uiGenerationTools.ts | UI component saves | async → atomicWrite |
recodeEngine.ts (recodeTool/) | AST transformation output | async → atomicWrite + rollback-on-failure |
refactorCodeTools & recodeEngineSource code protection — failed AST transformations automatically restore original file from .bak backup before returning error.
New aggregator modules, file modification tracking, protocol warnings documentation, and image analysis tool type-safety fixes.
src/tools/executionRegistry.ts)Consolidated execution tools into a single registration function to reduce import count in toolsProvider.ts.
Root Cause Addressed: Prior to this module, each execution tool was imported and filtered individually in toolsProvider.ts, creating ~50 lines of repetitive if (config.X || godMode) blocks. The aggregator pattern eliminates duplication while preserving per-tool gating.
src/tools/fileModTracker.ts)Tracks consecutive file modifications within a session to warn LLM about stale line numbers.
Map<string, FileModEntry> — increments counter on repeated operationsRoot Cause Addressed: When the LLM rapidly calls multiple tools on the same file, each tool reads from disk independently and operates correctly in isolation. The corruption happens because the LLM's context contains STALE line numbers that don't account for previous operations' effects. This tracker provides explicit guidance to switch strategies after repeated modifications.
src/tools/toolProtocolWarnings.ts)Critical protocol restrictions documentation for preventing file:// → HTTP tool misuse.
Root Cause Addressed: The searxng_* tools are external LM Studio system tools with descriptions that cannot be modified in this codebase. This module documents the restrictions here and enforces them via tool selection guidelines, preventing the error where local file paths were incorrectly passed to HTTP/HTTPS-only web fetching tools.
src/tools/utilityRegistry.ts)Consolidates multiple small utility modules into a single registration function.
registerUtilityTools() combines backup, cleanup-backups, data-visualization, line-operations, and markdown-preview toolstoolsProvider.ts replaces 5 separate importsPluginConfig for individual tool gatingRoot Cause Addressed: Similar to executionRegistry — reduces toolsProvider.ts import count and centralizes utility tool registration logic. Follows the same aggregator pattern established in v1.8.2 declarative registry.
src/utils/simulation.ts)Comprehensive test harness for Graphify-inspired architectural analysis features.
analyzeAiToolboxDependencies() runs clustering on actual 24-module source graph (50+ connections) — verifies hub identification, cluster formation, and quality metricsRoot Cause Addressed: Prior to this simulation, hub-exclusion clustering features had no systematic test harness. The simulation validates all 83 clustering tests plus additional integration scenarios (ToolPriority centrality scores, ContextGuard file cluster info) — ensuring architectural analysis tools produce correct results before deployment.
src/tools/imageAnalysisTools.ts)Resolved TypeScript compilation errors and ESLint warnings through ESM conversion and proper type assertions.
Impact:
imageAnalysisTools.tsDate: 2026-08-19
Version: v1.9.12
Status: ✅ Complete
pattern_scan tool + puppeteer connected property-read fix + dead-file removal; previous release v1.9.11 on 28.08, see CHANGELOG_v2.md)| Component | Status | Notes |
|---|---|---|
| Tool Count | ✅ 24 tool modules (131 unique tools, incl. v1.9.12 pattern_scan) | All registered via declarative pattern (v1.8.2+) |
| Context Management | ✅ Scoping + Heuristic Scoring + TTL Pruning | v1.9.1+ improvements active |
| Token Counting | ✅ Native History API × 0.24 ratio | Matches LM Studio sidebar <0.3% deviation |
| Graphify Intelligence Suite | ✅ Fully Implemented (v1.9.5) | Confidence tags, hub-exclusion clustering, project auto-detection, tier provenance, cluster-aware priority |
| Gateway Pattern | ⚠️ Abandoned (v1.8.0+) | Direct SDK registration + schema minification handles compatibility |
| Priority System | ❌ Removed (v1.6.4) | Replaced by toolsSchemaMinifier.ts description truncation |
The following tools ONLY accept HTTP/HTTPS URLs and will FAIL with "Only HTTP and HTTPS URLs are allowed" if given file:// paths:
searxng_batch_fetch(urls) — Batch fetch multiple REMOTE pages (HTTP/HTTPS only!)fetch_web_content(url) — Fetch single remote webpage (HTTP/HTTPS only!)searxng_search(query) — Web search ONLY, not local file access!read_file(file_name="CHANGELOG.md") — Read single local filefind_files(pattern=".md", max_depth=5) — Find files by name pattern, then read each result❌ WRONG (will fail): searxng_batch_fetch(urls=["file:///C:/path/file.md"])
✅ CORRECT: read_file(file_name="CHANGELOG.md", max_length=5000) for local files
See CRITICAL_TOOL_USAGE_RULES.md in project root for complete reference.
executedTool — post-v1.9.12 incident follow-up (2026-09-01)Every plain-object tool result now carries the ground truth of which registered implementation actually executed. Follow-up to the 2026-09-01 "silent tool substitution" incident: log forensics attributed it to model-side substitution + a transcript observability gap (NOT an ai_toolbox routing defect), and this fix closes that observability gap.
Hardened every web/RAG allocation path against host heap exhaustion and terminated the chunking loop that could spin forever on poison-length documents. Full test suite green (user-verified 25.08.2026 ~00:13); version stays at v1.9.10 — no bump.
node --max-old-space-size=2048 … --maxWorkers=1 cleannpm run build; bundles carry no version strings)AutoTracker now evaluates token thresholds against history count + running per-tool deltas during a tool loop (FIX #20), and [AutoTracker] [DELTA] log lines carry a live | chat used ≈ N tok estimate.
[TokenCheck] log Math.round fix (build#3) and en-US locale pins for model-facing strings (build#4, known-good install)Fixed the checkpoint warning that was generated but never surfaced in chat, and restored confirm-first working-directory switching with German JA/NEIN reply support.
Step 0.7 had been refactored to silently switch the working directory on project-keyword match — because every message mentions a registered-project keyword, it took an early-return path that buried the pending checkpoint warning (logs: "THRESHOLD PROMPT GENERATED", chat: nothing) and bypassed Step 0.6 reply handling. Reply detection accepted English YES/NO only; transitionTo() cleared pending warnings on any state change.
promptPreprocessor.ts): Step 0.7 injects a confirm-first "⚠️ REGISTERED PROJECT DETECTED" banner without changing CWD; the one-shot switch executes only after an explicit YES/JA reply in a later message, then resets.promptPreprocessor.ts): checkpoint reply detection accepts German JA/NEIN (normalized onto canonical YES/NO FSM inputs).autoTracker.ts): transitionTo() no longer clears pendingCheckpointWarning on unrelated state changes; the warning is injected into all preprocessor return paths while pending.Eliminated the "ai-toolbox not found" clarification loop by adding Step 0.7 project keyword detection in promptPreprocessor.ts and _syncFromSessionMemory() lazy registry sync.
When users mentioned a registered project name (e.g., "switch to ai-toolbox"), the AI would:
search_projects(query="ai-toolbox") → empty results (stale registry)Root Cause: The cross-project registry was never synced from session memory decisions. Projects detected via keyword matching in Step 0.7 were registered once but not auto-synced when search_projects was called later.
| Tool | Sync Trigger | Purpose |
|---|---|---|
search_projects | _syncFromSessionMemory() before query | Ensures registry includes projects from past decisions |
get_project_info | _syncFromSessionMemory() before lookup | Same — prevents stale registry entries |
register_project tool with explicitConfirmation=true still works as primary registration methodno-unsafe-assignment Hardening & Type-Safety Refinement — v1.9.3 (2026-08-09)Resolved unused eslint-disable directives and eliminated implicit any assignments in HTTP client tools through explicit type annotations.
Three architectural improvements to the memory system inspired by persistent-memory-v2 analysis — preserving ai-toolbox's performance advantages while adding context isolation and intelligent retrieval.
MemoryScope type)global/project/session scope types to all ContextEntry records'global' — session/project entries can now be filtered for isolation(RecencyDecay × 0.7) + (FrequencySaturation × 0.3)freq / (freq + 5))getRecentEntries() and searchContext() — results sorted by relevance score instead of insertion orderSESSION_TTL_MS = 86400000)pruneExpiredSessionEntries() runs automatically before every retrieval operationPerformance preserved: All improvements are deterministic (no AI inference, no WASM overhead) — retrieval latency remains <10ms.
Resolved critical token counting inaccuracy and missing checkpoint prompt injection issues.
Architectural overhaul of tool registration system.
TOOL_REGISTRIES) containing 20 entriesconfig, stateManager, and at definition time via arrow functionsFixed critical performance issue where grep_files searched ALL directories.
Resolved critical token undercounting bug.
ContextGuard.countTokens() now properly extracts text from arrays of content blocks.getText() method or .text property before JSON serialization@typescript-eslint/no-base-to-string error with explicit type checksThe priority system (maxToolsInSchema, tier-based filtering, toolPriorityOverrides) was removed in v1.6.4 because:
toolsSchemaMinifier.ts)Replacement: toolsSchemaMinifier.ts handles grammar parser compatibility via description truncation (~150 chars) and constraint capping. No manual limits needed.
The gateway pattern (src/tools/gatewayTools.ts) was introduced in v1.6.0, abandoned in favor of direct SDK registration (v1.8.0+), and the file itself has since been removed from the codebase (v1.9.10 session, 24.08 — no gateway definitions remain under src/).
The following corrections reflect the current v1.8.2 implementation:
| Category | Previous Count | Corrected Count | Changes |
|---|---|---|---|
| File System Tools | 21 → 22 | 22 tools | Added fuzzy_find_local_files (previously counted separately) |
| Web Research Tools | 4 | 4 tools | No change |
| Browser Automation Tools | 5 | 5 tools | No change |
| Git & GitHub Tools | 13 → 15 | 15 tools | Added git_stash, git_blame (v1.5.23) |
| Database Tools | 1 | 1 tool | No change |
| Document Parsing | 1 | 1 tool | No change |
| Background Commands | 3 | 3 tools | No change |
| Execution Tools | 4 → 5 | 5 tools |
The isSafeRegex() function in src/security.ts performs precise pattern analysis:
All dangerous tool categories are disabled by default:
| Category | Default State |
|---|---|
browserAutomation | false |
gitOperations | false |
databaseQueries | false |
executionJavaScript / executionPython | Enabled (sandboxed) |
executionTerminal / executionShell | false |
Debounced State Saves: _queueSave() in stateManager.ts coalesces rapid set/delete/clear calls within a 500ms window → single batched disk write instead of N individual writes (~90% I/O reduction during bulk ops).
| Cache | TTL / Window | Max Entries | Purpose |
|---|---|---|---|
State Key Cache (_keysCache) | 1s TTL + invalidate on mutation | N/A | O(1) getAllKeys() — eliminates disk reload during auto-tracker checks |
Size Estimation Cache (sizeValueCache) | Per-object, memoized JSON.stringify() | Unbounded | O(1) vs. O(n serialization) for repeated complex state values |
Project Path Cache (_projectPathCache) | 5s TTL with staleness check | N/A | Eliminates duplicate fs.stat() on getProjectMemoryFilePath() |
| Fuzzy Search Cache | 60s TTL + LRU eviction via Map order | 100 entries | File name similarity results; frequently queried paths stay cached |
AI_TOOLBOX_DEBUG unset): ~80% fewer console.warn() calls — threshold near-misses (~95%), state transitions, and buffer operations are suppressed.$env:AI_TOOLBOX_DEBUG="true" on Windows / export AI_TOOLBOX_DEBUG=true on Linux/macOS): Full diagnostic output for all auto-tracker checks, context guard token counting, compression steps, and file read operations.config.ts Zod schema exactlyindex.ts implementation| File | Changes Made |
|---|---|
README.md | Up-to-date (v1.9.x release history incl. v1.9.12; 131 unique tools across 24 modules) |
ARCHITECTURE.md | Gateway Pattern marked as ABANDONED; tool counts corrected to 20 modules |
TOOLS_REFERENCE.md | Up-to-date (~132 tools documented) |
DOCUMENTATION.md | Deprecated features clearly marked; tool count corrections applied |
CHANGELOG.md | Up-to-date (v1.8.0–v1.8.2 entries complete) |
CONTRIBUTING.md | Updated to show declarative registry pattern for adding new tools (v1.8.2+) |
SECURITY.md | Up-to-date (threat model, security controls) |
SUMMARY.md | Rebuilt with v1.8.2 status, deprecated features noted |
These documentation updates correspond to the following source code locations:
| Source File | Documentation Section | Verification Method |
|---|---|---|
src/config.ts | Configuration tables in README.md, ARCHITECTURE.md | Zod schema fields match documented settings exactly |
src/tools/*.ts (20 files) | Tool counts and descriptions in all MD files | Manual count of registered tools per category |
src/index.ts | Plugin lifecycle in ARCHITECTURE.md | Code flow matches documented initialization sequence |
src/security.ts | Security pipeline documentation | Validation functions match documented threat model |
src/toolsProvider.ts | Declarative registry pattern (v1.8.2+) | Closure-based registry with 20 entries, single for...of loop |
All changes verified with comprehensive test suite:
npx tsc --noEmit — 0 errors)npm run lint)npm run build)@/ → src/) in both tsconfig.json and docs: update documentation for v1.8.2 — declarative registry, deprecated features markednpm run testsrc/toolsProvider.ts.src/config.ts.src/security.ts and individual tool modules.Five major architectural improvements inspired by graphify repository analysis — confidence-tagged results, hub-exclusion clustering, project auto-detection, context tier provenance, and cluster-aware tool priority ranking.
src/types/confidenceTypes.ts)Typed confidence metadata for all tool execution outputs following graphify's confidence-tagging pattern.
Root Cause Addressed: Prior to this fix, LLMs had no way to distinguish between deterministic results and inferred outputs, leading to over-trusting of low-confidence semantic similarity scores or fallback-path results. This pattern follows graphify_integration_analysis.md Section 1 (Confidence-Tagged Results).
src/utils/hubExclusionClustering.ts)Louvain community detection with hub-exclusion for architectural transparency and refactoring guidance.
Root Cause Addressed: Prior to this feature, there was no systematic way to analyze module dependency structure. Hub-exclusion clustering enables architectural visibility with modularity scoring, cluster density metrics, and hub identification — all running synchronously under 10ms for typical plugin dependency graphs.
Verification: All 83 tests pass across clustering suites including graph construction from known edges (24 test cases), hub identification at various percentiles (10th, 50th, 80th, 95th), Louvain convergence on synthetic graphs (small/large/connected/disconnected), majority-vote reattachment correctness, cluster density and modularity calculations, edge case handling (empty graph, single node, isolated nodes).
src/projectAutoDetect.ts)Automatically detects and registers projects in the cross-project registry when searches return empty results.
searchWithAutoRegister() & initializeProjectDetection() (v1.9.8+)Both functions are deprecated as of v1.9.8+:
searchWithAutoRegister(): No longer called from any code path. Registration now requires explicit user confirmation via the register_project tool with confirmed path.initializeProjectDetection(): Removed from startup flow in index.ts. Added explanatory comment: "NO AUTO-REGISTRATION ON STARTUP". Projects must be registered explicitly.promptPreprocessor.ts) + Registry Sync (_syncFromSessionMemory())This two-layer approach eliminates the clarification loop:
_syncFromSessionMemory(): Lazy auto-sync from session memory when search_projects or get_project_info is called — ensures registered projects are always up-to-date without explicit user confirmationRoot Cause Addressed: Prior to this fix, the "ai-toolbox not found" issue occurred when cross-project registry searches returned empty results — no auto-discovery mechanism existed. User-mentioned project names were not used as registration signals, causing failed lookups even when the project was clearly present in CWD.
src/contextTiers.ts)Typed provenance markers for tier-scoped context replacement following graphify's build_merge pattern.
Root Cause Addressed: Prior to this feature, context updates replaced entire node sets without tracking data origin. This caused silent overwrites of unchanged tiers (e.g., AST nodes replaced even when only semantic insights changed). The tier-provenance system follows graphify_integration_analysis.md Section 2 (Context Tier Provenance) to enable incremental, lossless context updates.
src/tools/toolPriority.ts)Five-tier priority ranking with hub-exclusion clustering integration for intelligent tool filtering.
Root Cause Addressed: Prior to this feature, all enabled tools were sent to the LLM without priority ordering. When tool count exceeded grammar parser limits (llama.cpp EBNF recursion), there was no intelligent way to decide which tools to prune — alphabetical sorting was arbitrary and could exclude critical file system tools while keeping low-usage backup tools. The cluster-aware priority system ensures architecturally important modules (high centrality) are retained first.
Impact:
Eliminated all synchronous file writes; introduced shared crash-resilient atomic write utility with randomized temp filenames and rollback-on-failure protection.
atomicWrite Utility (src/utils/atomicWrite.ts)crypto.randomBytes(9) (72-bit entropy) — prevents collisions, survives process crashesatomicWriteBinaryFile() with raw buffer writesAll previously synchronous file-write tools converted to async with shared atomicWrite:
| Module | Tools Affected | Write Pattern |
|---|---|---|
lineOperations.ts | delete_lines, line_operations | async → atomicWrite |
refactorCodeTools.ts | rename_identifier, move_function, extract_function, unused_import_cleanup | async → atomicWrite + rollback-on-failure |
utilityTools.ts | ~25 tools (backup, chart, etc.) | All async → atomicWrite |
dataVisualizationTools.ts | generate_chart | async → atomicWriteBinaryFile |
imageProcessingTools.ts | describe_image, compare_images saves | async → atomicWriteBinaryFile |
markdownPreviewTools.ts | markdown_preview HTML save | async → atomicWrite |
browserAutomationTools.ts | screenshot_desktop PNG save | async → atomicWriteBinaryFile |
uiGenerationTools.ts | UI component saves | async → atomicWrite |
recodeEngine.ts (recodeTool/) | AST transformation output | async → atomicWrite + rollback-on-failure |
refactorCodeTools & recodeEngineSource code protection — failed AST transformations automatically restore original file from .bak backup before returning error.
New aggregator modules, file modification tracking, protocol warnings documentation, and image analysis tool type-safety fixes.
src/tools/executionRegistry.ts)Consolidated execution tools into a single registration function to reduce import count in toolsProvider.ts.
Root Cause Addressed: Prior to this module, each execution tool was imported and filtered individually in toolsProvider.ts, creating ~50 lines of repetitive if (config.X || godMode) blocks. The aggregator pattern eliminates duplication while preserving per-tool gating.
src/tools/fileModTracker.ts)Tracks consecutive file modifications within a session to warn LLM about stale line numbers.
Map<string, FileModEntry> — increments counter on repeated operationsRoot Cause Addressed: When the LLM rapidly calls multiple tools on the same file, each tool reads from disk independently and operates correctly in isolation. The corruption happens because the LLM's context contains STALE line numbers that don't account for previous operations' effects. This tracker provides explicit guidance to switch strategies after repeated modifications.
src/tools/toolProtocolWarnings.ts)Critical protocol restrictions documentation for preventing file:// → HTTP tool misuse.
Root Cause Addressed: The searxng_* tools are external LM Studio system tools with descriptions that cannot be modified in this codebase. This module documents the restrictions here and enforces them via tool selection guidelines, preventing the error where local file paths were incorrectly passed to HTTP/HTTPS-only web fetching tools.
src/tools/utilityRegistry.ts)Consolidates multiple small utility modules into a single registration function.
registerUtilityTools() combines backup, cleanup-backups, data-visualization, line-operations, and markdown-preview toolstoolsProvider.ts replaces 5 separate importsPluginConfig for individual tool gatingRoot Cause Addressed: Similar to executionRegistry — reduces toolsProvider.ts import count and centralizes utility tool registration logic. Follows the same aggregator pattern established in v1.8.2 declarative registry.
src/utils/simulation.ts)Comprehensive test harness for Graphify-inspired architectural analysis features.
analyzeAiToolboxDependencies() runs clustering on actual 24-module source graph (50+ connections) — verifies hub identification, cluster formation, and quality metricsRoot Cause Addressed: Prior to this simulation, hub-exclusion clustering features had no systematic test harness. The simulation validates all 83 clustering tests plus additional integration scenarios (ToolPriority centrality scores, ContextGuard file cluster info) — ensuring architectural analysis tools produce correct results before deployment.
src/tools/imageAnalysisTools.ts)Resolved TypeScript compilation errors and ESLint warnings through ESM conversion and proper type assertions.
Impact:
imageAnalysisTools.tssrc/toolsProvider.ts (instrumentedImplementation): after the FIX #20 measurement/guard block, plain-object results are returned as { ...result, executedTool }, where executedTool = the registered (post-minification) name of the implementation that actually ran. This is the same name the AutoTracker DELTA log lines already used for host-log attribution — now it also reaches the chat transcript via the payload itself.executedTool today (grep-verified before introduction); on any future collision the wrapper value is authoritative. Routing, side effects, timing and error propagation are unchanged; FIX #20 A1 bookkeeping still records the original payload with the same ground-truth name.tests/executedToolTransparency.test.ts (8 tests) exercises the real registration → minify → instrument pipeline via six side-effect-free probe tools, incl. a regression guard for FIX #20 A1 (recordToolResult once per success / zero on failure). Guard expression additionally verified offline: 14/14 payload-class edge cases pass.npx jest tests/executedToolTransparency.test.ts + full baseline (657 existing + 8 new expected green); live activation = sync src/toolsProvider.ts into the source-run LM Studio install + full restart. No version bump (v1.9.12 rev 23 stays current).src/performanceUtils.ts): readBoundedText / readCappedText (250K–500K char budgets) now gate fetch_web_content, all three search-engine fallbacks, the five HTTP-client body reads, and wikipedia_search; every fetchWithRetry attempt is bounded by a 30 s AbortController timeout.checkHeapPressure() pre-call probe logs a [HEAP-GUARD] ⚠️ N MB BEFORE "<tool>" started line when heap usage crosses 1 GB — names the suspect call in any future OOM crash log.src/tools/vectorRagTools.ts): soft 250K char cap (oversized pages → success:true + truncated:true with usable partial chunks), HTML markup stripped via html-to-text before chunking/embedding, top-5 cosine-ranked chunks in the result payload; dead duplicate registration removed (tool served exclusively by vectorRAG — dedup invariant: exactly 1× per dist bundle).chunkText / chunkDocxText / chunkPdfText now enforce strict forward progress (startIndex = Math.max(endIndex, startIndex + 1)) — eliminates the deterministic V8 OOM loop where certain word-count remainders stall the window start at a fixed point near end-of-text.tests/vectorRagTools.ragWebContent.test.ts (incl. case-insensitive heading assertion per html-to-text defaults); shared-mock isolation (mockReset() + re-seed) in tests/webResearchTools.test.ts beforeEach — closed the last order-dependent failure.tokenStatsManager.recordToolResult() into a running per-turn delta; threshold/compression decisions no longer wait for the next full history count, so 75%/90% triggers fire inside long multi-tool turns.src/tokenStatsManager.ts): [AutoTracker] [DELTA] lines append | chat used ≈ N tok, where N = turnBaselineTokens (TokenCheck baseline captured at turn start) + midLoopEstTokens. Nested-count semantics: tool +delta ⊆ turn total ⊆ chat used.src/tools/httpClientTools.ts and src/tools/networkToolsRegistry.ts, eslint-disable-next-line @typescript-eslint/no-unsafe-assignment comments were flagged as unused because assigning response.json() to variables with explicit : unknown type is already safe per TypeScript/ESLint rules.unknown annotations: Replaced implicit any assignments (const data = await response.json();) with typed declarations (const data: unknown = await response.json();) across all HTTP response parsing paths — 10 warnings resolved total..content casting with LM Studio's native history API (getLength(), at(i), getText()), matching vibe-lm's approach. The previous code assumed msg.content was always accessible via property access, but SDK messages use getter methods instead — resulting in 0 character counts and inaccurate token estimates.countTokens() × 65 calibration to History Text Length × 0.24 ratio. Empirical testing confirmed that historyChars × 0.24 matches LM Studio sidebar token counts exactly (verified at ~130K tokens for 544,578 chars), whereas SDK-native counting with ×65 overestimated by ~45k tokens (~124K vs ~80K).checkpointSuffix variable in promptPreprocessor.ts that guarantees the auto-tracking threshold prompt is injected into every possible code path (directory detection, RAG disabled, no files found). Previously, the warning was silently swallowed due to early-return gates.backgroundCommandManagerany[] types, replaced with typed closures (() => Tool[])for...of iteration replaces scattered conditional blocksDEFAULT_EXCLUDED_DIRS Set in walkDirectory() function within src/tools/fileSystemTools.tsnode_modules, .git, dist, build, .next, .nuxt, __pycache__, .cache, vendor, .vscode, .idea, .vsinclude pattern (backward compatible)Added run_tests (v1.5.23) |
| Utilities | ~29 → ~10 | ~10 tools | Refactored into dedicated modules (backup, data visualization, line operations, markdown preview) under utility config key |
| Image Processing | 4 | 4 tools | No change |
| HTTP Client | 3 | 3 tools | No change |
| Vector RAG | 4 → 7 | 7 tools | Added rag_index_pdf, rag_index_docx, rag_index_xlsx (v1.9.2) — PDF per-page chunking, DOCX word-bounded via mammoth, XLSX row-based with sheet-name prefix |
| Text Processing | 3 → 4 | 4 tools | Added line_operations with safety guardrails (v1.7.0) |
| Interactive UI Generation | 3 | 3 tools | No change |
| Context Management | 7 → 12 | 12 tools | Expanded to include all memory/context operations |
| AST Refactoring | N/A | 2 tools | refactor_code, unusedImports (v1.5.30+) |
| Backup Operations | N/A | 5 tools | Registered under utility toggle (v1.6.2+) |
| Data Visualization | N/A | 1 tool | generate_chart registered under utility toggle |
| Line Operations | N/A | 1 tool | With safety guardrails (v1.7.0) |
| Markdown Preview | N/A | 1 tool | Registered under utility toggle |
(.+)+, (a*)*), alternating groups with quantifiers (((a|b)+)+)(a|b)+, [a-z]+, ^import\s+ are correctly acceptedgrep_files returns a patternMode field — 'regex', 'literal' or 'auto_escaped' (forced-literal decisions include an explanatory hint string, REV-24)tsup.config.tsEXTRACTED (deterministic: file reads, grep matches), INFERRED (semantic: RAG queries, heuristic scoring), AMBIGUOUS (uncertain: fallback paths used)"file:src/utils.ts L42", "rag_query_vector") for traceabilitydetermineConfidence(), createToolResult<T>(), createErrorResult() — standardized confidence assignment across all tool modulesbuildDependencyGraph() analyzes TypeScript/JavaScript imports to build adjacency lists from source file relationshipsidentifyHubs() uses configurable percentile threshold (default: 80th) to detect high-degree modules that act as architectural gluelouvainCommunityDetection() runs greedy modularity optimization on non-hub subgraph for cluster formationreattachHubsByMajorityVote() assigns hubs to clusters based on neighbor membership — ties broken by lower cluster ID (stability)calculateClusterDensity() measures internal edge ratio [0-1] for each communitycalculateModularity() evaluates clustering quality (higher = better separation, 0-1 scale)analyzeAiToolboxDependencies() pre-populates graph from documented architecture in ARCHITECTURE.md for immediate analysispackage.json (+0.4), src/ or lib/ (+0.3), .git (+0.1), build config files (tsconfig.json, jest.config.*) (+0.2)"ai-toolbox" ↔ "ai_toolbox")generateNameVariants() creates multiple search variants including vowel-boundary splits (e.g., "aitoolbox" → "ai-tool-box")_origin: 'ast' | 'semantic' distinguishes raw file/AST content from derived AI insightsreplaceTier() replaces only changed tiers while preserving unchanged ones via ID matching — mimics graphify's incremental update patternmakeProvenanceId(sourceFile, label) generates deterministic IDs from source file and label (e.g., "ctx_utils_default")createAstNode(), createSemanticNode() simplify node creation with automatic provenance assignmentCRITICAL (1), HIGH (2), STANDARD (3), OPTIONAL (4), BACKGROUND (5) — 80 tools categorized across all tierssortToolsByClusterAwarePriority() integrates hub-exclusion clustering results — within each tier, tools are sorted by centrality score (module degree × hub bonus) then alphabeticallycomputeCentralityScores() calculates (degree / maxDegree) × hubBonus where hubBonus = 1.5 for hubs, 1.0 otherwise — capped at 1.0CATEGORY_TO_MODULE maps each tool category to source file(s) with dual-name support (bare + path-prefixed)generateClusterAwareFilterReport() generates human-readable reports showing which tools would be filtered given a limit, grouped by tier and clusteratomicWriteBinaryFile() uses raw buffer writes for image processing and chart generationwriteFileSync/renameSync eliminated from src/tools/run_javascript, run_python, run_in_terminal, execute_command, run_tests) registered via registerExecutionTools() — each gated by individual config toggles + GOD MODE overridetoolsProvider.ts with a single import of executionRegistry.tsconfig.godMode as fallback if its specific toggle is disabledsave_file or pattern-based replacement instead of line-number operationsresetTracking() clears all entries — call at session boundariessearxng_batch_fetchfetch_web_contentsearxng_searchsearxng_fetch_urlgetCriticalToolWarnings() function: Returns all warnings concatenated — can be injected into system promptsrequire('../attachmentManager.js') (CommonJS) with static ESM import import { listAttachments, getAttachment } from '../attachmentManager.js' — eliminates @typescript-eslint/no-require-imports warningtype FileHandleWithReadFile = { name: string; readFile?: () => Promise<Buffer>; read?: () => Promise<unknown> } and cast via as unknown as FileHandleWithReadFile | undefined — resolves TS2339 error where SDK's FileHandle type lacks .readFile() declaration (matching pattern from promptPreprocessor.ts:218-247)@typescript-eslint/no-unsafe-*) — file no longer imports Tesseract.jsStep 1: Is this a LOCAL file or REMOTE URL?
IF LOCAL FILE (file://, C:/path/, ./relative/):
→ Use read_file(file_name) for single files
→ Use find_files(pattern) to search first, then read each result
IF REMOTE URL (http://, https://):
→ Use fetch_web_content(url) for single page
→ Use searxng_batch_fetch(urls=[...]) for multiple pages at once
→ Use searxng_search(query="...") for web search only
// Layer 1: promptPreprocessor.ts — Step 0.7 (NEW)
async function detectProjectKeywords(message: string): Promise<string | null> {
const registry = await readProjectRegistry();
for (const word of extractCandidateWords(message)) {
if (normalizeProjectName(word) === normalizeProjectName(project.name)) {
return `REGISTERED PROJECT DETECTED: ${project.name}`;
}
}
}
// Layer 2: registryManager.ts — _syncFromSessionMemory() (NEW)
async function _syncFromSessionMemory(): Promise<void> {
const entries = await loadContextEntries(); // From .ai_toolbox_memory.msgpack
for (const entry of entries) {
if ('decision' in entry.data) {
const match = extractProjectNameFromDecision(entry.data.decision);
if (match) await registerProject(match.name, match.path);
}
}
}
// Step 0.7 in promptPreprocessor.ts (v1.9.8+) — NEW
async function detectProjectKeywords(message: string): Promise<string | null> {
const registry = await readProjectRegistry();
const words = extractCandidateWords(message); // Filter stop-words, lowercase
for (const word of words) {
for (const project of registry.projects) {
if (normalizeProjectName(word) === normalizeProjectName(project.name)) {
return `REGISTERED PROJECT DETECTED: ${project.name} at ${project.path}`;
}
}
}
return null; // No match → fall through to directory detection (Step 1)
}
// _syncFromSessionMemory() in registry manager — NEW (v1.9.8+)
async function _syncFromSessionMemory(): Promise<void> {
const entries = await loadContextEntries();
for (const entry of entries) {
if ('decision' in entry.data && typeof entry.data.decision === 'string') {
const match = extractProjectNameFromDecision(entry.data.decision as string);
if (match) {
await registerProject(match.name, match.path);
}
}
}
}
src/toolsProvider.ts (instrumentedImplementation): after the FIX #20 measurement/guard block, plain-object results are returned as { ...result, executedTool }, where executedTool = the registered (post-minification) name of the implementation that actually ran. This is the same name the AutoTracker DELTA log lines already used for host-log attribution — now it also reaches the chat transcript via the payload itself.executedTool today (grep-verified before introduction); on any future collision the wrapper value is authoritative. Routing, side effects, timing and error propagation are unchanged; FIX #20 A1 bookkeeping still records the original payload with the same ground-truth name.tests/executedToolTransparency.test.ts (8 tests) exercises the real registration → minify → instrument pipeline via six side-effect-free probe tools, incl. a regression guard for FIX #20 A1 (recordToolResult once per success / zero on failure). Guard expression additionally verified offline: 14/14 payload-class edge cases pass.npx jest tests/executedToolTransparency.test.ts + full baseline (657 existing + 8 new expected green); live activation = sync src/toolsProvider.ts into the source-run LM Studio install + full restart. No version bump (v1.9.12 rev 23 stays current).src/performanceUtils.ts): readBoundedText / readCappedText (250K–500K char budgets) now gate fetch_web_content, all three search-engine fallbacks, the five HTTP-client body reads, and wikipedia_search; every fetchWithRetry attempt is bounded by a 30 s AbortController timeout.checkHeapPressure() pre-call probe logs a [HEAP-GUARD] ⚠️ N MB BEFORE "<tool>" started line when heap usage crosses 1 GB — names the suspect call in any future OOM crash log.src/tools/vectorRagTools.ts): soft 250K char cap (oversized pages → success:true + truncated:true with usable partial chunks), HTML markup stripped via html-to-text before chunking/embedding, top-5 cosine-ranked chunks in the result payload; dead duplicate registration removed (tool served exclusively by vectorRAG — dedup invariant: exactly 1× per dist bundle).chunkText / chunkDocxText / chunkPdfText now enforce strict forward progress (startIndex = Math.max(endIndex, startIndex + 1)) — eliminates the deterministic V8 OOM loop where certain word-count remainders stall the window start at a fixed point near end-of-text.tests/vectorRagTools.ragWebContent.test.ts (incl. case-insensitive heading assertion per html-to-text defaults); shared-mock isolation (mockReset() + re-seed) in tests/webResearchTools.test.ts beforeEach — closed the last order-dependent failure.tokenStatsManager.recordToolResult() into a running per-turn delta; threshold/compression decisions no longer wait for the next full history count, so 75%/90% triggers fire inside long multi-tool turns.src/tokenStatsManager.ts): [AutoTracker] [DELTA] lines append | chat used ≈ N tok, where N = turnBaselineTokens (TokenCheck baseline captured at turn start) + midLoopEstTokens. Nested-count semantics: tool +delta ⊆ turn total ⊆ chat used.src/tools/httpClientTools.ts and src/tools/networkToolsRegistry.ts, eslint-disable-next-line @typescript-eslint/no-unsafe-assignment comments were flagged as unused because assigning response.json() to variables with explicit : unknown type is already safe per TypeScript/ESLint rules.unknown annotations: Replaced implicit any assignments (const data = await response.json();) with typed declarations (const data: unknown = await response.json();) across all HTTP response parsing paths — 10 warnings resolved total..content casting with LM Studio's native history API (getLength(), at(i), getText()), matching vibe-lm's approach. The previous code assumed msg.content was always accessible via property access, but SDK messages use getter methods instead — resulting in 0 character counts and inaccurate token estimates.countTokens() × 65 calibration to History Text Length × 0.24 ratio. Empirical testing confirmed that historyChars × 0.24 matches LM Studio sidebar token counts exactly (verified at ~130K tokens for 544,578 chars), whereas SDK-native counting with ×65 overestimated by ~45k tokens (~124K vs ~80K).checkpointSuffix variable in promptPreprocessor.ts that guarantees the auto-tracking threshold prompt is injected into every possible code path (directory detection, RAG disabled, no files found). Previously, the warning was silently swallowed due to early-return gates.backgroundCommandManagerany[] types, replaced with typed closures (() => Tool[])for...of iteration replaces scattered conditional blocksDEFAULT_EXCLUDED_DIRS Set in walkDirectory() function within src/tools/fileSystemTools.tsnode_modules, .git, dist, build, .next, .nuxt, __pycache__, .cache, vendor, .vscode, .idea, .vsinclude pattern (backward compatible)Added run_tests (v1.5.23) |
| Utilities | ~29 → ~10 | ~10 tools | Refactored into dedicated modules (backup, data visualization, line operations, markdown preview) under utility config key |
| Image Processing | 4 | 4 tools | No change |
| HTTP Client | 3 | 3 tools | No change |
| Vector RAG | 4 → 7 | 7 tools | Added rag_index_pdf, rag_index_docx, rag_index_xlsx (v1.9.2) — PDF per-page chunking, DOCX word-bounded via mammoth, XLSX row-based with sheet-name prefix |
| Text Processing | 3 → 4 | 4 tools | Added line_operations with safety guardrails (v1.7.0) |
| Interactive UI Generation | 3 | 3 tools | No change |
| Context Management | 7 → 12 | 12 tools | Expanded to include all memory/context operations |
| AST Refactoring | N/A | 2 tools | refactor_code, unusedImports (v1.5.30+) |
| Backup Operations | N/A | 5 tools | Registered under utility toggle (v1.6.2+) |
| Data Visualization | N/A | 1 tool | generate_chart registered under utility toggle |
| Line Operations | N/A | 1 tool | With safety guardrails (v1.7.0) |
| Markdown Preview | N/A | 1 tool | Registered under utility toggle |
(.+)+, (a*)*), alternating groups with quantifiers (((a|b)+)+)(a|b)+, [a-z]+, ^import\s+ are correctly acceptedgrep_files returns a patternMode field — 'regex', 'literal' or 'auto_escaped' (forced-literal decisions include an explanatory hint string, REV-24)tsup.config.tsEXTRACTED (deterministic: file reads, grep matches), INFERRED (semantic: RAG queries, heuristic scoring), AMBIGUOUS (uncertain: fallback paths used)"file:src/utils.ts L42", "rag_query_vector") for traceabilitydetermineConfidence(), createToolResult<T>(), createErrorResult() — standardized confidence assignment across all tool modulesbuildDependencyGraph() analyzes TypeScript/JavaScript imports to build adjacency lists from source file relationshipsidentifyHubs() uses configurable percentile threshold (default: 80th) to detect high-degree modules that act as architectural gluelouvainCommunityDetection() runs greedy modularity optimization on non-hub subgraph for cluster formationreattachHubsByMajorityVote() assigns hubs to clusters based on neighbor membership — ties broken by lower cluster ID (stability)calculateClusterDensity() measures internal edge ratio [0-1] for each communitycalculateModularity() evaluates clustering quality (higher = better separation, 0-1 scale)analyzeAiToolboxDependencies() pre-populates graph from documented architecture in ARCHITECTURE.md for immediate analysispackage.json (+0.4), src/ or lib/ (+0.3), .git (+0.1), build config files (tsconfig.json, jest.config.*) (+0.2)"ai-toolbox" ↔ "ai_toolbox")generateNameVariants() creates multiple search variants including vowel-boundary splits (e.g., "aitoolbox" → "ai-tool-box")_origin: 'ast' | 'semantic' distinguishes raw file/AST content from derived AI insightsreplaceTier() replaces only changed tiers while preserving unchanged ones via ID matching — mimics graphify's incremental update patternmakeProvenanceId(sourceFile, label) generates deterministic IDs from source file and label (e.g., "ctx_utils_default")createAstNode(), createSemanticNode() simplify node creation with automatic provenance assignmentCRITICAL (1), HIGH (2), STANDARD (3), OPTIONAL (4), BACKGROUND (5) — 80 tools categorized across all tierssortToolsByClusterAwarePriority() integrates hub-exclusion clustering results — within each tier, tools are sorted by centrality score (module degree × hub bonus) then alphabeticallycomputeCentralityScores() calculates (degree / maxDegree) × hubBonus where hubBonus = 1.5 for hubs, 1.0 otherwise — capped at 1.0CATEGORY_TO_MODULE maps each tool category to source file(s) with dual-name support (bare + path-prefixed)generateClusterAwareFilterReport() generates human-readable reports showing which tools would be filtered given a limit, grouped by tier and clusteratomicWriteBinaryFile() uses raw buffer writes for image processing and chart generationwriteFileSync/renameSync eliminated from src/tools/run_javascript, run_python, run_in_terminal, execute_command, run_tests) registered via registerExecutionTools() — each gated by individual config toggles + GOD MODE overridetoolsProvider.ts with a single import of executionRegistry.tsconfig.godMode as fallback if its specific toggle is disabledsave_file or pattern-based replacement instead of line-number operationsresetTracking() clears all entries — call at session boundariessearxng_batch_fetchfetch_web_contentsearxng_searchsearxng_fetch_urlgetCriticalToolWarnings() function: Returns all warnings concatenated — can be injected into system promptsrequire('../attachmentManager.js') (CommonJS) with static ESM import import { listAttachments, getAttachment } from '../attachmentManager.js' — eliminates @typescript-eslint/no-require-imports warningtype FileHandleWithReadFile = { name: string; readFile?: () => Promise<Buffer>; read?: () => Promise<unknown> } and cast via as unknown as FileHandleWithReadFile | undefined — resolves TS2339 error where SDK's FileHandle type lacks .readFile() declaration (matching pattern from promptPreprocessor.ts:218-247)@typescript-eslint/no-unsafe-*) — file no longer imports Tesseract.jsStep 1: Is this a LOCAL file or REMOTE URL?
IF LOCAL FILE (file://, C:/path/, ./relative/):
→ Use read_file(file_name) for single files
→ Use find_files(pattern) to search first, then read each result
IF REMOTE URL (http://, https://):
→ Use fetch_web_content(url) for single page
→ Use searxng_batch_fetch(urls=[...]) for multiple pages at once
→ Use searxng_search(query="...") for web search only
// Layer 1: promptPreprocessor.ts — Step 0.7 (NEW)
async function detectProjectKeywords(message: string): Promise<string | null> {
const registry = await readProjectRegistry();
for (const word of extractCandidateWords(message)) {
if (normalizeProjectName(word) === normalizeProjectName(project.name)) {
return `REGISTERED PROJECT DETECTED: ${project.name}`;
}
}
}
// Layer 2: registryManager.ts — _syncFromSessionMemory() (NEW)
async function _syncFromSessionMemory(): Promise<void> {
const entries = await loadContextEntries(); // From .ai_toolbox_memory.msgpack
for (const entry of entries) {
if ('decision' in entry.data) {
const match = extractProjectNameFromDecision(entry.data.decision);
if (match) await registerProject(match.name, match.path);
}
}
}
// Step 0.7 in promptPreprocessor.ts (v1.9.8+) — NEW
async function detectProjectKeywords(message: string): Promise<string | null> {
const registry = await readProjectRegistry();
const words = extractCandidateWords(message); // Filter stop-words, lowercase
for (const word of words) {
for (const project of registry.projects) {
if (normalizeProjectName(word) === normalizeProjectName(project.name)) {
return `REGISTERED PROJECT DETECTED: ${project.name} at ${project.path}`;
}
}
}
return null; // No match → fall through to directory detection (Step 1)
}
// _syncFromSessionMemory() in registry manager — NEW (v1.9.8+)
async function _syncFromSessionMemory(): Promise<void> {
const entries = await loadContextEntries();
for (const entry of entries) {
if ('decision' in entry.data && typeof entry.data.decision === 'string') {
const match = extractProjectNameFromDecision(entry.data.decision as string);
if (match) {
await registerProject(match.name, match.path);
}
}
}
}