# Environment Setup This guide shows how to create the backend `.env` file for local development. Source of truth: - the variables documented here are based on what the backend currently reads from `src/aibs_backend/settings.py` in the backend repository - if this page ever drifts from the code, trust `src/aibs_backend/settings.py` ## Before You Start The backend expects these runtime dependencies: - OpenSearch reachable from your machine - the local embedding model bundle under `models/embeddings/bge-m3` - the local reranker bundle under `models/reranking/bge-reranker-v2-m3` if reranking is enabled - local Docling artifacts under `models/docling` if `AIBS_DOCLING_ARTIFACTS_PATH` is configured - an Ollama-compatible LLM endpoint if grounded answering is enabled ## How To Create `.env` 1. Create a file named `.env` in the repository root. 2. Copy the template below into it. 3. Replace the placeholder values with your local settings. 4. Never commit real credentials into git. ## Recommended Template ```ini AIBS_ENV=development # API server AIBS_HOST=127.0.0.1 AIBS_PORT=5000 AIBS_DEBUG=true AIBS_CORS_ORIGINS=http://localhost:3000 # OpenSearch AIBS_OPENSEARCH_HOST=localhost AIBS_OPENSEARCH_PORT=9200 AIBS_OPENSEARCH_SSL=false AIBS_OPENSEARCH_VERIFY_CERTS=false AIBS_OPENSEARCH_USERNAME=YOUR_OPENSEARCH_USERNAME AIBS_OPENSEARCH_PASSWORD=YOUR_OPENSEARCH_PASSWORD AIBS_OPENSEARCH_INDEX_TEMPLATE=.abs-content-{workspace_id} AIBS_OPENSEARCH_REQUEST_TIMEOUT_SECONDS=45 AIBS_OPENSEARCH_BULK_REQUEST_TIMEOUT_SECONDS=60 AIBS_OPENSEARCH_BULK_BATCH_SIZE=100 AIBS_OPENSEARCH_MAX_CONNECTIONS=10 AIBS_OPENSEARCH_SEMANTIC_DOCUMENT_CANDIDATE_MULTIPLIER=4 AIBS_OPENSEARCH_SEMANTIC_DOCUMENT_MAX_CANDIDATES=500 AIBS_OPENSEARCH_SEMANTIC_DOCUMENT_SNIPPET_TIMEOUT_SECONDS=2 # Embeddings AIBS_EMBEDDING_MODEL=BAAI/bge-m3 AIBS_EMBEDDING_MODEL_PATH=models/embeddings/bge-m3 # Remote GPU worker path: # AIBS_EMBEDDING_MODEL_PATH=/opt/aibs/models/embeddings/bge-m3 AIBS_EMBEDDING_DIMENSION=1024 AIBS_EMBEDDING_BATCH_SIZE=32 AIBS_EMBEDDING_NORMALIZE=true AIBS_EMBEDDING_LOCAL_FILES_ONLY=true AIBS_EMBEDDING_DEVICE=cuda # Reranker AIBS_RERANKER_ENABLED=true AIBS_RERANKER_MODEL=BAAI/bge-reranker-v2-m3 AIBS_RERANKER_MODEL_PATH=models/reranking/bge-reranker-v2-m3 # Remote GPU worker path: # AIBS_RERANKER_MODEL_PATH=/opt/aibs/models/reranking/bge-reranker-v2-m3 AIBS_RERANKER_MAX_CANDIDATES=80 AIBS_RERANKER_LOCAL_FILES_ONLY=true AIBS_RERANKER_DIAGNOSTIC_TEXT_PREVIEW_ENABLED=false AIBS_RERANKER_SCORE_ACTIVATION=sigmoid AIBS_RERANKER_DEVICE=auto # Docling PDF parser AIBS_DOCLING_ARTIFACTS_PATH=models/docling # Remote GPU worker path: # AIBS_DOCLING_ARTIFACTS_PATH=/opt/aibs/models/docling AIBS_DOCLING_DEVICE=cuda AIBS_DOCLING_NUM_THREADS=1 AIBS_DOCLING_OCR_BATCH_SIZE=1 AIBS_DOCLING_LAYOUT_BATCH_SIZE=1 AIBS_DOCLING_TABLE_BATCH_SIZE=1 AIBS_DOCLING_QUEUE_MAX_SIZE=8 AIBS_PARSING_ENABLE_FORMULA_MODEL=false # Chunking AIBS_CHUNKING_ABSTRACTIVE_SUMMARIES_ENABLED=true # Ingestion worker AIBS_INGESTION_WORKER_ENABLED=true AIBS_INGESTION_WORKER_COUNT=2 AIBS_INGESTION_WORKER_CLAIM_SCAN_SIZE=10 AIBS_INGESTION_MAX_CONCURRENT_PARSE_STAGES=1 AIBS_INGESTION_MAX_CONCURRENT_CHUNK_STAGES=2 AIBS_INGESTION_MAX_CONCURRENT_INDEX_STAGES=1 AIBS_INGESTION_MAX_CONCURRENT_SEMANTIC_PROFILE_STAGES=2 AIBS_INGESTION_MAX_CONCURRENT_HEAVY_MODEL_STAGES=1 AIBS_INGESTION_MEMORY_HEAVY_PAGE_COUNT_THRESHOLD=100 AIBS_INGESTION_MEMORY_HEAVY_TABLE_COUNT_THRESHOLD=50 AIBS_INGESTION_MEMORY_HEAVY_SECTION_COUNT_THRESHOLD=250 AIBS_INGESTION_MEMORY_HEAVY_CHUNK_COUNT_THRESHOLD=500 AIBS_INGESTION_WORKER_IDLE_SLEEP_SECONDS=1.0 AIBS_INGESTION_WORKER_ERROR_SLEEP_SECONDS=5.0 AIBS_INGESTION_WORKER_LEASE_REAP_INTERVAL_SECONDS=60.0 AIBS_INGESTION_EXPIRED_LEASE_CLAIM_MULTIPLIER=10 AIBS_INGESTION_UPLOAD_QUIET_PERIOD_SECONDS=5.0 AIBS_INGESTION_UPLOAD_BATCH_TTL_SECONDS=3600.0 # Conversation memory AIBS_CONVERSATION_MEMORY_ENABLED=true AIBS_CONVERSATION_MEMORY_INDEX=aibs-conversation-memory AIBS_CONVERSATION_MEMORY_RECENT_TURNS=8 AIBS_CONVERSATION_MEMORY_RETRIEVAL_TOP_K=4 AIBS_CONVERSATION_MEMORY_MAX_TURN_CHARS=4000 # Answer pagination AIBS_ANSWER_PAGE_DOCUMENT_LIMIT=10 AIBS_ANSWER_CONTINUATION_TOKEN_TTL_SECONDS=86400 AIBS_ANSWER_MAP_CONCURRENCY_LIMIT=5 AIBS_DOCUMENT_WORK_MODEL_DEADLINE_SECONDS=45 AIBS_ANSWER_MAP_REDUCE_EVIDENCE_LIMIT=40 AIBS_ANSWER_FAST_MULTI_DOC_EVIDENCE_CAP=80 AIBS_ANSWER_MULTI_DOC_CHUNKS_PER_DOCUMENT=8 AIBS_ANSWER_MAP_REDUCE_FALLBACK_CHUNK_LIMIT=20 AIBS_ANSWER_SUMMARY_ANCHOR_MAX_DOCUMENTS=10 AIBS_STRICT_GROUNDED_QUARANTINED_MODEL_IDS=gpt-oss:20b AIBS_ANSWER_METRICS_SINCE=2026-07-07T10:25:45Z AIBS_DOCUMENT_DISCOVERY_MIN_ENTITY_ANCHOR_DOCUMENTS=2 AIBS_DOCUMENT_DISCOVERY_MIN_UNDELIMITED_ENTITY_ANCHOR_DOCUMENTS=3 AIBS_DOCUMENT_DISCOVERY_BRANCH_TIMEOUT_SECONDS=20 AIBS_DOCUMENT_PROFILE_BINDING_TIMEOUT_SECONDS=3 AIBS_PLANNING_CATALOG_CACHE_TTL_SECONDS=0 AIBS_PLANNING_CATALOG_CACHE_MAX_WORKSPACES=64 AIBS_DOCUMENT_COMPLEXITY_MEDIUM_PAGE_COUNT=25 AIBS_DOCUMENT_COMPLEXITY_LARGE_PAGE_COUNT=100 AIBS_DOCUMENT_COMPLEXITY_MEDIUM_CHUNK_COUNT=120 AIBS_DOCUMENT_COMPLEXITY_LARGE_CHUNK_COUNT=500 AIBS_DOCUMENT_COMPLEXITY_MEDIUM_SECTION_COUNT=50 AIBS_DOCUMENT_COMPLEXITY_LARGE_SECTION_COUNT=200 AIBS_DOCUMENT_COMPLEXITY_MEDIUM_TABLE_COUNT=4 AIBS_DOCUMENT_COMPLEXITY_LARGE_TABLE_COUNT=20 AIBS_DOCUMENT_COMPLEXITY_MEDIUM_REQUEST_SIGNAL_COUNT=4 AIBS_DOCUMENT_COMPLEXITY_LARGE_REQUEST_SIGNAL_COUNT=10 AIBS_DOCUMENT_COMPLEXITY_SMALL_EVIDENCE_BUDGET=4 AIBS_DOCUMENT_COMPLEXITY_MEDIUM_EVIDENCE_BUDGET=8 AIBS_DOCUMENT_COMPLEXITY_LARGE_EVIDENCE_BUDGET=12 AIBS_DOCUMENT_COMPLEXITY_DEGRADED_LARGE_EVIDENCE_BUDGET=16 AIBS_DOCUMENT_COMPLEXITY_PAGE_CONSTRAINED_EVIDENCE_BUDGET=6 AIBS_DOCUMENT_COMPLEXITY_STRUCTURED_OUTPUT_MIN_EVIDENCE_BUDGET=10 # Heavy-tier multi-doc combine AIBS_ANSWER_HEAVY_COMBINE_DOCUMENT_LIMIT=64 AIBS_ANSWER_HEAVY_EVIDENCE_CHUNK_CAP=320 AIBS_SCOPED_MULTI_DOC_CANDIDATE_MULTIPLIER_CAP=4 # Workspace document inventory AIBS_CATALOG_INVENTORY_MIN_SCORE=0 AIBS_CATALOG_INVENTORY_LIST_LIMIT=25 # Local storage AIBS_UPLOAD_ROOT=.aibs_data/uploads AIBS_DOCUMENT_ROOT=.aibs_data/documents AIBS_WORKSPACE_ROOT=.aibs_data/workspaces AIBS_CONVERSATION_ROOT=.aibs_data/conversations AIBS_QUERY_ROOT=.aibs_data/queries AIBS_ANSWER_ROOT=.aibs_data/answers AIBS_STAGING_ROOT=.aibs_data/staging AIBS_PROJECTION_WORKSPACE_ROUTING_ENABLED=false # This is a literal OpenSearch queue index name. It is not derived from AIBS_ENV. # Use a unique value per environment/developer unless processes intentionally share jobs. AIBS_INGESTION_JOB_INDEX=aibs-ingestion-jobs-dev AIBS_DOCUMENT_CATALOG_INDEX=aibs-document-catalog AIBS_DOCUMENT_PROFILE_INDEX=aibs-document-profiles AIBS_SECTION_SUMMARY_INDEX=aibs-section-summaries AIBS_PROJECTION_CONTROL_INDEX=aibs-projection-control AIBS_RETRIEVAL_PYRAMID_DISCOVERY_ENABLED=true AIBS_PROJECTION_CONTROL_SUCCEEDED_EVENT_RETENTION_DAYS=7 AIBS_PROJECTION_CONTROL_RECONCILED_EVENT_RETENTION_DAYS=20 AIBS_MAX_UPLOAD_SIZE_BYTES=104857600 # Auth and identity AIBS_AUTH_DEV_MODE=true AIBS_AUTH_DEV_ROLES=aibs-access,aibs-admin,developer AIBS_AUTH_SESSION_COOKIE_NAME=aibs-session AIBS_AUTH_SESSION_TTL_SECONDS=3600 AIBS_AUTH_COOKIE_SECURE=false AIBS_AUTH_COOKIE_SAMESITE=lax AIBS_AUTH_COOKIE_PATH=/ # AIBS_AUTH_COOKIE_DOMAIN= # Dev compatibility header; accepted only when AIBS_AUTH_DEV_MODE=true. AIBS_IDENTITY_HEADER_NAME=X-AIBS-User-Id # Logging AIBS_LOG_ROOT=logs AIBS_APP_LOG_FILENAME=aibs.log AIBS_MODEL_RESIDENCY_LOG_FILENAME=model_residency.log AIBS_INGESTION_LOG_FILENAME=ingestion.log AIBS_RETRIEVAL_LOG_FILENAME=retrieval.log AIBS_LOG_MAX_BYTES=5242880 AIBS_LOG_BACKUP_COUNT=5 # LLM AIBS_LLM_ENABLED=true AIBS_LLM_PROVIDER=ollama AIBS_LLM_BASE_URL=http://10.4.3.16:11434 AIBS_LLM_MODEL=gemma4:26b AIBS_LLM_MAX_CONCURRENT_REQUESTS=1 AIBS_LLM_REQUEST_TIMEOUT_SECONDS=600 AIBS_LLM_TEMPERATURE=0.1 AIBS_LLM_TOP_P=0.9 AIBS_LLM_TOP_K=40 AIBS_LLM_THINKING_ENABLED=true AIBS_LLM_THINKING_STRATEGY=auto AIBS_LLM_THINKING_MAX_OUTPUT_TOKENS=8192 AIBS_LLM_THINKING_CONTEXT_LENGTH=65536 AIBS_LLM_MAX_OUTPUT_TOKENS=8192 AIBS_LLM_CONTEXT_LENGTH=32768 AIBS_LLM_KEEP_ALIVE=10m AIBS_LLM_STARTUP_CHECK=true # AIBS_LLM_CAPABILITY_CONTEXT= # Optional ingestion summarizer lane. These values fall back to AIBS_LLM_* # when omitted, but keep the summarizer isolated from chat when enabled. # AIBS_LLM_SUMMARIZER_PROVIDER=ollama # AIBS_LLM_SUMMARIZER_BASE_URL=http://10.4.3.16:11434 AIBS_LLM_SUMMARIZER_MODEL=gpt-oss:20b AIBS_LLM_SUMMARIZER_MAX_CONCURRENT_REQUESTS=5 AIBS_LLM_SUMMARIZER_REQUEST_TIMEOUT_SECONDS=600 AIBS_LLM_SUMMARIZER_TEMPERATURE=0.1 AIBS_LLM_SUMMARIZER_TOP_P=0.9 AIBS_LLM_SUMMARIZER_TOP_K=40 AIBS_LLM_SUMMARIZER_MAX_OUTPUT_TOKENS=3072 AIBS_LLM_SUMMARIZER_CONTEXT_LENGTH=32768 AIBS_LLM_SUMMARIZER_KEEP_ALIVE=10m # Dedicated semantic-profile enrichment lane. This is separate from # AIBS_LLM_SUMMARIZER_* so bulk profile backfills do not steal the reduce/summarizer # model lane used by live answers. # AIBS_LLM_SEMANTIC_PROFILE_PROVIDER=ollama # AIBS_LLM_SEMANTIC_PROFILE_BASE_URL=http://10.4.3.16:11434 AIBS_LLM_SEMANTIC_PROFILE_MODEL=qwen3:8b AIBS_LLM_SEMANTIC_PROFILE_MAX_CONCURRENT_REQUESTS=2 AIBS_LLM_SEMANTIC_PROFILE_REQUEST_TIMEOUT_SECONDS=300 AIBS_LLM_SEMANTIC_PROFILE_ADMISSION_QUEUE_TIMEOUT_SECONDS=300 AIBS_LLM_SEMANTIC_PROFILE_TEMPERATURE=0.0 AIBS_LLM_SEMANTIC_PROFILE_TOP_P=0.9 AIBS_LLM_SEMANTIC_PROFILE_TOP_K=40 AIBS_LLM_SEMANTIC_PROFILE_MAX_OUTPUT_TOKENS=2048 AIBS_LLM_SEMANTIC_PROFILE_CONTEXT_LENGTH=16384 AIBS_LLM_SEMANTIC_PROFILE_KEEP_ALIVE=10m # Conversation title enrichment runs after first-message conversation creation. # It is metadata-only: it must not affect routing, answer language, or answer # generation. The generated title is skipped if the user manually renames the # conversation first. AIBS_LLM_CONVERSATION_TITLE_ENABLED=true AIBS_LLM_CONVERSATION_TITLE_MODEL=qwen2.5:7b-instruct AIBS_LLM_CONVERSATION_TITLE_MAX_OUTPUT_TOKENS=64 AIBS_LLM_CONVERSATION_TITLE_CONTEXT_LENGTH=2048 AIBS_LLM_CONVERSATION_TITLE_REQUEST_TIMEOUT_SECONDS=30 AIBS_LLM_CONVERSATION_TITLE_ADMISSION_QUEUE_TIMEOUT_SECONDS=5 AIBS_LLM_CONVERSATION_TITLE_KEEP_ALIVE=5m # Optional heavy reasoning lane for business/research and heavy multi-doc combine. # Leave AIBS_LLM_HEAVY_MODEL unset/blank unless a measured residency plan proves # a distinct heavy model can stay resident safely. The current recommendation is # to keep this lane dormant and serve heavy synthesis through the base model plus # map-reduce/agent orchestration. Do not use this setting to create a hidden # same-model "heavy chat" path; same-model heavy profiles belong in Agent Mode. AIBS_LLM_HEAVY_MODEL= # AIBS_LLM_HEAVY_TEMPERATURE=0.1 # AIBS_LLM_HEAVY_TOP_P=0.9 # AIBS_LLM_HEAVY_TOP_K=40 AIBS_LLM_HEAVY_MAX_OUTPUT_TOKENS=8192 AIBS_LLM_HEAVY_CONTEXT_LENGTH=65536 AIBS_LLM_HEAVY_THINKING_ENABLED=true AIBS_LLM_HEAVY_KEEP_ALIVE= # Model admission and document worker roster AIBS_MODEL_ADMISSION_ENABLED=true AIBS_MODEL_ADMISSION_SAFE_POOL_GB=0 AIBS_MODEL_ADMISSION_QUEUE_TIMEOUT_SECONDS=0.25 AIBS_MODEL_ADMISSION_KV_GB_PER_ACTIVE_BILLION_PER_TOKEN=0.000008 AIBS_MODEL_ADMISSION_WORKER_ROSTER=base,summarizer,semantic-profile,map-small,reasoning-small # Global model cluster scheduler foundation. Enable only when worker weights, # per-node budgets, coordination, and live node residency stamping are measured # and running. AIBS_MODEL_CLUSTER_ENABLED=true AIBS_MODEL_CLUSTER_BACKEND=valkey AIBS_MODEL_CLUSTER_FALLBACK_MODE=fail_closed # Explicit node ids become scheduler identity and can be audited while disabled. AIBS_MODEL_CLUSTER_NODE_IDS=box-c,box-e AIBS_MODEL_CLUSTER_NODE_URLS=http://10.4.3.16:11434,http://10.4.3.17:11434 # Compatibility name: on AMD APU/iGPU hosts this budget should reflect measured # usable GPU memory, including GTT where model buffers are resident. AIBS_MODEL_CLUSTER_NODE_VRAM_GB=90,104 # Per-node Ollama/server parallelism, in the same order as NODE_IDS. # Keep a node at 1 until its serving process is configured and load-tested. AIBS_MODEL_CLUSTER_NODE_MAX_PARALLEL_REQUESTS=1,1 # Optional read-only host telemetry agent URLs for System Health charts. # If set, provide one host telemetry URL per node in the same order. # AIBS_MODEL_CLUSTER_NODE_HOST_TELEMETRY_URLS=http://10.4.3.16:8765/telemetry,http://10.4.3.17:8765/telemetry AIBS_MODEL_CLUSTER_ROLE_AFFINITY_JSON={"semantic_profile":["box-c"]} AIBS_MODEL_CLUSTER_ACTIVE_LEASE_TTL_SECONDS=900 AIBS_MODEL_CLUSTER_HEARTBEAT_INTERVAL_SECONDS=30 AIBS_MODEL_CLUSTER_POLL_INTERVAL_SECONDS=0.1 AIBS_MODEL_CLUSTER_MAX_RESIDENCY_STALENESS_MS=20000 AIBS_MODEL_CLUSTER_RESIDENCY_MONITOR_ENABLED=true AIBS_MODEL_CLUSTER_RESIDENCY_MONITOR_POLL_INTERVAL_SECONDS=3 AIBS_MODEL_CLUSTER_RESIDENCY_MONITOR_POLL_TIMEOUT_SECONDS=2 AIBS_MODEL_CLUSTER_RESIDENCY_MONITOR_FAILURE_THRESHOLD=2 AIBS_MODEL_CLUSTER_RESIDENCY_MONITOR_LEASE_TTL_SECONDS=8 AIBS_MODEL_CLUSTER_RESIDENCY_MONITOR_STANDBY_RETRY_SECONDS=1 AIBS_MODEL_CLUSTER_TELEMETRY_TTL_SECONDS=30 AIBS_MODEL_CLUSTER_HEALTH_HISTORY_WINDOW_SECONDS=900 AIBS_MODEL_CLUSTER_HEALTH_HISTORY_MAX_SAMPLES=180 AIBS_MODEL_CLUSTER_HEALTH_HISTORY_TTL_SECONDS=1800 AIBS_WORKER_BASE_ROLES=planner,workspace_answer,document_map,structured_extract,reduce,final_answer AIBS_WORKER_BASE_RESIDENT_WEIGHT_GB=16.02 AIBS_WORKER_BASE_ACTIVE_PARAM_BILLIONS=26 AIBS_WORKER_SUMMARIZER_ROLES=document_map,structured_extract AIBS_WORKER_SUMMARIZER_RESIDENT_WEIGHT_GB=11.86 AIBS_WORKER_SUMMARIZER_ACTIVE_PARAM_BILLIONS=3.6 AIBS_WORKER_SEMANTIC_PROFILE_ROLES=semantic_profile AIBS_WORKER_SEMANTIC_PROFILE_RESIDENT_WEIGHT_GB=4.91 AIBS_WORKER_SEMANTIC_PROFILE_ACTIVE_PARAM_BILLIONS=8 AIBS_WORKER_MAP_SMALL_MODEL_ID=qwen2.5:7b-instruct AIBS_WORKER_MAP_SMALL_ROLES=document_map,structured_extract AIBS_WORKER_MAP_SMALL_CONTEXT_LENGTH=16384 AIBS_WORKER_MAP_SMALL_MAX_OUTPUT_TOKENS=2048 AIBS_WORKER_MAP_SMALL_MAX_CONCURRENT_REQUESTS=1 AIBS_WORKER_MAP_SMALL_RESIDENT_WEIGHT_GB=4.31 AIBS_WORKER_MAP_SMALL_ACTIVE_PARAM_BILLIONS=7 AIBS_WORKER_MAP_SMALL_KEEP_ALIVE=5m AIBS_WORKER_MAP_SMALL_TIMEOUT_SECONDS=240 AIBS_WORKER_REASONING_SMALL_MODEL_ID=qwen3:8b AIBS_WORKER_REASONING_SMALL_ROLES=document_map,structured_extract,reduce AIBS_WORKER_REASONING_SMALL_CONTEXT_LENGTH=32768 AIBS_WORKER_REASONING_SMALL_MAX_OUTPUT_TOKENS=4096 AIBS_WORKER_REASONING_SMALL_MAX_CONCURRENT_REQUESTS=1 AIBS_WORKER_REASONING_SMALL_RESIDENT_WEIGHT_GB=4.91 AIBS_WORKER_REASONING_SMALL_ACTIVE_PARAM_BILLIONS=8 AIBS_WORKER_REASONING_SMALL_KEEP_ALIVE=5m AIBS_WORKER_REASONING_SMALL_TIMEOUT_SECONDS=300 AIBS_DOCUMENT_COMPLEXITY_SMALL_MODEL_IDS=qwen2.5:7b-instruct,qwen3:8b,gpt-oss:20b AIBS_DOCUMENT_COMPLEXITY_MEDIUM_MODEL_IDS=qwen3:8b,qwen2.5:7b-instruct,gpt-oss:20b AIBS_DOCUMENT_COMPLEXITY_LARGE_MODEL_IDS=gpt-oss:20b,qwen3:8b,gemma4:26b AIBS_DOCUMENT_COMPLEXITY_DEGRADED_LARGE_MODEL_IDS=gpt-oss:20b,gemma4:26b,qwen3:8b AIBS_MODEL_ADMISSION_BASE_RESIDENT_WEIGHT_GB=0 AIBS_MODEL_ADMISSION_BASE_ACTIVE_PARAMS_BILLIONS=0 AIBS_MODEL_ADMISSION_SUMMARIZER_RESIDENT_WEIGHT_GB=11.86 AIBS_MODEL_ADMISSION_SUMMARIZER_ACTIVE_PARAMS_BILLIONS=3.6 AIBS_MODEL_ADMISSION_SEMANTIC_PROFILE_RESIDENT_WEIGHT_GB=4.91 AIBS_MODEL_ADMISSION_SEMANTIC_PROFILE_ACTIVE_PARAMS_BILLIONS=8 AIBS_MODEL_ADMISSION_HEAVY_RESIDENT_WEIGHT_GB=0 AIBS_MODEL_ADMISSION_HEAVY_ACTIVE_PARAMS_BILLIONS=0 ``` ## Section Notes ### API server - `AIBS_HOST` and `AIBS_PORT` control where FastAPI listens. - `AIBS_CORS_ORIGINS` should include the frontend origin you use during local development. ### OpenSearch - `AIBS_OPENSEARCH_INDEX_TEMPLATE` is workspace-scoped and should usually be left as-is. - `AIBS_OPENSEARCH_REQUEST_TIMEOUT_SECONDS` is the general client/search timeout. Use a higher value for agentic, multi-document retrieval because document-local neighbor and coverage searches can legitimately run longer than a small single search request under load. - `AIBS_OPENSEARCH_BULK_REQUEST_TIMEOUT_SECONDS` is used for chunk bulk indexing, where large PDFs can legitimately take longer than search requests - `AIBS_OPENSEARCH_BULK_BATCH_SIZE` controls how many vector records are sent per bulk indexing call; lower it for memory-constrained/local boxes, raise cautiously only after live ingestion stays stable - `AIBS_OPENSEARCH_MAX_CONNECTIONS` raises the OpenSearch HTTP connection pool above the tiny default so concurrent job/search activity does not churn connections unnecessarily - `AIBS_OPENSEARCH_SEMANTIC_DOCUMENT_CANDIDATE_MULTIPLIER` controls how many vector chunk hits are fetched per requested document result before document-level grouping - `AIBS_OPENSEARCH_SEMANTIC_DOCUMENT_MAX_CANDIDATES` caps that semantic search candidate window so large source-list searches do not pull an unbounded number of vectors - `AIBS_DOCUMENT_CATALOG_INDEX` names the lightweight OpenSearch read model used by `/documents/catalog`, search hydration, catalog summaries, and large-workspace paging - `AIBS_DOCUMENT_PROFILE_INDEX` names the OpenSearch profile projection used for cheap document discovery before chunk retrieval - `AIBS_SECTION_SUMMARY_INDEX` names the OpenSearch section/table summary projection used to narrow broad questions before full chunk evidence is fetched - `AIBS_PROJECTION_WORKSPACE_ROUTING_ENABLED=true` routes catalog/profile/summary projection writes and workspace-scoped reads by `workspace_id`. Enable it only with a full rebuild of those projection indexes; routed searches will not see documents previously indexed without routing. - `AIBS_RETRIEVAL_PYRAMID_DISCOVERY_ENABLED=true` makes broad discovery read profile/summary hits before chunk evidence. Keep it `false` for the clean pyramid-off baseline; enable it only for the pyramid-on A/B run. - keep `AIBS_OPENSEARCH_SSL` and `AIBS_OPENSEARCH_VERIFY_CERTS` aligned with the actual cluster you connect to - use real credentials only in your local `.env`, never in docs or committed files - if your OpenSearch cluster lives on a VM, you can forward it locally over SSH and still keep: - `AIBS_OPENSEARCH_HOST=localhost` - `AIBS_OPENSEARCH_PORT=9200` - example tunnel: ```powershell ssh -fN -L 9200::9200 @ ``` ### Embeddings - the repository is already set up for the local BGE-M3 embedding model - `AIBS_EMBEDDING_LOCAL_FILES_ONLY=true` is the safest default for offline or controlled environments - the current local `.env` pins `AIBS_EMBEDDING_DEVICE=cuda`; use `auto` only when you want portable CPU/GPU fallback across machines - supported explicit device values are `cpu`, `cuda`, `cuda:N`, `gpu`, `gpu:N`, and `auto` - explicit `cuda`/`gpu` values fail fast if `torch.cuda` is not available; use `auto` for portable local and VM configs - AMD/ROCm PyTorch exposes AMD GPUs through the `torch.cuda` compatibility API, so do not use a separate `rocm` value ### Reranker - reranking improves retrieval quality but depends on the local BGE reranker bundle - if you do not want reranking during a debugging session, set `AIBS_RERANKER_ENABLED=false` - `AIBS_RERANKER_MAX_CANDIDATES` controls how many assembled candidates are rescored by the cross-encoder. The demo/default value is `80`, matching the fast multi-document evidence cap so broad pages do not leave selected evidence in an unreredanked tail. - `AIBS_RERANKER_DIAGNOSTIC_TEXT_PREVIEW_ENABLED=false` keeps reranker diagnostic logs to document/chunk identifiers and scores. Enable it only in private dev investigations when text previews are acceptable in logs. - `AIBS_RERANKER_SCORE_ACTIVATION=sigmoid` converts BGE reranker logits into probability-like scores before sufficiency checks. Use `identity` only when a reranker model already returns calibrated scores on the scale you want to evaluate directly. - `AIBS_RERANKER_DEVICE` accepts the same device values as embeddings: `cpu`, `cuda`, `cuda:N`, `gpu`, `gpu:N`, and `auto` - keep `AIBS_RERANKER_DEVICE=auto` unless you intentionally want startup to fail when CUDA/ROCm torch acceleration is unavailable ### Docling PDF parser - if `AIBS_DOCLING_ARTIFACTS_PATH` is set, the parser passes that folder to Docling as its local `artifacts_path` - this is the right setup if you want document parsing to stay independent from live Hugging Face availability after the artifacts have been downloaded locally - the current recommended repo-local path is `models/docling` - the current local `.env` pins `AIBS_DOCLING_DEVICE=cuda` so parsing fails fast if the expected CUDA/ROCm torch acceleration is unavailable - use `AIBS_DOCLING_DEVICE=auto` when you want the parser to fall back to CPU on machines without GPU-capable torch - `AIBS_DOCLING_NUM_THREADS` and the `AIBS_DOCLING_*_BATCH_SIZE` values cap Docling's page-processing pressure; the template favors smoother ingestion over peak throughput - `AIBS_DOCLING_QUEUE_MAX_SIZE` limits Docling's internal queued work and helps avoid memory spikes on large PDFs - use a lower value such as `8` on laptop/local-dev machines that share memory with the API, embedding, reranker, and LLM clients; keep higher values such as `50` for dedicated AI-box style deployments that have enough memory headroom - `AIBS_PARSING_ENABLE_FORMULA_MODEL=false` is the safe default; formula enrichment is expensive and remains disabled unless this flag is explicitly set to `true` - when formula enrichment is enabled, the parser still gates CodeFormulaV2 behind the PDF formula signal threshold, so ordinary PDFs do not load the formula model unnecessarily - with `AIBS_DOCLING_ARTIFACTS_PATH=models/docling`, CodeFormulaV2 must be available locally under `models/docling/docling-project--CodeFormulaV2` before enabling formula enrichment - when the downloaded RapidOCR bundle uses Docling's older `*_infer` filenames, the backend now creates the compatible torch aliases it needs during startup so OCR warmup can still stay local ### Chunking - `AIBS_CHUNKING_ABSTRACTIVE_SUMMARIES_ENABLED=true` enables the optional assistant-backed summarization stage during chunking in the current local `.env` - set it to `false` only when you intentionally want deterministic extractive summaries without model enrichment ### Ingestion worker - `AIBS_INGESTION_WORKER_ENABLED` controls whether the API process starts its embedded ingestion worker - `AIBS_INGESTION_WORKER_COUNT` controls how many worker lanes run inside the process; set it to `0` to pause API-hosted ingestion without disabling job submission - `AIBS_INGESTION_WORKER_CLAIM_SCAN_SIZE` controls how many claimable jobs a lane scans when an OpenSearch optimistic-concurrency conflict occurs - `AIBS_INGESTION_MAX_CONCURRENT_PARSE_STAGES` bounds concurrent parse/OCR stages across worker lanes in one process - `AIBS_INGESTION_MAX_CONCURRENT_CHUNK_STAGES` bounds concurrent chunk/summarization stages across worker lanes in one process - `AIBS_INGESTION_MAX_CONCURRENT_INDEX_STAGES` bounds concurrent embedding/indexing stages across worker lanes in one process - `AIBS_INGESTION_MAX_CONCURRENT_SEMANTIC_PROFILE_STAGES` bounds post-index semantic profile enrichment across worker lanes; documents are already searchable before this stage completes - `AIBS_INGESTION_MAX_CONCURRENT_HEAVY_MODEL_STAGES` bounds local heavyweight model stages shared by parse/OCR, embedding/indexing, and memory-heavy chunking so they do not compete for the same GPU/torch memory at the same time - `AIBS_INGESTION_MEMORY_HEAVY_PAGE_COUNT_THRESHOLD`, `AIBS_INGESTION_MEMORY_HEAVY_TABLE_COUNT_THRESHOLD`, `AIBS_INGESTION_MEMORY_HEAVY_SECTION_COUNT_THRESHOLD`, and `AIBS_INGESTION_MEMORY_HEAVY_CHUNK_COUNT_THRESHOLD` classify expanded document artifacts that should reserve the heavy-model lane during chunking; this prevents annual-report-size documents from chunking while another heavy parse is active - `AIBS_INGESTION_WORKER_IDLE_SLEEP_SECONDS` controls how long the background worker sleeps when no job is available - `AIBS_INGESTION_WORKER_ERROR_SLEEP_SECONDS` controls the backoff after worker-loop errors - `AIBS_INGESTION_WORKER_LEASE_REAP_INTERVAL_SECONDS` controls how often an idle worker reaps expired job leases so jobs abandoned by another process become retryable without requiring an API restart - `AIBS_INGESTION_EXPIRED_LEASE_CLAIM_MULTIPLIER` bounds true worker crash loops separately from caught stage failures; with the default `10`, a job with `max_attempts=3` can survive up to roughly `30` expired-lease claim cycles before the reaper marks it failed as a suspected worker crash loop - `AIBS_INGESTION_UPLOAD_QUIET_PERIOD_SECONDS` pauses API-hosted ingestion claims while document upload requests are active and for the configured quiet window afterward; this is only a fallback for single uploads and clients that do not open an explicit batch - `AIBS_INGESTION_UPLOAD_BATCH_TTL_SECONDS` is the abandoned-browser failsafe for explicit upload batches; each file upload refreshes the batch, and explicit batch completion releases API-hosted ingestion immediately - `AIBS_INGESTION_JOB_INDEX` is the operational queue for durable ingestion jobs, not just a storage name; it is a literal OpenSearch index name and is not automatically derived from `AIBS_ENV` - if `AIBS_INGESTION_JOB_INDEX` is unset, the backend default is `aibs-ingestion-jobs`; the sample `.env` uses `aibs-ingestion-jobs-dev` only as a development namespace convention - use a unique value per local developer, branch sandbox, VM, staging, and production deployment unless all API/worker processes are intentionally cooperating on the same ingestion queue - do not run production or demo VM ingestion against `aibs-ingestion-jobs-dev` unless that is deliberately the shared queue for that deployment - sharing OpenSearch credentials or search indices is not the same as sharing ingestion ownership; any process polling the same `AIBS_INGESTION_JOB_INDEX` can claim and mutate jobs in that queue - keep these values conservative on the demo VM to avoid unnecessary polling pressure - set `AIBS_INGESTION_WORKER_ENABLED=false` only for a deliberate API-only deployment where a separate worker process/node owns parse/chunk/index execution; do not use it as a bulk-upload workaround - run the standalone worker with `python -m aibs_backend.interfaces.cli.ingestion_worker` on the worker node; the CLI runs when invoked and still uses the sleep/reap timings above ### Conversation memory - `AIBS_CONVERSATION_MEMORY_ENABLED` enables Phase 5 Slice A conversation context. - recent turns are read from authoritative query/answer JSON history, not from the memory index. - `AIBS_CONVERSATION_MEMORY_INDEX` is the separate OpenSearch index used only for derived older-turn retrieval. - `AIBS_CONVERSATION_MEMORY_RECENT_TURNS` controls how many latest persisted turns are placed in the planner and answer prompts. - `AIBS_CONVERSATION_MEMORY_RETRIEVAL_TOP_K` controls how many older relevant turns are retrieved for the answer prompt; these are intentionally not shown to the planner. - `AIBS_CONVERSATION_MEMORY_MAX_TURN_CHARS` clips each turn before it is embedded or inserted into prompts. - memory indexing is best effort and logs warning telemetry on failure; deleting a conversation purges memory first and fails loudly if that purge fails. ### Answer pagination - `AIBS_ANSWER_PAGE_DOCUMENT_LIMIT` controls how many ranked documents are sent into one grounded answer page before the backend returns a continuation token. The default is `10` for Gemma4:26b-era context discipline; raise it only after benchmark evidence shows the larger model/context budget can preserve answer quality. - `AIBS_ANSWER_CONTINUATION_TOKEN_TTL_SECONDS` controls how long continuation tokens remain valid on disk. The default is `86400` seconds (24 hours), which keeps browser reloads and long automatic continuation chains usable while reaping orphaned tokens from failed or abandoned runs. - `AIBS_ANSWER_MAP_CONCURRENCY_LIMIT` caps parallel per-document map calls when a large evidence page uses map-reduce answering. The global agent-flow scheduler may reduce this request-local ceiling when current node capacity is lower. The default is `5`; raise it only after VM benchmarks show the worker fleet handles the concurrency. - `AIBS_DOCUMENT_WORK_MODEL_DEADLINE_SECONDS` is the absolute wall-clock budget for all model attempts on one map/refine document. The default is `45` seconds. An assistant exceeding it is cancelled, its model lease is released, and the document is published as a typed reliability gap instead of holding the complete answer indefinitely. This is distinct from the HTTP inactivity timeout, which may reset while a stalled provider streams data. - `AIBS_ANSWER_MAP_REDUCE_EVIDENCE_LIMIT` controls when a large evidence pack switches from single-pass answering to map-reduce answering. The default is `40` evidence items. - `AIBS_ANSWER_FAST_MULTI_DOC_EVIDENCE_CAP` controls how many evidence items a fast-tier multi-document query may retrieve before mapping. This is separate from the map-reduce trigger, so broad non-heavy questions can carry more evidence without changing when map-reduce activates. Keep it at or above `AIBS_ANSWER_MAP_REDUCE_EVIDENCE_LIMIT`. The default is `80`. - `AIBS_ANSWER_MULTI_DOC_CHUNKS_PER_DOCUMENT` controls the per-document evidence target before the total cap is applied. The demo/default value is `8`, so a 10-document broad page can use the full `80`-item fast multi-document cap. - `AIBS_ANSWER_MAP_REDUCE_FALLBACK_CHUNK_LIMIT` triggers map-reduce when too many raw chunk excerpts are needed because summaries are missing, degraded, or too thin. The default is `20` chunk excerpts. - `AIBS_STRICT_GROUNDED_QUARANTINED_MODEL_IDS` is the comma-separated deny-list for strict Grounded map, refine, and compose model admission. Quarantined models remain available to non-strict Creative work. The default excludes `gpt-oss:20b` until contract and latency evaluations explicitly clear it. - `AIBS_ANSWER_METRICS_SINCE` resets operational answer metrics without deleting persisted answers. Set it to an ISO-8601 UTC timestamp such as `2026-07-07T10:25:45Z`; answer counts, token totals, model token distribution, and the answer trend ignore answers created before that boundary. Leave it blank to report over the normal recent-answer window. - `AIBS_DOCUMENT_DISCOVERY_MIN_ENTITY_ANCHOR_DOCUMENTS` and `AIBS_DOCUMENT_DISCOVERY_MIN_UNDELIMITED_ENTITY_ANCHOR_DOCUMENTS` control the catalog entity-bound fast path. Delimited identity references such as quoted, parenthesized, or bracketed document names must all resolve before the fast path is allowed, and the first value is the minimum number of anchored documents for that mode. Undelimited identity references use the second, stricter floor so a broad topic/category query does not steal scope from semantic/profile discovery. Defaults: `2`, `3`. - `AIBS_DOCUMENT_PROFILE_BINDING_TIMEOUT_SECONDS` bounds profile-index lookup when fast-binding explicit document references. Catalog/title binding runs first; slow profile lookup is skipped after this budget and normal discovery continues. The default is `3` seconds. - `AIBS_PLANNING_CATALOG_CACHE_TTL_SECONDS` controls the short in-process cache for dispatcher planning catalogs loaded from lightweight catalog projection entries. The default is `0` (disabled) because knowledge enable/disable and delete operations require a shared invalidation/version signal before this can be safely enabled in multi-worker production. - `AIBS_PLANNING_CATALOG_CACHE_MAX_WORKSPACES` bounds the optional in-process planning-catalog cache when the TTL is enabled. The default is `64`. ### Document Complexity Document-local map work is classified from intrinsic document metadata before a worker is admitted. This avoids treating a 2-page table-heavy file and a 150-page narrative file as equivalent work. - `AIBS_DOCUMENT_COMPLEXITY_MEDIUM_PAGE_COUNT` and `AIBS_DOCUMENT_COMPLEXITY_LARGE_PAGE_COUNT` classify size pressure from parsed page count. Defaults: `25`, `100`. - `AIBS_DOCUMENT_COMPLEXITY_MEDIUM_CHUNK_COUNT` and `AIBS_DOCUMENT_COMPLEXITY_LARGE_CHUNK_COUNT` classify size pressure from indexed chunk count. Defaults: `120`, `500`. - `AIBS_DOCUMENT_COMPLEXITY_MEDIUM_SECTION_COUNT` and `AIBS_DOCUMENT_COMPLEXITY_LARGE_SECTION_COUNT` classify section-rich documents. Defaults: `50`, `200`. - `AIBS_DOCUMENT_COMPLEXITY_MEDIUM_TABLE_COUNT` and `AIBS_DOCUMENT_COMPLEXITY_LARGE_TABLE_COUNT` classify table-heavy documents. Defaults: `4`, `20`. - `AIBS_DOCUMENT_COMPLEXITY_MEDIUM_REQUEST_SIGNAL_COUNT` and `AIBS_DOCUMENT_COMPLEXITY_LARGE_REQUEST_SIGNAL_COUNT` classify typed request complexity from the number of requested fields and subjects, not from natural-language keywords. Defaults: `4`, `10`. - `AIBS_DOCUMENT_COMPLEXITY_SMALL_EVIDENCE_BUDGET`, `AIBS_DOCUMENT_COMPLEXITY_MEDIUM_EVIDENCE_BUDGET`, `AIBS_DOCUMENT_COMPLEXITY_LARGE_EVIDENCE_BUDGET`, and `AIBS_DOCUMENT_COMPLEXITY_DEGRADED_LARGE_EVIDENCE_BUDGET` bound how many evidence items a document-local worker receives for each complexity class. Defaults: `4`, `8`, `12`, `16`. - `AIBS_DOCUMENT_COMPLEXITY_PAGE_CONSTRAINED_EVIDENCE_BUDGET` keeps page/section-constrained work compact while still giving the worker enough local context. Default: `6`. - `AIBS_DOCUMENT_COMPLEXITY_STRUCTURED_OUTPUT_MIN_EVIDENCE_BUDGET` gives chart and table outputs a minimum evidence budget when enough evidence exists. Default: `10`. ### Heavy-tier multi-doc combine The execution planner's intent signals classify each request into a latency tier. Fast queries (facts, inventory, single-doc, small multi-doc) keep the tight `AIBS_ANSWER_PAGE_DOCUMENT_LIMIT` page budget. Broad discovered/planner scopes also page first so users receive an inspectable first answer and continuation token instead of waiting for a monolithic all-document synthesis. Explicit selected-document heavy queries ("compare these named documents") can cover the selected set in one combined answer: - `AIBS_ANSWER_HEAVY_COMBINE_DOCUMENT_LIMIT` is the maximum number of documents a single explicit heavy combine folds in before it falls back to paging. The default is `64`. Retrieval/document binding already narrows the selected set, so this bounds work, not the workspace size. Raise it only if your model/context budget can synthesize more documents at once; lower it to cap explicit heavy-query latency. - `AIBS_ANSWER_HEAVY_EVIDENCE_CHUNK_CAP` is the total evidence-chunk budget for a heavy combine, so every document keeps its full chunks-per-document share instead of being thinned to fit the fast-tier evidence cap (`AIBS_ANSWER_MAP_REDUCE_EVIDENCE_LIMIT`). The default is `320`. It does not change when map-reduce is triggered, only how much evidence a heavy combine may gather. - `AIBS_SCOPED_MULTI_DOC_CANDIDATE_MULTIPLIER_CAP` caps reranker/search overfetch after the document set is already known. It does not change document scoping. Higher values preserve sparse-document recall at higher latency; lower values reduce latency after broad scopes. The default is `4`. ### Workspace document inventory - `AIBS_CATALOG_INVENTORY_LIST_LIMIT` controls how many documents the deterministic workspace document table (the `WORKSPACE ANSWER` catalog listing) shows per page. The default is `25`. When the workspace holds more documents than one page, the answer states the true total (for example, "Showing the first 25 of 930 documents in this workspace") and returns a continuation token so the client can request the next page ("show more"). Raise it to show more rows per page; lower it for tighter responses. It does not affect grounded answering, which is governed by `AIBS_ANSWER_PAGE_DOCUMENT_LIMIT`. ### Remote GPU Worker Environment Use this split when the API node should accept uploads and submit durable jobs, while a separate GPU worker node claims and executes parse/chunk/index. API node: ```text AIBS_INGESTION_WORKER_ENABLED=false ``` GPU worker node: ```text AIBS_DOCLING_ARTIFACTS_PATH=/opt/aibs/models/docling AIBS_EMBEDDING_MODEL=BAAI/bge-m3 AIBS_EMBEDDING_MODEL_PATH=/opt/aibs/models/embeddings/bge-m3 AIBS_EMBEDDING_DIMENSION=1024 AIBS_RERANKER_MODEL_PATH=/opt/aibs/models/reranking/bge-reranker-v2-m3 AIBS_RERANKER_MAX_CANDIDATES=80 AIBS_RERANKER_SCORE_ACTIVATION=sigmoid AIBS_DOCLING_DEVICE=cuda AIBS_EMBEDDING_DEVICE=cuda AIBS_RERANKER_DEVICE=auto ``` `AIBS_INGESTION_WORKER_ENABLED` is only used by the FastAPI lifespan to decide whether the API process should start its embedded worker. The standalone worker CLI always runs when invoked, so the GPU worker env does not need that flag. The GPU worker must use the same OpenSearch service as the API process and must be able to read/write the upload and document storage roots that ingestion actually touches. For the demo topology where the VM owns storage and the AI-on-prem box is compute-only, mount only the VM-owned ingestion roots on the GPU worker. Do **not** mount over the GPU server's whole `/opt/aibs` directory, because that path also holds the local model cache and worker virtual environment: ```text GPU server local runtime: /opt/aibs/models /opt/aibs/models/.venvs/aibs-dependecy-gpu-ingestion VM-owned shared data mounted on GPU server: /opt/aibs/.aibs_data/uploads /opt/aibs/.aibs_data/documents ``` These two paths are the required ingestion filesystem boundary: ```text AIBS_UPLOAD_ROOT=.aibs_data/uploads AIBS_DOCUMENT_ROOT=.aibs_data/documents ``` If the worker might be launched from another directory, prefer absolute paths for those two roots on the worker node: ```text AIBS_UPLOAD_ROOT=/opt/aibs/.aibs_data/uploads AIBS_DOCUMENT_ROOT=/opt/aibs/.aibs_data/documents ``` `AIBS_WORKSPACE_ROOT`, `AIBS_CONVERSATION_ROOT`, `AIBS_QUERY_ROOT`, `AIBS_ANSWER_ROOT`, and `AIBS_STAGING_ROOT` are API-side storage concerns for workspace management, conversations, query records, answer artifacts, and upload staging. The standalone ingestion worker does not need shared access to them for parse/chunk/index. Mounting the parent `.aibs_data` directory is still acceptable for operational simplicity, as long as it does not overwrite `/opt/aibs/models`. The worker also needs to point at the same OpenSearch service used by the API. OpenSearch is a separate search/storage boundary and may live on its own VM or service host. Do not infer the OpenSearch host from the GPU/LLM/model server IP unless OpenSearch is actually deployed there. In this codebase OpenSearch is configured with the `AIBS_OPENSEARCH_*` variables, not `OPENSEARCH_URL`: ```text AIBS_OPENSEARCH_HOST= AIBS_OPENSEARCH_PORT=9200 AIBS_OPENSEARCH_SSL=false AIBS_OPENSEARCH_VERIFY_CERTS=false AIBS_OPENSEARCH_USERNAME= AIBS_OPENSEARCH_PASSWORD= AIBS_OPENSEARCH_INDEX_TEMPLATE=.abs-content-{workspace_id} AIBS_INGESTION_JOB_INDEX=aibs-ingestion-jobs-gpu-worker AIBS_DOCUMENT_CATALOG_INDEX=aibs-document-catalog AIBS_DOCUMENT_PROFILE_INDEX=aibs-document-profiles AIBS_SECTION_SUMMARY_INDEX=aibs-section-summaries AIBS_RETRIEVAL_PYRAMID_DISCOVERY_ENABLED=true ``` On AMD/ROCm boxes, `cuda` and `auto` both route through PyTorch's `torch.cuda` compatibility layer. Keep the current pinned `cuda` values when you want fail-fast GPU startup; use `auto` for portable worker images that may run on CPU-only hosts. For the current AI-on-prem Ryzen APU worker, the validated Python runtime uses AMD's ROCm 7.2.1 PyTorch wheels from `repo.radeon.com`, not the public PyTorch ROCm wheels: ```text torch 2.9.1+rocm7.2.1.lw.gitff65f5bc torchvision 0.24.0+rocm7.2.1.gitb919bd0c triton 3.5.1+rocm7.2.1.gita272dfa8 ``` The worker venv lives under the model folder so it stays isolated from system Python: ```text /opt/aibs/models/.venvs/aibs-dependecy-gpu-ingestion ``` ### Local storage These paths are relative to the repo root by default and back the local JSON/file persistence used in development. - `AIBS_INGESTION_JOB_INDEX` is the OpenSearch index used for durable ingestion job state and worker claiming. Treat it as a queue namespace, not an environment-aware setting: `AIBS_ENV=production` does not rename the queue. Use one shared value only for processes that are meant to cooperate on the same jobs, and use separate values for local developer machines, branch sandboxes, demo VMs, staging, production, and GPU worker environments that must not claim each other's work. - `AIBS_DOCUMENT_CATALOG_INDEX` is the OpenSearch index used for the workspace document catalog projection. It is not the aggregate store; rebuild it from JSON documents if it is missing or stale. - `AIBS_DOCUMENT_PROFILE_INDEX` is the OpenSearch profile projection used before broad chunk retrieval. - `AIBS_SECTION_SUMMARY_INDEX` is the OpenSearch section/table summary projection used before broad chunk retrieval. - `AIBS_PROJECTION_CONTROL_INDEX` is the OpenSearch control-plane index used for the catalog projection manifest and mutation outbox. The default is `aibs-projection-control`. Keep this shared by every API and ingestion worker process that reads/writes the same document store. A missing or dirty manifest intentionally makes broad planning fall back to JSON source truth until reconciliation marks the projection clean. - `AIBS_PROJECTION_CONTROL_SUCCEEDED_EVENT_RETENTION_DAYS` controls pruning for terminal `succeeded` catalog projection outbox events after successful catalog reconciliation. The default is `7`; set `0` to retain succeeded events indefinitely. - `AIBS_PROJECTION_CONTROL_RECONCILED_EVENT_RETENTION_DAYS` controls pruning for terminal `reconciled` outbox events. The default is `20`; set `0` to retain reconciled events indefinitely. - Pending and failed events are never pruned by this retention policy. They are unresolved drift/audit signals and must remain visible until reconciliation converts them to terminal `reconciled` events. - `AIBS_PROJECTION_WORKSPACE_ROUTING_ENABLED` optionally routes catalog/profile/summary projection documents by `workspace_id`. Leave it `false` unless you are prepared to rebuild `AIBS_DOCUMENT_CATALOG_INDEX`, `AIBS_DOCUMENT_PROFILE_INDEX`, and `AIBS_SECTION_SUMMARY_INDEX` after enabling it. ### Document Catalog Rebuild CLI The catalog projection is write-through for new document saves and deletes, with retry and best-effort logging if OpenSearch is briefly unavailable. Existing workspaces, restored backups, renamed catalog indexes, or suspected projection drift need a rebuild: ```powershell python -m pdm run python -m aibs_backend.interfaces.cli.rebuild_document_catalog [--workspace ] [--batch-size 200] [--dry-run] ``` The command loads `.env` from the current working directory and writes to the OpenSearch index named by `AIBS_DOCUMENT_CATALOG_INDEX`. Successful non-`--only-missing` rebuilds also reconcile the catalog projection manifest in `AIBS_PROJECTION_CONTROL_INDEX`. This establishes the clean baseline required by the dispatcher to trust projection-backed broad planning. `--only-missing` does not clear dirty state because it cannot prove stale rows were removed. If the catalog mapping schema is outdated, run a full rebuild without `--workspace`. The full rebuild drops and recreates only the catalog projection from JSON source-of-truth documents. A workspace-scoped rebuild refuses stale schemas because the catalog index is shared across workspaces. ### Document Catalog Reconciliation CLI Use this command when the dispatcher logs dirty catalog projection fallback, after restoring backups, after changing projection indexes, or before enabling projection-first planning for existing workspaces: ```powershell python -m pdm run python -m aibs_backend.interfaces.cli.reconcile_document_catalog_projection [--workspace ] [--batch-size 200] ``` The reconciler replays the authoritative JSON documents into the catalog projection, removes stale projection rows, refreshes OpenSearch, and marks the workspace catalog manifest clean. It does not offer an `--only-missing` mode because partial repair cannot prove broad completeness. ### Retrieval Pyramid Rebuild CLI The retrieval pyramid projections are write-through for new document saves and deletes. Existing workspaces, restored backups, renamed projection indexes, or suspected drift need a rebuild: ```powershell python -m pdm run python -m aibs_backend.interfaces.cli.rebuild_retrieval_pyramid [--workspace ] [--batch-size 200] [--dry-run] ``` The command writes to `AIBS_DOCUMENT_PROFILE_INDEX` and `AIBS_SECTION_SUMMARY_INDEX`. If either mapping schema is outdated, run a full rebuild without `--workspace`; scoped rebuilds refuse stale schemas because the indexes are shared across workspaces. ### Auth and identity Auth uses OpenSearch credentials and stores an opaque server-side session id in an HTTP-only cookie. AIBS product access still requires `aibs-access`, and workspace operations are then governed by the workspace role matrix. The OpenSearch auth adapter supports standard OpenSearch Security authinfo responses and Logserver clusters that authenticate through `/_logserver/login`. ```ini AIBS_AUTH_DEV_MODE=true AIBS_AUTH_DEV_ROLES=aibs-access,aibs-admin,developer AIBS_AUTH_SESSION_COOKIE_NAME=aibs-session AIBS_AUTH_SESSION_TTL_SECONDS=3600 AIBS_AUTH_COOKIE_SECURE=false AIBS_AUTH_COOKIE_SAMESITE=lax AIBS_AUTH_COOKIE_PATH=/ # AIBS_AUTH_COOKIE_DOMAIN= ``` - `AIBS_AUTH_DEV_MODE=true` allows the legacy `X-AIBS-User-Id` header for local development only. - `AIBS_AUTH_DEV_ROLES` controls the exact roles assigned by the development auth stub. Keep production role grants in OpenSearch/Logserver; do not use dev-mode roles outside local development. - For local operator testing of admin/developer-only diagnostics, such as System Health node ids and model-node endpoint visibility, include the required roles explicitly, for example: ```ini AIBS_AUTH_DEV_ROLES=aibs-access,aibs-admin,developer ``` After changing dev-mode roles, restart the API and sign in again so the process-local session contains the updated principal roles. - When `AIBS_AUTH_DEV_MODE=false`, the header is ignored and requests require a valid session cookie. - Browser sessions use a fixed TTL in this phase and are process-local. Keep API deployment single-worker until a persistent session store lands. - Bootstrap the first production admin before first login with: ```powershell python -m pdm run python -m aibs_backend.interfaces.cli.bootstrap_admin --username ``` - If the user does not exist, the CLI prompts for a password and creates the user with `aibs-access` and `aibs-admin`. - If the user already exists, the CLI grants missing AIBS roles idempotently. - Passwords are read through a masked prompt and are never printed. - The CLI loads `.env` from the current working directory before reading `AIBS_OPENSEARCH_*` settings. - `AIBS_IDENTITY_HEADER_NAME` remains only for the explicit dev-mode compatibility path. - Workspace-local roles are `owner`, `admin`, `member`, and `viewer`. Owners can delete workspaces and manage ownership, owners/admins can manage non-owner members and documents, members can upload/process documents, and viewers can read workspace state and ask questions. ### Language Backfill CLI Newly ingested documents have their primary language detected automatically during parse. Documents ingested before language detection joined the pipeline carry no `language` metadata and appear under `unknown` in the workspace inventory `by_language` breakdown. Populate the metadata for those documents with: ```powershell python -m pdm run python -m aibs_backend.interfaces.cli.backfill_language [--workspace ] [--dry-run] [--force] ``` - with no flags the CLI processes every workspace - `--workspace ` restricts the run to a single workspace - documents that already have language metadata are skipped unless `--force` is passed - `--dry-run` prints what would be persisted without writing; every output line is prefixed with `[DRY-RUN]` - the CLI uses the same `JsonDocumentRepository` and `LinguaLanguageDetector` the running backend uses, so the `.env` and `AIBS_*` settings above govern its behavior - AIBS product user administration requires both `aibs-access` and `aibs-admin`. - Product admins can create users, reset passwords, toggle `aibs-access`, toggle `aibs-admin`, and disable users when the backing Logserver/OpenSearch provider supports it. - Workspace member management is separate from product user administration. Users should be created or granted `aibs-access` first, then added to a workspace by username. - Workspace member-add still accepts `user_id` as a deprecated compatibility field, but the intended operator path is username-based and directory-backed. - A valid Logserver user without `aibs-access` should receive `AIBS_ACCESS_REQUIRED` in non-dev mode. That is an access-policy failure, not an invalid password. ### Logging The backend writes: - application logs to `logs/aibs.log` - ingestion logs to `logs/ingestion.log` - retrieval logs to `logs/retrieval.log` ### LLM - `AIBS_LLM_PROVIDER` is currently expected to be `ollama` - `AIBS_LLM_BASE_URL` should point to your Ollama-compatible endpoint - the current demo/default example above uses the dedicated ELS Compass node at `10.4.3.16:11434`; global placement also admits `10.4.3.17:11434` - `AIBS_LLM_MODEL` can be switched between available server models such as `gemma4:26b` and `gpt-oss:20b` depending on the quality/latency tradeoff you want to test - `AIBS_LLM_MAX_CONCURRENT_REQUESTS` serializes assistant calls through a concurrency gate; keep it at `1` when the local serving lane handles one request at a time - `AIBS_LLM_TEMPERATURE`, `AIBS_LLM_TOP_P`, and `AIBS_LLM_TOP_K` map directly to Ollama sampling options; the template keeps them aligned with Gemma 4's recommended conversational sampling settings - `AIBS_LLM_THINKING_ENABLED=true` enables provider thinking for natural-language assistant calls in the current local `.env`; the backend exposes returned thinking separately from answer text for inspection - `AIBS_LLM_THINKING_STRATEGY=auto` uses Gemma 4's documented system-token strategy for `gemma4*` models and Ollama's native `think=true` field for other models; explicit values are `ollama` and `gemma4_system_token` - structured JSON calls, such as execution planning, explicitly send `think=false` even when this flag is true so machine-readable response contracts stay reliable - if a structured JSON call still returns provider thinking without final message content, the backend retries once with a stricter final-content instruction - `AIBS_LLM_THINKING_MAX_OUTPUT_TOKENS` and `AIBS_LLM_THINKING_CONTEXT_LENGTH` optionally give thinking-enabled answer calls a larger output and context budget than normal calls - if a thinking-enabled natural-language call returns thinking but no final answer content, the backend retries that same call once with thinking disabled and keeps the original thinking trace as response metadata - `AIBS_LLM_MAX_OUTPUT_TOKENS` maps to the Ollama output-token option and should be high enough for descriptive grounded answers - `AIBS_LLM_CONTEXT_LENGTH` is optional and maps to the Ollama context-length option when set - chat execution planning now uses the main answer model from `AIBS_LLM_MODEL` so routing, document scope, metadata facts, language, and answer shape come from one typed plan - `AIBS_LLM_STARTUP_CHECK=true` is useful during setup because the app will fail fast if the LLM is unreachable - `AIBS_LLM_CAPABILITY_CONTEXT` is optional; if omitted, the backend uses the branch-default capability context - `AIBS_LLM_SUMMARIZER_*` controls the optional ingestion-time abstractive summarizer lane used when `AIBS_CHUNKING_ABSTRACTIVE_SUMMARIES_ENABLED=true` - summarizer provider, base URL, model, timeout, output, context, keep-alive, top-p, and top-k fall back to the matching `AIBS_LLM_*` value when omitted - `AIBS_LLM_SUMMARIZER_MAX_CONCURRENT_REQUESTS` is a separate concurrency gate from chat, so long-running document enrichment does not block user answers - `AIBS_LLM_SUMMARIZER_TEMPERATURE=0.0` is the recommended default because enrichment writes retrieval metadata and should be stable - `AIBS_LLM_SEMANTIC_PROFILE_*` controls the dedicated semantic-profile enrichment lane. Its default model is `qwen3:8b`; it is intentionally separate from `AIBS_LLM_SUMMARIZER_*` so profile backfills and ingestion enrichment do not contend with the summarizer/reduce lane used by live answers. - `AIBS_LLM_SEMANTIC_PROFILE_MAX_CONCURRENT_REQUESTS` is the per-model gate for the qwen semantic-profile lane. Role-level admission still coordinates this lane with the global model-residency planner. - `AIBS_LLM_SEMANTIC_PROFILE_ADMISSION_QUEUE_TIMEOUT_SECONDS` controls how long semantic enrichment waits for a model-admission slot. Keep this much higher than the interactive query admission timeout because backfills are expected to serialize behind the same resident model lane instead of failing fast. - `AIBS_LLM_CONVERSATION_TITLE_*` controls asynchronous first-message title enrichment. The title generator should use a small structured-extraction model such as `qwen2.5:7b-instruct`; it only updates conversation metadata, never the query text, reply language, routing decision, or answer content. - Generated titles are skipped when the user has manually renamed the conversation, and model-admission busy states leave the deterministic placeholder title in place instead of delaying the chat. ### LLM heavy reasoning lane The execution planner emits `reasoning_effort=high` for business/research and deep cross-document analysis; those queries route to the heavy tier. The `AIBS_LLM_HEAVY_*` lane lets that tier use a more capable model with its own budget, while fast queries stay on the base `AIBS_LLM_*` model. Every field falls back to the matching `AIBS_LLM_*` value when omitted, so the lane is **dormant by default**. - `AIBS_LLM_HEAVY_MODEL` activates a distinct heavy serving lane only when set to a model different from `AIBS_LLM_MODEL`. Keep it unset/blank unless a measured residency plan proves the distinct model can stay resident safely. When it equals `AIBS_LLM_MODEL` (or is unset) the heavy tier reuses the base answer lane and its concurrency gate, so a single model is never double-gated. If a future large heavy model is needed, prefer a measured Apache-licensed candidate such as Qwen3-30B-A3B over reintroducing the retired 31B Gemma lane. - When the heavy tier reuses the base answer lane, heavy sampling/output/context values are applied as per-call generation overrides on that shared assistant. This preserves the heavy answer budget without creating a second same-model residency or concurrency lane. This is a bridge for the current chat pipeline; broad or heavy work should still move toward Agent Mode execution over typed work items rather than one larger prompt. - `AIBS_LLM_HEAVY_TEMPERATURE`, `AIBS_LLM_HEAVY_TOP_P`, `AIBS_LLM_HEAVY_TOP_K`, `AIBS_LLM_HEAVY_MAX_OUTPUT_TOKENS`, `AIBS_LLM_HEAVY_CONTEXT_LENGTH`, override the base sampling/output/context for the heavy tier. When provider thinking is enabled, the heavy output/context values also become the thinking output/context budget for the call. - `AIBS_LLM_HEAVY_KEEP_ALIVE` should stay blank while the heavy tier shares `AIBS_LLM_MODEL`. Ollama residency is per model and last-request-wins; passing a shorter heavy keep-alive for the shared model would evict the base answer model earlier than `AIBS_LLM_KEEP_ALIVE` intends. Configure a heavy keep-alive only when `AIBS_LLM_HEAVY_MODEL` names a distinct model with a measured residency plan. - `AIBS_LLM_HEAVY_THINKING_ENABLED` turns provider thinking on for the heavy lane independently of the base lane, since heavy reasoning benefits from it. - The heavy lane shares the base `AIBS_LLM_MAX_CONCURRENT_REQUESTS` limit. Because a distinct heavy model and the base model are different models, allowing one concurrent call each is correct per-model serialization; keep enough resident memory for both to stay loaded so requests do not trigger model reloads. - Backend request serialization is not the same thing as Ollama model residency. Ollama can keep several model runners loaded after requests finish, and other clients connected to the same Ollama host bypass this backend's concurrency gate. Before retrieval-latency benchmarks, check `ollama ps` and `amd-smi`; ideally only the base model should be hot before the run starts. ### Shared coordination port The backend exposes shared coordination ports for best-effort cache values and safety-critical expiring leases. Local development can use process-local memory, but multi-worker or multi-box deployments must use Valkey so all API workers share one scheduler/cache state. ```ini AIBS_COORDINATION_BACKEND=valkey AIBS_COORDINATION_NAMESPACE=aibs AIBS_COORDINATION_DEFAULT_CACHE_TTL_SECONDS=86400 AIBS_COORDINATION_DEFAULT_LEASE_TTL_SECONDS=300 AIBS_COORDINATION_VALKEY_URL=redis://localhost:6379/0 AIBS_COORDINATION_VALKEY_KEY_PREFIX=aibs:coordination AIBS_COORDINATION_VALKEY_SOCKET_TIMEOUT_SECONDS=3 ``` - `AIBS_COORDINATION_BACKEND=memory` keeps coordination state inside the current API process. This is correct for local development and tests, but it is not a cross-process safety mechanism. - Durable cleanup workers use the application coordination pool with a shorter, renewable lease. Configure their crash-recovery and retry budgets separately: ```ini AIBS_CLEANUP_WORKER_LEASE_TTL_SECONDS=30 AIBS_CLEANUP_WORKER_RETRY_DELAY_SECONDS=5 AIBS_CLEANUP_WORKER_MAX_ATTEMPTS=3 ``` The cleanup lease is renewed while a job is running. Its TTL controls crash failover time, not the maximum duration of a cleanup operation. - `AIBS_COORDINATION_BACKEND=valkey` stores cache and lease state in Valkey. The `redis` backend value remains readable only for configuration compatibility. - Global model placement requires Valkey 8+. The scheduler Lua scripts use server-side `TIME` while writing placement state so all API processes share one clock authority. - `AIBS_COORDINATION_VALKEY_URL` points to the shared Valkey endpoint used by every API worker in the deployment. - `AIBS_COORDINATION_VALKEY_KEY_PREFIX` separates environments that share one Valkey service. - `AIBS_COORDINATION_VALKEY_SOCKET_TIMEOUT_SECONDS` bounds coordination calls so scheduler decisions fail closed instead of hanging indefinitely. When the residency monitor is enabled, it must be shorter than one third of the monitor lease TTL so a single socket wait cannot consume the renewal budget. The three-second default tolerates slower Windows-to-Linux Valkey networking while preserving that safety margin for the default monitor lease. - `AIBS_MODEL_CLUSTER_NODE_MAX_PARALLEL_REQUESTS` is an authoritative per-node ceiling. It must not exceed the serving process's actual `OLLAMA_NUM_PARALLEL`/runner capacity. The scheduler applies the minimum of this node limit, the model lane limit, current flow admission, and memory capacity. - `AIBS_COORDINATION_NAMESPACE` scopes future shared keys so different deployments or model hosts do not accidentally share scheduler/cache state. - Cache users should treat cache failures as best-effort misses. Scheduler users should treat lease failures conservatively because those leases protect model host capacity. - Agent-flow queue and active-flow state are already stored through this coordination boundary. With the memory adapter the behavior remains single-process; with Valkey, the same state path is shared across the deployment. - Document projection freshness also uses this boundary for fallback dirty-state records when the OpenSearch projection control plane cannot record a mutation. In multi-process or multi-box deployments, use Valkey here; the memory adapter cannot carry projection repair state across API processes. - Legacy `AIBS_COORDINATION_REDIS_*` names remain readable during migration, but canonical `AIBS_COORDINATION_VALKEY_*` values take precedence. ### Shared model coordination When separate application deployments share the same physical model nodes, they must share one model-scheduling authority without sharing application coordination. Configure the dedicated model pool in both deployments: ```ini AIBS_MODEL_COORDINATION_BACKEND=valkey AIBS_MODEL_COORDINATION_NAMESPACE=aibs-shared-models AIBS_MODEL_COORDINATION_VALKEY_URL=redis://127.0.0.1:6380/2 AIBS_MODEL_COORDINATION_VALKEY_USERNAME=aibs-model-dev-api AIBS_MODEL_COORDINATION_VALKEY_PASSWORD_FILE=C:\path\to\model-valkey-dev-api.secret AIBS_MODEL_COORDINATION_VALKEY_KEY_PREFIX=aibs:shared:model-coordination AIBS_MODEL_COORDINATION_VALKEY_SOCKET_TIMEOUT_SECONDS=1.5 ``` - Model placement, admission leases, node telemetry, monitor election, and agent-flow admission use this pool. - All deployments sharing model nodes must use the same model coordination namespace, database, and key prefix. - Use a distinct revocable ACL user and password file for each API deployment and monitor. The password file must contain exactly one non-empty secret; the loader URL-encodes it and rejects ambiguous credentials embedded in the URL. - Application coordination must retain a deployment-specific namespace and key prefix. Do not point two environments at the same application-coordination keyspace merely because they share GPUs. - When model scheduling or residency monitoring is enabled, this dedicated block is mandatory. Startup fails if it is absent; application coordination is never inherited as an implicit model-capacity pool. - Migrate every API and monitor that shares the model nodes as one rollout. Start at least one monitor against the new pool first, wait for fresh healthy snapshots, then restart the APIs against exactly the same URL, database, namespace, and prefix. Two live pools create split-brain admission even when they point to the same Valkey server. - An SSH-forwarded URL is suitable for development participation, but the always-on deployment and standalone residency monitor must reach the Valkey service without depending on a developer workstation. ### Best-effort application cache Natural-language typed routing, document discovery, and query embeddings use a separate best-effort cache policy. Keep this cache in the owning deployment's application Valkey, with a separate logical database and key prefix from application coordination. Do not place it in the dedicated shared model-coordination Valkey: cache eviction, cache failure, and environment data must never affect or cross scheduler leases. ```ini AIBS_APPLICATION_CACHE_BACKEND=valkey AIBS_APPLICATION_CACHE_NAMESPACE=aibs-application-cache AIBS_APPLICATION_CACHE_DEFAULT_TTL_SECONDS=300 AIBS_APPLICATION_CACHE_VALKEY_URL=redis://localhost:6379/1 AIBS_APPLICATION_CACHE_VALKEY_KEY_PREFIX=aibs:application-cache AIBS_APPLICATION_CACHE_VALKEY_SOCKET_TIMEOUT_SECONDS=1 ``` - The default Valkey URL uses logical database `1`; coordination defaults to database `0`. - `AIBS_APPLICATION_CACHE_BACKEND=memory` is process-local and is suitable for tests or a single local API process. - Cache reads, writes, and single-flight lease failures degrade to cache misses. Authoritative catalog, conversation, result-set, and scheduler state never depends on this cache. - Conversation history and conversation memory are durable environment-local state, not cache entries. A cache flush may increase latency but cannot erase or alter conversational scope. - Route keys include the normalized question, public-mode/source context, bounded recent turns, authoritative conversation routing state, and router version. Reusing a cached classification across different conversational contexts is therefore forbidden by construction. - Legacy `AIBS_APPLICATION_CACHE_REDIS_*` names remain readable during migration, but canonical `AIBS_APPLICATION_CACHE_VALKEY_*` values take precedence. ### Model admission and residency planning Map-reduce answering can create many role-level model calls from one user question. The model admission boundary is the first control point for that work: each document-map and reduce call asks for a lease before calling the LLM, and whole-query agent-flow admission gates broad map fleets before workers spawn. This prevents one broad query from blindly spawning a full model fleet while another broad query is already running. ```ini AIBS_MODEL_ADMISSION_ENABLED=true AIBS_MODEL_ADMISSION_SAFE_POOL_GB=0 AIBS_MODEL_ADMISSION_QUEUE_TIMEOUT_SECONDS=0.25 AIBS_MODEL_ADMISSION_KV_GB_PER_ACTIVE_BILLION_PER_TOKEN=0.000008 # Whole-query agent flow admission. This is above per-model admission and # controls whether broad agentic map fleets start immediately, wait with an ETA, # or fail fast when the queue is full. AIBS_AGENT_FLOW_SCHEDULER_ENABLED=true AIBS_AGENT_FLOW_MAX_CONCURRENT=0 AIBS_AGENT_FLOW_MAX_QUEUED=8 AIBS_AGENT_FLOW_QUEUE_TIMEOUT_SECONDS=300 AIBS_AGENT_FLOW_MODEL_QUEUE_TIMEOUT_SECONDS=30 AIBS_PLANNER_MODEL_QUEUE_TIMEOUT_SECONDS=30 AIBS_AGENT_FLOW_ETA_SECONDS_PER_DOCUMENT=30 AIBS_AGENT_FLOW_ETA_DISPLAY_BUFFER_RATIO=0.15 AIBS_AGENT_FLOW_ACTIVE_LEASE_TTL_SECONDS=60 AIBS_AGENT_FLOW_STATE_LOCK_TTL_SECONDS=5 AIBS_AGENT_FLOW_POLL_INTERVAL_SECONDS=0.25 AIBS_AGENT_FLOW_ACTIVE_STATE_REAP_ENABLED=true AIBS_AGENT_FLOW_ACTIVE_STATE_REAP_AFTER_SECONDS=3600 AIBS_AGENT_FLOW_RELEASE_RETRY_TIMEOUT_SECONDS=30 # Active model worker roster. If omitted, the backend derives base, summarizer, # semantic-profile, and optional heavy workers from the AIBS_LLM_* lane settings. AIBS_MODEL_ADMISSION_WORKER_ROSTER=base,summarizer,semantic-profile,map-small,reasoning-small # Built-in worker names inherit their model/runtime envelope from the matching # LLM lane. Keep roles and measured capacity explicit here. AIBS_WORKER_BASE_ROLES=planner,workspace_answer,document_map,structured_extract,reduce,final_answer AIBS_WORKER_BASE_RESIDENT_WEIGHT_GB=16.02 AIBS_WORKER_BASE_ACTIVE_PARAM_BILLIONS=26 AIBS_WORKER_SUMMARIZER_ROLES=document_map,structured_extract AIBS_WORKER_SUMMARIZER_RESIDENT_WEIGHT_GB=11.86 AIBS_WORKER_SUMMARIZER_ACTIVE_PARAM_BILLIONS=3.6 AIBS_WORKER_SEMANTIC_PROFILE_ROLES=semantic_profile AIBS_WORKER_SEMANTIC_PROFILE_RESIDENT_WEIGHT_GB=4.91 AIBS_WORKER_SEMANTIC_PROFILE_ACTIVE_PARAM_BILLIONS=8 # Document-map subagent workers. These definitions are active when the worker # name is listed in AIBS_MODEL_ADMISSION_WORKER_ROSTER. AIBS_WORKER_MAP_SMALL_MODEL_ID=qwen2.5:7b-instruct AIBS_WORKER_MAP_SMALL_ROLES=document_map,structured_extract AIBS_WORKER_MAP_SMALL_CONTEXT_LENGTH=16384 AIBS_WORKER_MAP_SMALL_MAX_OUTPUT_TOKENS=2048 AIBS_WORKER_MAP_SMALL_MAX_CONCURRENT_REQUESTS=1 AIBS_WORKER_MAP_SMALL_RESIDENT_WEIGHT_GB=4.31 AIBS_WORKER_MAP_SMALL_ACTIVE_PARAM_BILLIONS=7 AIBS_WORKER_MAP_SMALL_KEEP_ALIVE=5m AIBS_WORKER_MAP_SMALL_TIMEOUT_SECONDS=240 AIBS_WORKER_REASONING_SMALL_MODEL_ID=qwen3:8b AIBS_WORKER_REASONING_SMALL_ROLES=document_map,structured_extract,reduce AIBS_WORKER_REASONING_SMALL_CONTEXT_LENGTH=32768 AIBS_WORKER_REASONING_SMALL_MAX_OUTPUT_TOKENS=4096 AIBS_WORKER_REASONING_SMALL_MAX_CONCURRENT_REQUESTS=1 AIBS_WORKER_REASONING_SMALL_RESIDENT_WEIGHT_GB=4.91 AIBS_WORKER_REASONING_SMALL_ACTIVE_PARAM_BILLIONS=8 AIBS_WORKER_REASONING_SMALL_KEEP_ALIVE=5m AIBS_WORKER_REASONING_SMALL_TIMEOUT_SECONDS=300 AIBS_DOCUMENT_COMPLEXITY_SMALL_MODEL_IDS=qwen2.5:7b-instruct,qwen3:8b,gpt-oss:20b AIBS_DOCUMENT_COMPLEXITY_MEDIUM_MODEL_IDS=qwen3:8b,qwen2.5:7b-instruct,gpt-oss:20b AIBS_DOCUMENT_COMPLEXITY_LARGE_MODEL_IDS=gpt-oss:20b,qwen3:8b,gemma4:26b AIBS_DOCUMENT_COMPLEXITY_DEGRADED_LARGE_MODEL_IDS=gpt-oss:20b,gemma4:26b,qwen3:8b # Optional legacy/default-roster sizing. Keep at 0 until measured on the target # Ollama host. These values feed the fallback base/summarizer/heavy roster only # when AIBS_MODEL_ADMISSION_WORKER_ROSTER is not set. AIBS_MODEL_ADMISSION_BASE_RESIDENT_WEIGHT_GB=0 AIBS_MODEL_ADMISSION_BASE_ACTIVE_PARAMS_BILLIONS=0 AIBS_MODEL_ADMISSION_SUMMARIZER_RESIDENT_WEIGHT_GB=0 AIBS_MODEL_ADMISSION_SUMMARIZER_ACTIVE_PARAMS_BILLIONS=0 AIBS_MODEL_ADMISSION_HEAVY_RESIDENT_WEIGHT_GB=0 AIBS_MODEL_ADMISSION_HEAVY_ACTIVE_PARAMS_BILLIONS=0 ``` - `AIBS_MODEL_ADMISSION_ENABLED=true` enables the application-level admission boundary. If the planner itself fails, answering continues through a no-op fallback so correctness does not depend on the scheduler. - `AIBS_MODEL_ADMISSION_SAFE_POOL_GB=0` disables memory-budget admission while preserving per-model concurrency admission. Set a positive value only after measuring the Ollama host's safe resident-model pool. - `AIBS_MODEL_ADMISSION_QUEUE_TIMEOUT_SECONDS` controls how long a role waits for its preferred model lane before trying a smaller eligible lane. If no eligible lane is available, the request is treated as busy instead of hanging indefinitely. - `AIBS_AGENT_FLOW_SCHEDULER_ENABLED=true` enables whole-query admission for agentic map/reduce work. This is the user-visible queue/ETA layer; per-model admission still controls each individual LLM call after the flow starts. - `AIBS_AGENT_FLOW_MAX_CONCURRENT=0` derives the active-flow ceiling from the enabled model-node count. A positive value sets an explicit ceiling. The scheduler reduces per-flow map parallelism as active flows increase; atomic model placement still applies live GPU pressure, memory headroom, per-model, per-node, and per-flow limits before every model call. - When `AIBS_MODEL_ADMISSION_SAFE_POOL_GB` is positive, agent-flow admission clamps broad-query map parallelism to the largest value that fits the estimated pool. If even sequential map work cannot fit, the backend returns an explicit capacity response instead of overcommitting the model host. - `AIBS_AGENT_FLOW_MAX_QUEUED` bounds accepted-but-waiting agent flows. When full, the backend returns a busy answer instead of silently hanging. - `AIBS_AGENT_FLOW_QUEUE_TIMEOUT_SECONDS` is the maximum time a queued agentic flow may wait on the open SSE stream before it degrades to busy. - `AIBS_AGENT_FLOW_MODEL_QUEUE_TIMEOUT_SECONDS` is the bounded wait for each map/refine/compose model call after the flow starts. The preferred lane begins degrading after `AIBS_MODEL_ADMISSION_QUEUE_TIMEOUT_SECONDS`; the longer flow-scoped wait lets live placement pressure serialize work instead of misclassifying temporary saturation as a failed document. - `AIBS_PLANNER_MODEL_QUEUE_TIMEOUT_SECONDS` is the bounded wait for an intent or scope-planning call to obtain scheduler-selected model capacity. Planning uses the same global node placement and lane envelopes as scheduled document work, so a busy or stalled direct LLM host cannot serialize every chat request. - `AIBS_AGENT_FLOW_ETA_SECONDS_PER_DOCUMENT` seeds ETA calculation before the process has observed real completed agent-flow durations. Runtime observations replace this seed for later ETA estimates. - `AIBS_AGENT_FLOW_ETA_DISPLAY_BUFFER_RATIO` pads the user-visible queued ETA without changing scheduling decisions. The default `0.15` shows about 15% more time than the raw estimate so users are more likely to receive an answer before the displayed time expires. - `AIBS_AGENT_FLOW_ACTIVE_LEASE_TTL_SECONDS`, `AIBS_AGENT_FLOW_STATE_LOCK_TTL_SECONDS`, and `AIBS_AGENT_FLOW_POLL_INTERVAL_SECONDS` control the coordinated scheduler's active-flow heartbeat lease, fenced state-lock lease, and queue polling cadence. Keep these conservative unless a saturation run proves the lock/heartbeat windows are too slow. - `AIBS_AGENT_FLOW_ACTIVE_STATE_REAP_ENABLED=true` enables a generous dead-man cleanup for leaked active-flow records. Reap is driven by `AIBS_AGENT_FLOW_ACTIVE_STATE_REAP_AFTER_SECONDS`, not the short heartbeat lease. The default one-hour ceiling is intentionally much longer than normal broad-query work: a missed heartbeat should not over-admit another map fleet, but a crashed or leaked active record should eventually self-heal. - `AIBS_AGENT_FLOW_RELEASE_RETRY_TIMEOUT_SECONDS` controls how long an authoritative flow release keeps trying to acquire the scheduler state lock before logging a leaked slot. Keep this longer than `AIBS_AGENT_FLOW_STATE_LOCK_TTL_SECONDS`. - For multi-process Valkey, active-flow dead-man liveness must be based on server-side TTL/key expiry rather than API-worker epoch-clock comparisons. - `AIBS_MODEL_ADMISSION_WORKER_ROSTER` is an ordered list of worker names. Each worker is configured through `AIBS_WORKER__*`, where `` is the upper-case roster name with non-alphanumeric characters converted to underscores. For example, `map-small` becomes `AIBS_WORKER_MAP_SMALL_*`. - Built-in worker names inherit from the corresponding LLM lane unless an `AIBS_WORKER__*` override is present: - `base` inherits model, context length, output budget, keep-alive, timeout, and concurrency from `AIBS_LLM_*`. - `summarizer` inherits from `AIBS_LLM_SUMMARIZER_*`, then falls back to `AIBS_LLM_*`. - `semantic-profile` inherits from `AIBS_LLM_SEMANTIC_PROFILE_*`. Its default role is `semantic_profile` and its default model is `qwen3:8b`. - `heavy` inherits from `AIBS_LLM_HEAVY_*`, then falls back to `AIBS_LLM_*`, but the default heavy worker is created only when `AIBS_LLM_HEAVY_MODEL` names a distinct model. - Custom worker names, such as `map-small`, do not have an inherited LLM lane. They must provide at least `AIBS_WORKER__MODEL_ID` and `AIBS_WORKER__ROLES`. - Custom workers become callable only when listed in `AIBS_MODEL_ADMISSION_WORKER_ROSTER`. In the current demo roster, `qwen2.5:7b-instruct` and `qwen3:8b` can be active document-map subagents alongside `gpt-oss:20b`; the built-in `semantic-profile` qwen lane is reserved for ingestion-time semantic profile enrichment. - `AIBS_DOCUMENT_COMPLEXITY_*_MODEL_IDS` controls which active map-agent models are eligible for each document class. This is deterministic structural routing: small/medium documents can use lighter qwen agents, while large or degraded-large documents can prefer `gpt-oss:20b` before falling back to Gemma. Admission still picks the concrete available model under concurrency and residency constraints. - `AIBS_WORKER__ROLES` controls which answer roles the worker can serve. Supported roles are `planner`, `document_map`, `structured_extract`, `reduce`, and `final_answer`. The complexity policy sets eligibility; the admission planner still picks the concrete worker under the memory/concurrency budget. - `AIBS_WORKER__CONTEXT_LENGTH`, `MAX_OUTPUT_TOKENS`, `KEEP_ALIVE`, and `TIMEOUT_SECONDS` are optional overrides for built-in workers and required runtime-envelope declarations for custom workers. - `AIBS_WORKER__MAX_CONCURRENT_REQUESTS`, `RESIDENT_WEIGHT_GB`, and `ACTIVE_PARAM_BILLIONS` are used by admission to serialize model calls and estimate safe residency. For MoE models, active parameters should be the active parameter count, not total parameters. - `*_RESIDENT_WEIGHT_GB` should reflect the measured loaded model weight for the lane. `*_ACTIVE_PARAMS_BILLIONS` is used for context/KV estimates and should be the active parameter count for MoE models, not the total parameter count. For example, a 20B MoE model with about 3.6B active parameters should use `3.6` here for KV budgeting. - With `AIBS_MODEL_CLUSTER_ENABLED=true`, the current local `.env` uses the Valkey global placement adapter and routes each leased model call to the selected node. Set it to `false` only for a single-process local deployment where model residency admission should remain a process-local guard. - Keep the global model cluster enabled only when all of the following are true: worker resident weights are measured, node VRAM budgets are measured, `AIBS_MODEL_COORDINATION_BACKEND=valkey`, and a live node-residency monitor is writing fresh health observations. Without fresh residency, the placement primitive fails closed as `node_stale`. - `AIBS_MODEL_CLUSTER_ACTIVE_LEASE_TTL_SECONDS` controls the lifetime of an active model-placement lease. Keep it longer than the longest expected LLM call timeout. - `AIBS_MODEL_CLUSTER_HEARTBEAT_INTERVAL_SECONDS` controls renewal cadence for active placement leases. It must be shorter than `AIBS_MODEL_CLUSTER_ACTIVE_LEASE_TTL_SECONDS`; transient renewal failures are retried with bounded backoff. - `AIBS_MODEL_CLUSTER_MAX_RESIDENCY_STALENESS_MS` is the maximum age of a node residency health observation accepted by placement. If the monitor stops, the scheduler should fail closed instead of placing work onto unknown node state. - `AIBS_MODEL_CLUSTER_RESIDENCY_MONITOR_ENABLED` enables the elected monitor that polls configured Ollama nodes and writes two records: a tiny placement health stamp and a separate telemetry snapshot. Keep `AIBS_MODEL_CLUSTER_ENABLED=true` only while audit and live tests prove those stamps stay fresh. - `AIBS_MODEL_CLUSTER_RESIDENCY_MONITOR_RUN_IN_API=false` delegates that monitor to the standalone `python -m aibs_backend.interfaces.cli.model_residency_monitor` process. This is the recommended multi-node deployment: answer-model latency or API event loop pressure cannot delay the control-plane election heartbeat. Keep the value `true` only for a small single-process development environment. - API startup waits for at least one fresh healthy residency snapshot whenever global model scheduling is enabled. Stale or absent control-plane state keeps the API unready instead of accepting chats that are guaranteed to fail with `node_stale`. - `AIBS_MODEL_CLUSTER_RESIDENCY_MONITOR_POLL_INTERVAL_SECONDS` must be no more than one third of `AIBS_MODEL_CLUSTER_MAX_RESIDENCY_STALENESS_MS`. `AIBS_MODEL_CLUSTER_RESIDENCY_MONITOR_POLL_TIMEOUT_SECONDS` must be shorter than the poll interval. - `AIBS_MODEL_CLUSTER_RESIDENCY_MONITOR_FAILURE_THRESHOLD` is the number of consecutive failed polls before a node is marked down. A single success restores the node to healthy. - `AIBS_MODEL_CLUSTER_RESIDENCY_MONITOR_LEASE_TTL_SECONDS` bounds monitor failover and must be longer than one poll interval plus one poll timeout. This ownership TTL may exceed the placement staleness window: placement fails closed as soon as telemetry exceeds `AIBS_MODEL_CLUSTER_MAX_RESIDENCY_STALENESS_MS`, even while a paused monitor still owns its election lease. This prevents transient host scheduling pauses from creating false leader flaps without trusting stale placement state. - `AIBS_MODEL_CLUSTER_RESIDENCY_MONITOR_STANDBY_RETRY_SECONDS` controls how often standby API processes retry monitor election after another process owns the monitor lease. - `AIBS_MODEL_CLUSTER_TELEMETRY_TTL_SECONDS` controls retention for the richer per-node telemetry snapshot used by audit/health surfaces. It is separate from the placement hash to keep admission reads lean. - `AIBS_MODEL_CLUSTER_HEALTH_HISTORY_WINDOW_SECONDS`, `AIBS_MODEL_CLUSTER_HEALTH_HISTORY_MAX_SAMPLES`, and `AIBS_MODEL_CLUSTER_HEALTH_HISTORY_TTL_SECONDS` control the rolling samples used by the System Health page. Model-node history is stored in the shared model-coordination pool. Ingestion, answer, and runtime workload history is sampled independently by each API and stored in that environment's application-coordination pool. Keep TTL greater than or equal to the displayed history window. - Before setting `AIBS_MODEL_CLUSTER_ENABLED=true`, run `python -m aibs_backend.interfaces.cli.audit_model_capacity --json` and `python -m aibs_backend.interfaces.cli.prove_model_monitor_takeover --json --repeat 3`. After enabling, restart the API and run the audit again; the expected state is `ready_for_global_scheduler=true`. - When the standalone monitor is selected, start it before the API and keep it under the same service supervisor as the API. Run at least two supervised monitor candidates on independent always-on hosts; Valkey election keeps one active while the other remains ready to take over. See `docs/maintainance_&_operations/model_residency_monitor.md`. ## Common Setup Checks Before starting the API, it helps to confirm: - OpenSearch is reachable - the embedding model path exists - the reranker model path exists if reranking is enabled - `AIBS_DOCLING_ARTIFACTS_PATH` exists if local Docling artifacts are configured - device values are `auto` unless this machine is known to have a CUDA/ROCm-enabled torch runtime - the LLM endpoint is reachable if `AIBS_LLM_ENABLED=true` Then start the backend with: ```powershell python -m pdm run uvicorn aibs_backend.interfaces.api.app:app --host 127.0.0.1 --port 5000 --reload ``` Open: ```text http://127.0.0.1:5000/docs ``` ## Related Docs - [HTTP API reference](../09-0-0-API/http-api.md) - [Model capacity audit](../06-0-0-Operations/model-capacity-audit.md) - [Architecture overview](../03-0-0-Architecture/overview.md)