# Rebuild Document Catalog CLI

Module: `aibs_backend.interfaces.cli.rebuild_document_catalog`

## Purpose

Rebuild or validate the OpenSearch document catalog projection from persisted
documents.

This CLI is a projection repair tool. It does not generate semantic profiles,
repair parsed documents, or infer new document truth. If existing documents need
semantic profiles, run [backfill_semantic_profiles.md](backfill-semantic-profiles.md)
first.

## Command

```bash
pdm run python -m aibs_backend.interfaces.cli.rebuild_document_catalog \
  [--workspace <uuid>] [--document <uuid>] [--batch-size 200] \
  [--dry-run|--validate-only] [--recreate] [--only-missing]
```

or:

```bash
python -m aibs_backend.interfaces.cli.rebuild_document_catalog \
  [--workspace <uuid>] [--document <uuid>] [--batch-size 200] \
  [--dry-run|--validate-only] [--recreate] [--only-missing]
```

## Options

| Option | Meaning |
| --- | --- |
| `--workspace <uuid>` | Rebuild one workspace. Omit to process all workspaces. |
| `--document <uuid>` | Repair or validate one document projection inside `--workspace`. |
| `--batch-size <int>` | Batch size, clamped to `1..1000`; default is `200`. |
| `--dry-run` | Validate/count without OpenSearch writes. |
| `--validate-only` | Alias for read-only validation. |
| `--recreate` | Drop and recreate the shared catalog index before rebuilding. |
| `--only-missing` | Skip catalog rows that already exist. |

Invalid combinations:

- `--document` without `--workspace`
- `--recreate` with `--workspace`
- `--recreate` with `--document`
- `--recreate` with `--dry-run` or `--validate-only`
- `--recreate` with `--only-missing`

## Execution Flow

1. Validate arguments and optional workspace/document IDs.
2. Build the application container from `.env`.
3. Resolve `document_projection_maintenance.rebuild_catalog_index`.
4. Run validation mode or rebuild mode.
5. Refresh the index after writes.
6. Print a report.

## What Gets Projected

Catalog schema `5` projects document identity and planning fields from persisted
document state, including:

- recovered content title and aliases
- aboutness summary
- language/source/date/version fields
- processing `document_family` for diagnostics
- semantic profile fields and completeness/confidence
- compact search text for planner/catalog discovery

The processing `document_family` is not a semantic category. It is preserved for
diagnostics and ingestion behavior, while semantic category signals come from the
document-owned `semantic_profile`.

## Report Fields

Validation reports include counts for expected, projected, matched, missing, and
stale documents plus profile/source-truth completeness fields. Rebuild reports
include deleted, indexed, skipped-existing, and recreate flags.

Read-only modes print a validation prefix and do not write to OpenSearch.

## Schema Upgrade

Current catalog schema: `5`.

When a deployed catalog index is older than schema `5`, run an unscoped recreate
rebuild:

```bash
pdm run python -m aibs_backend.interfaces.cli.rebuild_document_catalog --recreate
pdm run python -m aibs_backend.interfaces.cli.rebuild_document_catalog --validate-only
```

Do not pass `--workspace` for schema recreation. The catalog index is shared.

## Operating Notes

- Run semantic profile backfill first for documents missing current semantic
  profiles.
- Use `--validate-only` before and after a recreate.
- Use `--workspace <uuid> --document <uuid>` for one-row repair after a single
  upload/delete/status issue. This reprojects or deletes only that document row.
  It does not mark the workspace manifest clean; run workspace validation or
  workspace repair to certify the whole workspace.
- Use `--only-missing` for outage repair when schema is current and stale rows
  are not the problem.
- In multi-process or multi-box deployments, set
  `AIBS_COORDINATION_BACKEND=redis` or `valkey`. Catalog projection repair clears
  shared dirty-state records through the coordination backend; process-local
  memory is only suitable for local development and tests.