Configuration

Two YAML files that work as a pair:

• the main config — connection, detection, history, LLM, masking, validation. Copy config/dbmask.config.example.yaml and edit; every option is commented there too.
• the field overrides file — your manual sensitive/safe toggles, referenced from the main config via detection.overrides_file. Copy config/dbmask.fields.example.yaml.

Any string value can use ${ENV_VAR} (or ${ENV_VAR:default}) placeholders, so credentials never live in the file:

database:
  password: ${DB_PASSWORD}

database — what to scan/mask

Either a full SQLAlchemy URL, or parts and dbmask builds it:

database:
  url: "postgresql+psycopg2://user:${DB_PASSWORD}@staging-host:5432/mydb"
  # -- or --
  dialect: postgresql        # postgresql | mysql | mssql | oracle | sqlite | ...
  driver: psycopg2
  host: staging-host
  port: 5432
  username: myuser
  password: ${DB_PASSWORD}
  database: mydb
  name: mydb                 # logical name used in history/reports
  schemas: []                # empty = all visible schemas
  connect_args: {}           # passed to SQLAlchemy create_engine

Point this at a copy

mask --apply rewrites this database in place. It must be the staging / snapshot copy — never production.

source_database — the original (for validate)

Same shape as database. Only used by dbmask validate, which compares the masked database against this untouched original.

detection

detection:
  sample_size: 100              # values sampled per column for patterns
  use_patterns: true
  use_history: true
  overrides_file: config/dbmask.fields.yaml
  skip_column_patterns: []      # regex, case-insensitive: [".*_id$"]
  skip_table_patterns: []       # ["^tmp_", "_bkp$"]

Field overrides

The overrides file always wins over every automatic layer:

sensitive:
  - match: "public.customers.email"   # exact schema.table.column
    rule: email
  - match: "ssn"                      # this column name anywhere
    rule: ssn
  - match: "pattern:*_password"       # glob on column names
    rule: redact
  - "date_of_birth"                   # shorthand: sensitive, rule auto-picked

not_sensitive:
  - match: "public.orders.order_number"
    note: "Looks like an ID but is safe"

Match precedence: schema.table.column → table.column → column → pattern: globs.

history

Every conclusive decision is stored and reused on later runs, keeping masking reproducible and fast. UNKNOWN results are deliberately not stored — they are re-evaluated every run.

history:
  enabled: true
  url: "sqlite:///dbmask_history.db"   # any SQLAlchemy URL

llm

Optional fallback for columns the patterns cannot decide — see LLM detection for providers, local setups, and the privacy controls (send_values: false metadata-only mode).

masking

masking:
  dry_run: true                     # library default; the CLI enforces --apply anyway
  seed: ${DBMASK_SEED}              # PRIVATE seed -> deterministic masking
  default_strategy: format_random
  column_strategies:                # highest priority, your explicit call
    notes: blank
    public.users.bio: redact
  rule_strategies:                  # per detected rule
    email: fake_email
    credit_card: fake_credit_card
  seed_map:
    enabled: true
    url:                            # blank = sqlite:///dbmask_seedmap.db
    salt: ${DBMASK_SEED_SALT}       # keep the salt out of the store
    untracked_strategies: ["null", "blank", "redact"]

Strategy resolution order and the full catalogue: Masking strategies. Durable consistency: The seed map.

validation

validation:
  enabled: true
  check_row_counts: true
  check_schema_elements: true
  check_masking_completeness: true
  pk_row_limit: 5000            # rows compared per column (PK-aligned mode)
  distinct_value_limit: 5000    # ┐
  max_common_values: 100        # │ fallback heuristic tuning
  max_rows_per_value: 50        # │ (keyless tables only)
  unmasked_evidence_threshold: 10  # ┘
  ignore_test_data: true        # skip 'test', 'n/a', single chars, ...
  columns: []                   # explicit list, or empty = from history/scan
    # - public.customers.email

What each check proves — and what it can't: Validation.

CLI flags worth knowing

Command	Flag	Effect
mask	--apply	actually write (otherwise always a dry run)
mask	--allow-partial	proceed despite scan errors (unscanned columns stay unmasked)
mask	--show-values	show original values in the preview instead of redacting
scan/mask	—	non-zero exit on scan errors (3 / 2)
validate	--strict	warnings and skipped checks also fail
scan/validate/seeds	--json	machine-readable output