Skip to content

Standalone Auditor CLI

The CT Toolkit CLI provides a powerful way to audit LLM endpoints for identity continuity without writing any Python code. It's designed for security researchers, auditors, and developers who need to stress-test a model's alignment with a specific Constitutional Kernel.

Installation

The CLI is included automatically when you install the toolkit:

pip install ct-toolkit

Verify the installation:

ct-toolkit --help

Core Commands

audit

Run an L3 Identity Consistency Metric (ICM) audit against an LLM provider.

Usage:

ct-toolkit audit --url <PROVIDER_URL> --kernel <KERNEL_NAME> [OPTIONS]

Options:

  • --url: The base URL of the LLM provider (e.g., http://localhost:11434/v1 for Ollama).
  • --kernel: Name of the Constitutional Kernel to use (default: general).
  • --template: Identity template for embedding comparison (default: general).
  • --model: Specific model name to request (default is provider-dependent, e.g. gpt-4o-mini for OpenAI).

Example: Auditing a local Ollama model

ct-toolkit audit --url http://localhost:11434/v1 --kernel defense

list-kernels

List all available Constitutional Kernels installed in the toolkit or your local project.

ct-toolkit list-kernels

serve

Start a FastAPI server for LiteLLM Generic Guardrail API.

ct-toolkit serve --kernel <KERNEL_NAME> --port <PORT>

list-templates

List all available Identity Templates (categories for behavioral probes).

ct-toolkit list-templates

ASCII Banner & Versioning

Upon startup, the CLI greets you with the Theseus Guard banner, ensuring you are operating within the CT Toolkit environment:

  _______ _    _ ______  _____ ______ _    _  _____    _____ _    _
 |__   __| |  | |  ____|/ ____|  ____| |  | |/ ____|  / ____| |  | |   /\
    | |  | |__| | |__  | (___ | |__  | |  | | (___   | |  __| |  | |  /  \
    | |  |  __  |  __|  \___ \|  __| | |  | |\___ \  | | |_ | |  | | / /\ \
    | |  | |  | | |____ ____) | |____| |__| |____) | | |__| | |__| |/ ____ \
    |_|  |_|  |_|______|_____/|______|_____/|_____/   \_____|\____//_/    \_\

  computational theseus toolkit (ct toolkit) v0.3.20
  identity continuity guardrails for agentic systems

Example Output

When an audit runs, the CLI provides a detailed table of results and a final Identity Health Score:

(Representative screenshot removed from docs assets. Run ct-toolkit audit ... to see the live terminal table output with PASSED/FAILED probe rows.)

Interpreting Results:

  • Health Score: 0% to 100%. A score below 80% usually indicates significant Identity Drift.
  • Risk Level: Categorized as LOW, MEDIUM, HIGH, or CRITICAL based on the severity of failed probes.
  • Critical Violations: Lists specific probes that violated non-negotiable axiomatic anchors.

Troubleshooting

  • Connection Error: Ensure your LLM provider (Ollama, LM Studio, etc.) is running and accessible at the provided URL.
  • Kernel Not Found: Use ct-toolkit list-kernels to verify you are using a valid kernel name.
  • Empty Probes: Ensure the ct_toolkit/divergence/probes/ directory contains valid .json probe files.