AI Models

Claude writes text and code. It can't transcribe a voice memo, read a screenshot with a vision model, embed a paragraph, rank search results, make a calibrated decision, or render an image, a song or a clip. I run models that do those things on my own hardware, and some of them sit behind a LiteLLM gateway as well. This bundle is how Claude calls them.

You list your models in a CSV. One Python script turns a short command (transcribe clip.wav) into the request that model's server expects. It prints one JSON object with the text, the numbers or the path of the file it wrote. When nothing of a type is configured, it says so and Claude tells you, rather than guessing at an endpoint.

What's inside

Path	Purpose
`skills/ai-models/ai.py`	The script. Python 3.9+ standard library only, no `pip install`.
`skills/ai-models/SKILL.md`	When Claude reaches for each command and how it reads the result.
`skills/ai-models/models.example.csv`	Fake rows, at least one per server shape. Copy it and edit.
`skills/ai-models/workflows/example-image.json`	A ComfyUI workflow in API format, with placeholders.
`CLAUDE.md`	A pointer block to paste into your project's `CLAUDE.md`.

Setup

mkdir -p ~/.config/ai-models
cp .claude/skills/ai-models/models.example.csv ~/.config/ai-models/models.csv
cp -R .claude/skills/ai-models/workflows ~/.config/ai-models/   # only if you use ComfyUI

Edit the rows to point at your servers. Put any keys in ~/.config/ai-models/.env:

# ~/.config/ai-models/.env
AI_GATEWAY_KEY=<your-gateway-key>

Then check what answers:

python3 .claude/skills/ai-models/ai.py list --check

Where config lives

Two pairs of files are read, and the project wins:

	Models	Keys
Personal	`~/.config/ai-models/models.csv`	`~/.config/ai-models/.env`
Project	`.claude/ai-models.csv`	`.claude/.env`

$XDG_CONFIG_HOME replaces ~/.config when it is set. Rows merge by name, so a project row replaces the personal row with the same name and everything else is kept. Project rows come first. A key already exported in your shell wins over both .env files, and .claude/.env wins over the personal one. I use the project layer for a scoped gateway key, so one app never sees my master key.

.claude/ai-models.csv is gitignored by default because it tends to hold LAN addresses. Take the line out of the managed block if your team wants to share it.

The project is $CLAUDE_PROJECT_DIR, or the nearest folder above the working directory that has a .claude/ in it. The script never works it out from its own location, so a copy in ~/.claude/skills/ works in every project.

The CSV

Column	Example	Meaning
`name`	`home-stt`	Unique. `--model home-stt` picks this row.
`type`	`stt`	`stt`, `tts`, `vision`, `embed`, `rerank`, `decide`, `image`, `video`, `music` or `chat`.
`shape`	`openai`	Which request format the server speaks (below).
`base_url`	`http://localhost:8000/v1`	Everything before the endpoint path.
`model`	`whisper-large-v3-turbo`	The model name the server expects.
`key_env`	`AI_GATEWAY_KEY`	The variable holding the key. Empty means no auth.
`options`	`voice=alloy;dimensions=1024`	Defaults merged into every request for this row.
`example`	`transcribe meeting.wav`	One call that works, shown by `list`.
`notes`	`WAV only`	Hints for Claude, shown by `list`.

options and --param take key=value. Numbers, true, false and null are sent as those types. Wrap a value in double quotes to keep it a string. On the command line the shell eats one layer, so write --param 'seed="42"'. In a CSV cell it is "seed=""42""". ; separates options, so an options value can't contain one. Use --param for that. A dotted key nests: options.duration_sec=20 becomes {"options": {"duration_sec": 20}}. A request is built from the script's defaults, then the row's options, then the command's flags, then --param. --body @file.json replaces the lot and is sent as written.

Lines that start with # are ignored. A row with an unknown type or shape is reported by list and skipped. The rest still load.

Shapes

Shape	Serves	Notes
`openai`	chat, vision, embed, stt, tts, image	LiteLLM, OpenRouter, vLLM, llama.cpp, TEI, Ollama. `wav_only=true` converts audio to WAV before transcription, for a gateway route that ends at a WAV-only server.
`cohere-rerank`	rerank	`POST /rerank` with `query`, `documents`, `top_n`. LiteLLM's rerank route speaks it.
`systemone`	decide	Clef-Flash on llama.cpp and TypeSafe's Jev share this body. Bearer auth when `key_env` is set.
`audiocpp`	tts, stt, music	audio.cpp's server. Transcription takes WAV, so other formats go through `ffmpeg` first when it is installed.
`comfyui`	image, video	Submits a workflow, polls, downloads the outputs. Reserved options: `workflow`, `poll`, `timeout`.

base_url is the prefix, and the script adds the endpoint path to it. That's what lets one shape run direct or through a gateway. audio.cpp music is http://host:8885/v1 plus /tasks/run direct, and https://gateway.example.com/music plus /tasks/run through a LiteLLM pass-through.

ComfyUI workflows

Export your graph with "Save (API Format)" and point the row at it with workflow=<file>. A relative path resolves from the CSV that names it. Any string value that is exactly {prompt}, {negative}, {seed}, {width}, {height} or {frames} is filled in, and numbers stay numbers. --param steps=30 fills a {steps} you add yourself. A placeholder left without a value stops the call and names it. The seed is random unless you pass --seed, and it comes back in the result.

Fallback

Rows of one type are tried in order. A row whose key variable is unset is skipped. A row that is unreachable (DNS failure, connection refused, or no connection within 3 s) hands over to the next row. A server that answers with an error stops the call there, so a bad request is never quietly sent on to a paid gateway. --model <name> turns fallback off.

Output and exit codes

Every command prints one JSON object: ok, type, the model row it used, tried, result and files. Files go to --out, or to a temp folder when it is left off.

Code	Meaning
`0`	It worked.
`1`	The server answered with an error, timed out, or sent something unreadable.
`2`	Nothing configured for that type, a missing key, a missing input file or a bad flag.
`3`	Every candidate row was unreachable.

embed prints each vector's size, norm and first values plus a similarity matrix rather than thousands of floats. --full prints the vectors, and --out vectors.json writes them.

No key value is ever printed. A server's own error text is passed through as it was received.

HTTPS on macOS

The python.org installer ships Python without a CA bundle, so every https:// row fails with CERTIFICATE_VERIFY_FAILED. The script reports that as exit 2 with a hint, rather than skipping to the next row. Run Install Certificates.command from /Applications/Python 3.x/ once, or export SSL_CERT_FILE=/etc/ssl/cert.pem. I hit this on my own machine the first time I pointed it at my gateway.