Catalog
JuliusBrussee/caveman-discover

JuliusBrussee

caveman-discover

Find and label every LLM workflow in the repository so Caveman Cloud groups spend by workflow instead of one bucket. Use for "discover workflows" or breaking LLM spend down by workflow.

NewUpdated Sep 9, 2026

You are labeling this repository's LLM workflows for Caveman Cloud. A workflow is a job the code performs — "answer a support ticket", "build the nightly digest", "run the eval suite" — not a technology. Every gateway request can carry a workflow label; unlabeled traffic all lands in one unlabeled-workflow bucket. Your job: find the workflows, name them well, wire the labels, and verify nothing broke.

This changes code, so it goes through the user's normal review: propose the table first, apply after the user agrees. Re-running on an already-labeled repo must change nothing (idempotent).

This skill is operator-invoked. An unlabeled-traffic Cave Plan observation is review-only and does not create an advisory file, proposal, or Draft PR. Do not infer that telemetry selected a callsite or authorized an edit. Independently inventory the repository, present the labeling table, and wait for the user's approval before changing code.

Step 1 — Inventory the workflows

Walk the repo from its entry points, not from its imports:

  • HTTP/RPC handlers that call an LLM (directly or through layers)
  • Scheduled jobs: cron definitions, queue consumers, workers, GitHub Actions that invoke LLM code
  • CLI commands and scripts (scripts/, bin/, package.json scripts)
  • Eval / test harnesses that burn real tokens
  • Distinct agents or chains inside a framework (each LangGraph graph, each crew, each agent definition is usually its own workflow)

One workflow = one job a human would name. Ten callsites inside the same request handler are one workflow; one shared llm.ts helper used by three jobs is three workflows (label at the callers, never the shared helper).

Step 2 — Name them

Slug grammar (the gateway enforces this): lowercase [a-z0-9_-], 1–96 chars. Name the job, not the tech:

  • Good: support-reply, nightly-digest, pr-review, eval-suite, onboarding-email
  • Bad: openai-calls (tech), main (says nothing), SupportReply (invalid), johns-test-3 (won't age)

Names are forever-ish — renaming later splits the spend history. When a job's purpose isn't clear from the code, derive the slug from the file name and mark it review in the table rather than inventing a purpose.

Step 3 — Propose, then apply

Present this table and ask to proceed:

| workflow | job | where | how it gets labeled |
|---|---|---|---|
| support-reply | answers inbound tickets | src/bot/reply.ts:41 | defaultHeaders on the reply client |
| nightly-digest | 02:00 summary job | jobs/digest.ts:12 | header on the digest client |
| eval-suite (review) | scripts/eval.ts:8 — purpose inferred from filename | scripts/eval.ts:8 | env override at invocation |

Then wire each label with the lightest mechanism available at that callsite:

  • @caveman-ai/sdk / caveman_cloud SDK: per-trace workflow option, or defaultWorkflow on the client a single-job service constructs.
  • Raw provider SDKs (OpenAI/Anthropic/LangChain/LiteLLM/Vercel): add "x-cave-workflow": "<slug>" to the same defaultHeaders / default_headers / extra_headers block that already carries x-cave-api-key. Shared client used by several jobs → pass the header per call (every SDK above accepts per-request header overrides), or give each job its own thin client.
  • Wrapped coding agents (caveman wrap): --workflow <slug> flag or CAVE_WORKFLOW=<slug> env at the invocation site (cron line, CI step).
  • Raw HTTP: add the x-cave-workflow header to the request.

Label the callers, keep the diff minimal, match the repo's style. If a callsite is not routed through the Caveman gateway at all, don't label it — list it under "not wired" in the report (labels only travel on gateway traffic; wiring is the caveman-setup skill's job).

Step 4 — Verify

Run whatever the repo already uses to exercise one labeled path (a test, a dev script, one curl). Then confirm: the request still succeeds (the gateway rejects an invalid label with 400 cave_invalid_request_header — fix the slug if so). Labeled spend appears on the dashboard at /activity?tab=workflows as each workflow next runs; jobs on a schedule show up when the schedule fires, and that's worth saying in the report rather than pretending they're live.

Step 5 — Report

## Workflows labeled

| workflow | job | where |
|---|---|---|
| support-reply | answers inbound tickets | src/bot/reply.ts:41 |
| nightly-digest | 02:00 summary job | jobs/digest.ts:12 |

Verified: <the labeled path you actually exercised, and what you observed>
Lands at: <DASHBOARD>/activity?tab=workflows — each row appears as that workflow
next runs. Anything still unlabeled shows as `unlabeled-workflow`.
Not wired (no gateway routing, so no label): <list or "none">
Marked review: <slugs whose purpose was inferred from filenames, or "none">

If you found no LLM entry points at all: say exactly that, and point at the setup skill (<docs origin>/docs/agent-setup.md) instead of manufacturing a table.

Files1
1 files · 1.4 KB

Select a file to preview

Overall Score

86/100

Grade

A

Excellent

Grades are signals, not a certification. Always review a skill yourself before use.

Safety

85

Quality

88

Clarity

87

Completeness

82

Summary

This skill guides an agent to discover and label LLM workflows in a repository for Caveman Cloud spend tracking. It walks entry points (handlers, jobs, CLI scripts), names them according to slug rules, proposes changes to the user for approval, wires labels through SDK mechanisms, verifies functionality, and reports results.

Detected Capabilities

file readingcode analysispattern matchingdata inventorycode modification (after user approval)test executionHTTP request verification

Trigger Keywords

Phrases that agents use to match this skill to user intent.

discover workflowslabel llm callsgroup spend by workflowinventory entry pointscaveman cloud labelingtrack api costs

Risk Signals

INFO

Code modification allowed after user approval

Step 3 — Propose, then apply
INFO

Test execution to verify labeling

Step 4 — Verify
INFO

HTTP header injection via x-cave-workflow

Step 3 — Raw provider SDKs

Use Cases

  • Break down LLM API spend by workflow instead of lumping into unlabeled buckets
  • Inventory all LLM entry points in a codebase systematically
  • Label HTTP handlers, scheduled jobs, CLI scripts, and agents for cost tracking
  • Verify workflow labels work correctly before deployment
  • Generate a report showing which workflows are labeled and which remain unlabeled

Quality Notes

  • Well-structured five-step process with clear instructions at each stage
  • Good examples of workflow naming (good vs. bad)
  • Appropriate emphasis on user approval before code changes (operator-invoked, review-first pattern)
  • Comprehensive coverage of entry points: HTTP handlers, cron jobs, CLI scripts, agents, eval harnesses
  • Clear guidance on the lightest mechanism for labeling at each callsite (SDK options, headers, env vars)
  • Good idempotence requirement — re-running should change nothing
  • Verification step includes concrete validation (test success, 400 error handling)
  • Report template provided with clear sections (verified path, dashboard location, not wired, marked for review)
  • Handles edge case of no LLM entry points found
  • Slug naming rules clearly specified with character constraints
  • Distinguishes between workflow (the job) and callsite (where to label)
  • Mentions label immutability ('names are forever-ish') to guide naming decisions
Model: claude-haiku-4-5-20251001Analyzed: Sep 9, 2026

Reviews

Add this skill to your library to leave a review.

No reviews yet

Be the first to share your experience.

Version History

  1. v2.0

    Contract changed: description

    ✦ AIDescription simplified, removes repo-setup precondition and specific activation phrases.

    triggering2026-09-09

    LATEST
  2. v1.0

    2026-08-17

    View This VersionInitial version

Use JuliusBrussee/caveman-discover in your dev environment

Command Palette

Search for a command to run...