Itô Inference

Serve a model on rented Itô metal by delegating to the canonical Itô compute backend (Layer 0.2). ECC does not implement a parallel serving stack, launch adapter, or inference server, and does no browser automation. This skill chains off a completed booking produced by ito-compute; it never books, reserves, or spends.

Prerequisite

A completed booking from the ito-compute skill: booking id, node IPs, SSH access, GPU SKU, node count, and fabric, already recorded in harness memory. Without a booking record, stop — this skill does not provision.

Delegation

ECC calls the canonical backend through the ecc ito bridge; it never re-implements serving. Authenticate once with ecc ito login (device authorization; no key in arguments, files, logs, or chat), exactly as ito-compute documents.

ecc ito serve \
  --booking <booking-id> \
  --model <hf-model-id> \
  [--quantization <scheme>] \
  [--ttft-ms <target>] [--tpot-ms <target>]

The --ttft-ms / --tpot-ms SLO is optional; supplying it turns on disaggregated prefill/decode, which is off by default.

What the backend does (Layer 0.2)

The desk backend, not ECC, runs the stages, and this skill only reports them:

Fabric gate — never launch on unverified metal. Blocks below 80% of fabric-expected bus bandwidth; advisory between 80% and 92%; fails loud on silent NCCL socket fallback.
Weights download and shard to the serving layout (desk-side sharded cache keyed by model, quantization, TP degree).
Topology plan (AIConfigurator): TP inside the NVLink domain, PP across nodes; engine flags emitted as a reviewable file before launch.
Launch (vLLM, Dynamo when disaggregating) under systemd, warmup, SLO canary, and registration of the endpoint URL and config to Graphiti memory.

Unavailable today

The serving operation is not yet wired: the canonical CLI's inference verb and the desk serve-on-booking backend are scaffolds. Until they land, this skill reports the missing capability and stops. Never substitute a local runner or a purchase endpoint.

Files1

1 files · 1.0 KB

Select a file to preview

Overall Score

82/100

Grade

B

Good

Safety

88

Quality

80

Clarity

78

Completeness

76

Summary

A delegating skill that instructs an agent to serve a machine learning model on rented GPU infrastructure (Itô metal) by calling the canonical Itô backend through the `ecc ito serve` command. The skill chains off a completed compute booking from `ito-compute`, provides the correct CLI invocation syntax, and explicitly documents that it does not implement its own serving stack, provision infrastructure, or perform browser automation.

Detected Capabilities

CLI command execution (ecc ito serve)Backend delegation (no local implementation)Model serving orchestration (vLLM, Dynamo)Device-based authentication (ecc ito login)

Trigger Keywords

Phrases that MCP clients use to match this skill to user intent.

serve model on gpuinference endpointitô metal servingvllm deploymentmodel inference booking

Use Cases

Serve a fine-tuned LLM on rented GPU nodes after booking compute
Deploy vLLM inference endpoint with topology planning and SLO monitoring
Configure disaggregated prefill/decode for latency-sensitive serving
Launch a quantized model on multi-node Itô metal with fabric verification
Delegate model serving to the canonical backend after compute reservation

Quality Notes

Excellent scope boundaries: skill explicitly documents what it does NOT do (no provisioning, no serving stack, no automation)
Clear prerequisite articulation: requires completed booking from ito-compute, stopping if absent
Strong delegation model: all infrastructure concerns delegated to canonical backend (Layer 0.2), reducing attack surface
Device-based auth documented correctly: no credentials in arguments, files, logs, or chat
Well-structured explanation of backend stages with fabric verification as safety gate
Honest about current state: 'Unavailable today' section documents that the feature is scaffolding and instructs not to substitute local runners
Security-forward stance: 'never launch on unverified metal' and block on fabric degradation show thoughtful risk mitigation
Minor clarity improvement needed: 'Graphiti memory' is unexplained jargon; context would help agents understand where endpoint registration occurs

Model: claude-haiku-4-5-20251001Analyzed: Aug 7, 2026

Reviews

Add this skill to your library to leave a review.

No reviews yet

Be the first to share your experience.

ito-inference