GPU Selection Rules

Automatic selection

When you call run_on_gpu without specifying a GPU, Volspan automatically selects the cheapest tier that can handle your workload. The algorithm considers:

• Declared VRAM requirements (if specified in your code)
• Package imports (e.g., importing large model libraries)
• Historical usage patterns for your agent
• Current availability across GPU tiers

VRAM estimation

Common workloads and their approximate VRAM requirements:

Setting GPU preferences per agent

You can pin a specific GPU tier to an agent in your dashboard under Settings → Agents. This overrides automatic selection for all runs made with that API key.

// Or specify per-call via MCP:
run_on_gpu(
  code="...",
  gpu="A100"  // "4090" | "A100" | "H100"
)

If the requested GPU is unavailable, the run queues until one is free. It will not fall back to a different tier unless you enable allow_fallback: true.

Cost optimization tips

• Use quantized models. A 4-bit quantized 70B model fits on an A100 instead of requiring H100-class memory.
• Batch your work. One 60-second run is cheaper than six 10-second runs due to cold start overhead.
• Start small. Default to 4090 and only move up when you hit VRAM limits or need more throughput.
• Check warm status. Warm workers avoid cold start time. The dashboard shows current warm worker count per tier.