Launch log

Improvement

v3.8

Scale to zero in under four seconds

Cold starts for 70B-parameter models dropped from 19 seconds to 3.6, thanks to streamed weight loading.

Idle deployments cost nothing on Karman, but a slow cold start used to make scale-to-zero a trade-off. Not any more.

Weights now stream from regional cache directly into GPU memory while the container starts, so the first request is served in 3.6 seconds on average.

Who benefits most

Teams with bursty traffic — internal tools, agents and batch jobs — see the largest savings.

Create a free website with Framer, the website builder loved by startups, designers and agencies.