Skip to content

Overview

With our Containers service, you can create your own inference endpoints to serve your models while paying only for the compute that is in active use. We support loading containers from any registry and are quite flexible about how the container is built — including your own Container Registry.

Deploying a container turns an existing image into a scaling API endpoint. Here's the fastest path to a working one.

  1. Go to Serverless Containers -> New deployment and name it.
  2. Pick a Compute Type with enough VRAM for what you're running.
  3. Point it at a public image to start — for example the official vLLM Docker container (docker.io/vllm/vllm-openai), toggled Public. Deploying your own image instead? See Container registries for adding credentials.
  4. Set the exposed HTTP port and health check to match what your container serves on (for vLLM: port 8000, health path /health).
  5. Deploy, then generate an inference API key under Credentials -> Inference API Keys and call the endpoint shown on the deployment page.

Always deploy with a specific version tag (e.g. :1.0) — Verda rejects :latest.

For the full walkthrough this is based on, see Quickstart: deploy with vLLM. Building your own image first? Start with Publish your first Docker image.

Pricing

You're billed for every minute in which a replica processed any usage, including the time spent spinning up or down. The number of currently running replicas will depend on your scaling settings. Charges are aggregated and displayed on your bill in 10-minute intervals. See here for pricing.

Features

  • Scale to hundreds of GPUs when needed with our battle-tested inference cluster
  • Scale to zero when idle, so you only pay while your container is running
  • Support for any container registry, using either registry-specific authentication methods or a vanilla Docker config.json-style auth
  • Both manual and request queue-based autoscaling, with adjustable scaling sensitivity
  • Logging and metrics in the console
  • RESTful API for managing your deployments
  • Python SDK
  • Support for async / polling requests
  • Shared storage between the Containers and Cloud GPU instances
  • Batch jobs - recommended for long inference durations > 3min