Beta9 vs Modal vs AWS Lambda: Serverless GPU Comparison for AI Inference

By Wingston Sharon | March 2026

Running inference at scale has a frustrating economics problem. Keep GPUs warm and you pay for idle time. Let them cool and your users wait 30–120 seconds for cold starts. The serverless GPU platforms have emerged specifically to solve this — pay only for active inference time, with the platform handling scaling and cold start optimization.

There are now three credible options for most EU-based AI teams: Modal (US-based, excellent DX), AWS Lambda with GPU support (the safe enterprise choice), and Beta9 (EU-native, open-source, optimized for heterogeneous hardware including Apple Silicon).

This comparison is based on running actual workloads — Llama 3.1 8B inference, FLUX.1 image generation, and Whisper Large v3 transcription — across all three platforms. I'll focus on what actually matters for production: cold start behavior, pricing transparency, EU compliance, and the specific failure modes each platform has.

I'll be clear where I'm biased: we built Beta9. I'll also be clear about where Modal genuinely outperforms it.

The Core Trade-off: DX vs Control vs Cost

Before the numbers, it helps to understand the philosophy behind each platform:

Modal is built for developer experience. Their Python SDK is genuinely excellent — you decorate a function with @app.function(gpu="A100") and it works. Cold starts are fast because they've invested heavily in container caching and image optimization. The pricing is US-market-optimized, which means EU teams pay for transatlantic latency and face CLOUD Act exposure.

AWS Lambda with GPU is not actually "serverless GPU" in the traditional sense — you're provisioning Graviton instances in a Lambda-compatible wrapper, with GPU support via SageMaker integration. It's the most operationally familiar choice for teams already in the AWS ecosystem, but the cold start behavior for GPU workloads is poor and the pricing is complex.

Beta9 takes a different approach: a distributed GPU scheduler that treats external contributors (bare metal, cloud, Mac Apple Silicon) as first-class workers alongside owned infrastructure. It's open source, self-hostable, and designed for EU workloads with GDPR compliance built in rather than bolted on.

Cold Start Latency

Cold start is the most important operational metric for conversational inference. Users tolerate 2–3 seconds. They don't tolerate 30.

Test setup: Llama 3.1 8B (Q4_K_M quantized, 4.7GB), first request after 10 minutes of inactivity, measured time to first token.

What these numbers mean: Modal's cold start performance is genuinely good. Their container image caching is sophisticated — if the same container image has run recently on that A10G instance, the "cold start" is really just model loading from local cache.

Beta9 with external contributors is faster because contributor nodes are often partially warm — the machine is running, the model may already be in memory from a recent job. The Mac M3 Max numbers reflect MLX's efficiency advantage for Apple Silicon: model loading via MLX is faster than the CUDA path for comparable memory bandwidth.

AWS Lambda's numbers are poor because the GPU provisioning step happens at request time. This is the fundamental problem with wrapping non-serverless infrastructure in a serverless interface.

Caveat: These numbers vary significantly by time of day, geographic region, and queue depth. P95 latency is more important than P50 for user-facing applications.

Pricing Comparison

GPU compute pricing is notoriously hard to compare because providers price differently (per second vs per minute vs per hour), charge differently for cold starts, and have different network egress pricing.

I'll use a standardized workload: 1 million tokens of Llama 3.1 8B inference per day, assuming 50% of requests are "warm" (running container) and 50% are "cold" (first request after idle).

Assumptions:
- Average generation: 200 tokens/request, 5,000 requests/day
- Average generation time: 8 seconds at 25 tok/s on A10G-class GPU
- Cold start fraction: 50% (conservative for intermittent workloads)
- Network egress: 1GB/day

Important caveats:
- Modal pricing above is from their public pricing page; actual invoices vary based on container caching efficiency
- AWS pricing includes SageMaker inference endpoint costs; using just Lambda with GPU is more complex to price
- Beta9 OCI pricing reflects actual EUR costs on Oracle Cloud EU Frankfurt region
- Beta9 external contributor pricing assumes $AGENTO token rate at presale pricing

The pricing delta between US-based platforms and EU infrastructure is real. EUR 210/month vs $310/month is a 32% difference before exchange rate movement. For teams spending $10K+/month on inference, that's meaningful.

EU Compliance and Data Residency

This is where the platforms diverge sharply, and where most EU teams underestimate risk.

Modal: US company (Y Combinator-backed), primary infrastructure in US AWS regions. EU region available (eu-west-1 via AWS), but the parent company is US-incorporated and subject to CLOUD Act. Legal analysis: Modal can be compelled by US government to disclose EU customer data even when stored in EU datacenters, under 18 USC 2713.

AWS Lambda + SageMaker: Same CLOUD Act exposure as any AWS service. AWS is a US company. Data stored in eu-central-1 (Frankfurt) is still accessible to US government under valid legal process. AWS publishes transparency reports showing national security request volumes.

Beta9 (self-hosted): If you self-host Beta9 on EU infrastructure with a EU-registered operating entity, you have maximum data residency control. No US jurisdiction involved. This is the approach Agentosaurus uses — Beta9 on OCI Frankfurt, operating entity Dutch KVK registration, no US subprocessors.

Beta9 (Agentosaurus hosted): We operate from Oracle Cloud Frankfurt (EU data center) with no US data routing. Our legal entity is Dutch. We are not subject to CLOUD Act as a non-US company. GDPR Article 30 records of processing maintained.

For EU teams handling:
- Personal data under GDPR → Modal and AWS have CLOUD Act exposure risk
- Health data, financial data → EU-sovereign infrastructure is not optional
- Public sector data → Many contracts prohibit US-cloud subprocessors explicitly

This matters more than it used to. Post-Schrems II, the legal basis for EU→US data transfers is under sustained challenge. Building on US-cloud infrastructure is a bet that GDPR enforcement stays lenient toward large US providers.

Developer Experience

Honest assessment, not cheerleading:

Modal wins on DX, clearly. Their Python SDK is the best in the category. You can go from idea to deployed serverless function in 15 minutes. Their documentation is excellent. Their community is active. If you're building a prototype or a startup where iteration speed matters more than everything else, Modal is the right choice.

# Modal: genuinely this simple
import modal

app = modal.App()

@app.function(gpu="A10G", image=modal.Image.debian_slim().pip_install("vllm"))
def run_inference(prompt: str) -> str:
    from vllm import LLM
    llm = LLM("meta-llama/Llama-3.1-8B")
    return llm.generate([prompt])[0].outputs[0].text

Beta9 DX is functional but rougher. The SDK works, container builds are fast, and the Python client is straightforward. But the documentation gaps are real, error messages are sometimes cryptic, and the self-hosted setup requires more infrastructure knowledge.

# Beta9: similar pattern, more configuration
from beta9 import endpoint, Image

@endpoint(
    cpu=1,
    memory=4096,
    gpu="T4",
    gpu_count=1,
    image=Image.from_registry("nvcr.io/nvidia/cuda:12.4.0-runtime-ubuntu22.04")
        .pip_install(["vllm"]),
    name="llama-inference",
)
def run_inference(prompt: str) -> str:
    from vllm import LLM
    llm = LLM("meta-llama/Llama-3.1-8B")
    return llm.generate([prompt])[0].outputs[0].text

AWS Lambda DX is poor for GPU workloads. Lambda was designed for CPU-bound, stateless functions. GPU support via SageMaker is bolted on, and the programming model mismatch shows. Setting up SageMaker inference endpoints is significantly more complex than either Modal or Beta9.

Heterogeneous Hardware Support

This is Beta9's clearest differentiation.

Modal runs on NVIDIA hardware exclusively (A10G, A100, H100). AWS Lambda + SageMaker similarly. Both platforms have no Apple Silicon support and no ARM GPU support.

Beta9 was designed from the start for heterogeneous compute pools:
- NVIDIA GPUs (T4, A10G, A100, H100) via CUDA
- Apple Silicon (M1/M2/M3/M4) via MLX
- AMD GPUs (via ROCm, experimental)
- CPU-only nodes for preprocessing and postprocessing

This matters for two reasons:

1. Cost optimization: For inference, Apple Silicon often delivers better cost-per-token than NVIDIA hardware at moderate scale, because the hardware cost is amortized differently (own vs rent). A Mac Studio M2 Ultra purchased outright runs Llama 3.1 70B at roughly EUR 0.12/million tokens after hardware amortization. A100 cloud rental is approximately EUR 1.20/million tokens.

2. Contributor network: The distributed GPU model only works if contributor hardware is diverse. Mac-owning developers, NVIDIA gaming rig owners, and OCI GPU instance operators can all contribute to the same job queue if the scheduler handles hardware heterogeneity.

Failure Modes and Reliability

Every platform has failure modes. Knowing them before you depend on the platform prevents production incidents.

Modal failure modes:
- Cold start latency spikes during high-demand periods (everyone wants A100s at 9am)
- Container build failures due to dependency conflicts (PyPI versions are not pinned by default)
- EU region has lower capacity than US region; some GPU types unavailable
- Service outages affect all customers (no isolated deployment option)

AWS Lambda + SageMaker failure modes:
- Provisioning failures when GPU capacity is constrained (happens more than AWS admits)
- Cold start latency is highly variable (24s median, 90s+ at p99 during peaks)
- SageMaker endpoint management is operationally complex; drift between dev and prod configurations
- Cost surprises are common; metering is opaque

Beta9 failure modes:
- Self-hosted setups require operational expertise; no managed ops team
- External contributor nodes are less reliable than owned infrastructure (contributor disconnects)
- Documentation gaps mean debugging requires reading source code
- Smaller community means fewer Stack Overflow answers

Key reliability difference: Modal and AWS are managed services — you give up control for reliability guarantees. Beta9 gives you full control, which means full responsibility for availability.

For production at scale, the managed service trade-off usually makes sense. For teams that need EU data sovereignty or want to use Apple Silicon hardware, the control trade-off is worth it.

When to Choose Each Platform

Choose Modal when:
- You're in the US or don't have strict EU data residency requirements
- Developer experience and iteration speed are top priorities
- You're prototyping or in early stages (fast to start, easy to leave if needed)
- You need NVIDIA A100/H100 without capital expenditure
- Your team doesn't have infrastructure engineering capacity

Choose AWS Lambda + SageMaker when:
- You're already deeply in the AWS ecosystem and switching cost is high
- You need AWS compliance certifications for enterprise contracts
- Your workload is bursty but predictable (reserved capacity helps)
- Your team knows SageMaker and the operational cost is already paid

Choose Beta9 when:
- EU data sovereignty is required (GDPR strict interpretation, public sector contracts)
- You have or want to use Apple Silicon hardware (Mac Studio, MacBook Pro)
- You want to access the contributor network (cheaper at scale)
- You need heterogeneous hardware scheduling
- You want to self-host and avoid vendor lock-in
- Your inference volume is high enough that 30%+ cost savings matter

The Numbers Side-by-Side

Try Beta9

Beta9 is open source: github.com/Wingie/beta9

Agentosaurus runs a hosted version with EU data residency guarantees and access to the distributed contributor network. If you're evaluating options for production inference with EU compliance requirements, we offer a pilot period.

Request access or email infrastructure@agentosaurus.com.

Related Reading

• How to Earn EUR 10–50/Day with Your Idle Mac GPU — Joining the contributor network
• Mac M3 as a GPU Server: Running Real AI Workloads on Apple Silicon — Setting up Mac hardware for inference
• Why the CLOUD Act Makes EU AI Infrastructure Vulnerable — The compliance context
• Distributed GPU Architecture: Beta9 at Scale — Technical deep dive

Wingston Sharon is the founder of Agentosaurus and a contributor to Beta9. Pricing data was collected from public pricing pages and direct testing in February–March 2026; prices change frequently. This comparison reflects one team's production experience and should not be treated as a definitive benchmark — your workload characteristics, team expertise, and compliance requirements should drive your decision.