GPU Spot Lotto

Seoul control plane dispatching GPU jobs to the cheapest US Spot region

GPU Spot Lotto Seoul control plane dispatching GPU jobs to the cheapest US Spot region Seoul ap-northeast-2 · Control Plane US Spot Regions · us-east-1 / us-east-2 / us-west-2 Amazon Bedrock AgentCore User · browser / CLI · Architecture component User browser / CLI Edge · CloudFront + WAF + ALB · Seoul ap-northeast-2 · Control Plane Edge CloudFront + WAF + ALB API Server · FastAPI · 19 ops · Seoul ap-northeast-2 · Control Plane API Server FastAPI · 19 ops Price Watcher · spot price poll · 30s · Seoul ap-northeast-2 · Control Plane Price Watcher spot price poll · 30s Redis · ElastiCache · prices + queue · Seoul ap-northeast-2 · Control Plane Redis ElastiCache · prices + queue Dispatcher · BRPOP · cheapest region · Seoul ap-northeast-2 · Control Plane Dispatcher BRPOP · cheapest region Reaper · cleanup loop · 10s · Seoul ap-northeast-2 · Control Plane Reaper cleanup loop · 10s S3 Hub Bucket · data / checkpoints · Seoul ap-northeast-2 · Control Plane S3 Hub Bucket data / checkpoints EKS GPU Clusters · Karpenter · Spot only · US Spot Regions · us-east-1 / us-east-2 / us-west-2 EKS GPU Clusters Karpenter · Spot only FSx for Lustre · per-region spoke · US Spot Regions · us-east-1 / us-east-2 / us-west-2 FSx for Lustre per-region spoke MCP Gateway · 6 REST ops as MCP tools · Amazon Bedrock AgentCore · external agents MCP Gateway 6 REST ops as MCP tools external agents Strands Agent · AgentCore Runtime · 5 tools · Amazon Bedrock AgentCore Strands Agent AgentCore Runtime · 5 tools HTTPS LPUSH job ZADD prices BRPOP create GPU Pod pod phase poll DRA auto sync /data (fsx mode) Mountpoint CSI (default) 5 tools MCP calls Legend Backend Database Cloud External

Control plane (Seoul)

  • • Redis sorted set keeps prices ordered on write (ZADD)
  • • Dispatcher takes the cheapest region with atomic capacity DECR
  • • Reaper polls pod phase every 10s and reclaims capacity

Hub-and-Spoke storage

  • • Users upload data straight to the hub via S3 presigned URLs
  • • Default storage_mode=s3 mounts the hub via Mountpoint CSI
  • • storage_mode=fsx uses per-region FSx Lustre with DRA auto import/export

Known caveats

  • • values k8sMode "real" is not recognized by code (expects "live")
  • • requeue_fn is not wired: failed jobs are not rescheduled cross-region