Bedrock LLM Monitor - Probe Pipeline

37-channel LLM probing platform (v2.19.2) - serving path and measurement path fully separated

Bedrock LLM Monitor - Probe Pipeline 37-channel LLM probing platform (v2.19.2) - serving path and measurement path fully separated VPC private subnets - ap-northeast-2 (Seoul) Operator · browser (HTTPS only) · Architecture component Operator browser (HTTPS only) CloudFront + WAF · VPC Origin edge · Architecture component · Edge stack CloudFront + WAF VPC Origin edge Edge stack Internal ALB · HTTPS :443, not public · VPC private subnets - ap-northeast-2 (Seoul) Internal ALB HTTPS :443, not public Next.js Dashboard · Fargate / 9 screens · VPC private subnets - ap-northeast-2 (Seoul) · ECS service Next.js Dashboard Fargate / 9 screens ECS service FastAPI Backend · Fargate / probe status API · VPC private subnets - ap-northeast-2 (Seoul) · ECS service FastAPI Backend Fargate / probe status API ECS service RDS PostgreSQL 16 · t4g.micro Single-AZ · VPC private subnets - ap-northeast-2 (Seoul) RDS PostgreSQL 16 t4g.micro Single-AZ EventBridge Scheduler · 4 schedules: 5m / 5m / 15m / 12h · Architecture component EventBridge Scheduler 4 schedules: 5m / 5m / 15m / 12h Probe Tasks (one-off) · AutoProber / Insights / Parity / GptBench · VPC private subnets - ap-northeast-2 (Seoul) · Fargate RunTask Probe Tasks (one-off) AutoProber / Insights / Parity / GptBench Fargate RunTask Bedrock Runtime · 17 channels (Global/US) · Architecture component Bedrock Runtime 17 channels (Global/US) Anthropic CP on AWS · 7 channels · Architecture component Anthropic CP on AWS 7 channels Bedrock Mantle (OpenAI) · 13 GPT channels · Architecture component Bedrock Mantle (OpenAI) 13 GPT channels HTTPS VPC Origin UI routes /api routes read ProbeRun ecs:RunTask store results Legend Frontend Backend Database Cloud Security Message bus External

Measure / Serve Separation

  • • Probers are one-off Fargate tasks, not backend threads (ADR-003)
  • • Dashboard reads only recent ProbeRun rows in RDS
  • • A prober failure can never take the dashboard down

37 Active Channels

  • • Bedrock inference profiles: 17 (Claude Global/US + Nova)
  • • Anthropic Claude Platform on AWS: 7
  • • OpenAI GPT via Bedrock Mantle: 13

Evidence-based Parity

  • • 12h sweep: 6 API surfaces x 19 features
  • • Supported only on execution evidence, never HTTP 200
  • • Verdicts: supported / unsupported / broken / skipped

Architecture diagram • Built with Archify • Create yours ↗ • Hover to trace • R route • Click to focus • +/− zoom • M radar • [/] views • P play story • T theme • E export