Surviving Finals Week: How We Benchmarked 25,000 Concurrent Exam Submissions

Most SaaS platforms experience relatively gentle traffic curves. Users log in throughout the workday, read documents, chat with colleagues, and log off.

Educational platforms and certification testing engines operate on an entirely different traffic pattern: the catastrophic synchronized spike.

At 11:50 PM on exam night, a university or corporate training cohort has 500 active users. At 11:59:30 PM, exactly thirty seconds before the deadline, 25,000 candidates attempt to submit 40-question exam payloads simultaneously.

On legacy platforms, this is when databases deadlock, HTTP 504 Gateway Timeouts cascade across Nginx, and students take screenshots of error screens to plead for deadline extensions.

Before deploying Cognaxa to multi-campus higher-education institutions, we built a distributed load testing harness to simulate this exact worst-case failure mode. Here is what we learned and how we optimized the stack to survive it.

---

Simulating the Synchronized Stampede

To replicate realistic candidate behavior, we spun up a distributed testing cluster across four AWS regions using k6 and custom headless browser agents:

[ Distributed k6 Cluster (4 AWS Regions) ]
               │
               ▼ (25,000 Virtual Candidates)
[ Cloudflare Global Edge Network (DDoS & TLS Termination) ]
               │
               ▼ (Sub-millisecond Edge Handshake)
[ Load Balancer (Least Connections) ]
               │
               ▼
[ Stateless Node.js / Express Application Cluster (12 Pods) ]
         ┌─────────────┴─────────────┐
         ▼                           ▼
[ In-Memory Redis Cluster ]   [ PgBouncer Connection Pooler ]
(Active Session State Locks)   (Transaction Mode Pooling)
                                     │
                                     ▼
                      [ PostgreSQL Dedicated Primary ]
                      (Row-Level Security + Write Buffering)

The load profile was modeled on observed telemetry from 2025 university exam cycles:

• Phase 1 (0 to 45 mins): 25,000 virtual users steadily answering questions, triggering autosave heartbeats every 15 seconds.
• Phase 2 (45 to 55 mins): Sporadic early submissions (approx. 20% of cohort).
• Phase 3 (59th Minute): The Stampede. 18,500 candidates clicking the final “Submit Exam” button within a 45-second window.

---

Bottle-Neck 1: PostgreSQL Connection Pool Exhaustion

In our initial unoptimized run, the database collapsed within 8 seconds of Phase 3.

Node.js application pods each opened dozens of dedicated PostgreSQL connections. When thousands of requests flooded in, the database exceeded its max_connections limit, spending 92% of its CPU time on context switching rather than query execution.

The Fix: PgBouncer in Transaction Pooling Mode

We introduced PgBouncer configured strictly in pool_mode = transaction.

Instead of holding open an expensive PostgreSQL backend process for the entire lifecycle of an HTTP request, application pods grab a database connection only for the few milliseconds required to execute the write transaction, releasing it back to the pool instantly:

# PgBouncer configuration
[databases]
cognaxa_production = host=10.0.1.10 port=5432 dbname=cognaxa_production pool_mode=transaction

[pgbouncer]
listen_port = 6432
listen_addr = *
auth_type = scram-sha-256
max_client_conn = 30000
default_pool_size = 40
reserve_pool_size = 15
reserve_pool_timeout = 2

This single architectural change allowed 12 application pods to serve 25,000 concurrent submissions over just 55 real database backend connections.

---

Bottle-Neck 2: Write Amplification on Question Submissions

The naive schema wrote each answer selection directly to disk:

-- The slow way: 40 individual INSERT/UPDATE queries per student
UPDATE attempt_answers SET selected_option_id = $1 WHERE id = $2;

Multiply 40 questions by 25,000 candidates, and the database was asked to execute 1,000,000 row writes in under 45 seconds.

The Fix: Batched JSONB Append Operations

We redesigned the submission endpoint to accept a compressed, validated array of answers in a single transaction.

The entire exam submission executes as an atomic upsert:

-- The optimized way: Single atomic batch write with RLS
UPDATE exam_attempts
SET 
  submitted_at = NOW(),
  status = 'SUBMITTED',
  answers_payload = $1::jsonb,
  objective_score = evaluate_objective_answers($1::jsonb, exam_id)
WHERE id = $2 AND tenant_id = current_setting('app.current_tenant_id')::uuid;

By computing the objective score (MCQs, boolean) directly inside the database using an in-memory SQL function and writing the answer array as a single JSONB blob, we reduced disk I/O operations by 94.6%.

---

The Benchmark Results

With PgBouncer transaction pooling, Redis heartbeat buffering, and batched JSONB writes in place, we re-ran the full 25,000-user synchronized finals stampede:

Benchmark Metric	Naive Legacy Architecture	Cognaxa Production Engine
Peak Concurrency	25,000 Virtual Users	25,000 Virtual Users
HTTP 5xx Error Rate	24.3% (Total Collapse)	0.00% (Zero dropped requests)
Median Response Time (p50)	1,840ms	14ms
99th Percentile Response Time (p99)	18,200ms (Timeout)	64ms
Database CPU at Peak Stampede	100% (Pegged)	41% (Stable headroom)

---

What This Means for Enterprise Operations

When a university dean or training director selects Cognaxa, they are not taking a gamble on whether the platform will survive the final hour of the term.

Our architecture has been pushed to the absolute breaking point in synthetic multi-region distributed stress tests and emerged with sub-65ms responsiveness.

Exam day should be stressful for students studying their material—not for administrators praying their software stays online.