Load testing bursty call arrivals

TL;DR: Schedule arrivals independently of response completion, retain scheduled and actual start times, and count work the generator could not offer. Separate generator saturation from application refusal, then report queue age, successful response boundaries, expiry, cancellation, and recovery against the complete offered schedule.

---

A load test reports healthy response time because each virtual caller waits for a response before starting another call. When the service slows down, the test sends less work, so the workload becomes easier at the moment it should expose saturation.

A second test schedules independent arrivals but runs out of generator workers. It reports only the requests that started, and the missing arrivals disappear from the summary.

Start with a schedule ledger. The voice outcome metrics guide explains why a useful denominator includes the population offered to the system; burst testing also needs the population that the generator intended to offer.

This guide provides a consent-gated synthetic replay function, negative fixtures, a saturation report template, and a workload design method. It can support evaluation of an application using 60db Agent without placing unapproved live calls or claiming a production benchmark.

The public Grafana k6 open and closed workload documentation describes the distinction between arrival-driven and completion-driven schedules. Choose the model that answers the workload question rather than assuming one fixed virtual-user count reproduces a burst of new calls.

Table of contents

1. Define the arrival process
2. Keep call behavior separate from arrival timing
3. Protect the test boundary
4. Run a bounded synthetic replay
5. Distinguish two saturation points
6. Measure the full voice path
7. Build a saturation report
8. Test recovery and cancellation storms
9. Choose the next capacity change
10. Frequently asked questions
11. Keep the missing arrivals visible

Define the arrival process

An arrival process specifies when new calls attempt to enter the system. It is separate from how long those calls remain connected or how many turns each caller takes after admission.

A constant-rate workload is useful for steady-state capacity questions. A burst workload needs a time-varying schedule or explicit arrival offsets that represent a brief concentration of offered work.

Do not label a ramp as an instantaneous burst. A gradual increase gives reactive scaling and caches time to adapt, while simultaneous arrivals exercise the capacity already available at the start of the event.

Use several shapes when the operational question requires them. A short spike tests immediate headroom, a sustained step tests adaptation, and repeated waves test whether cleanup and queues recover before the next wave arrives.

The cold-start capacity guide connects arrival timing to useful ready slots. A burst that ends before new resources pass readiness mainly tests the warm floor and waiting policy.

Define schedule units explicitly. An offset in seconds from the experiment's local start is easy to replay, while wall-clock campaign timestamps need conversion and a documented policy for clock changes.

Keep equal-time arrivals. They represent a simultaneous offered cohort under the schedule and should not be spread out merely because the generator is easier to implement that way.

Record actual start time separately. A generator that starts a request late has changed the delivered workload, and that lateness belongs in the report rather than being hidden by replacing scheduled time with start time.

The Grafana ramping-arrival-rate reference documents a public way to express changing arrival rates. The small reference in this guide instead accepts explicit offsets so its synthetic fixtures remain inspectable without another dependency.

When using a sampled schedule, retain the seed and generation rule. Do not present a random workload as a reproduction unless another engineer can generate the same arrival sequence and workload assignments.

Choose the time resolution from the failure you are trying to expose. Aggregating arrivals into a minute can conceal a brief simultaneous cohort that exhausts available slots, while recording many decimal places does not improve the fidelity of a generator whose actual starts are delayed by its own event loop.

Keep the arrival schedule as data rather than embedding it in a collection of sleeps. Data can be reviewed, hashed for reproducibility, assigned workload classes, and replayed by another harness without reconstructing the timing logic from execution history.

A schedule can also include a quiet recovery interval. That interval should remain in the experiment window so the report captures how long resources and queues take to return to baseline after the last offered arrival, rather than ending at the visually dramatic peak.

A fixed seed does not make a workload realistic. Its distribution and correlations still need a reason, such as an approved aggregate arrival pattern or a deliberately labeled stress scenario.

Use synthetic schedules when production evidence is unavailable. State the shape and the question it tests instead of inventing a customer cohort or implying that a hand-built burst is an observed business pattern.

The conceptual figure separates arrival control from response completion. Slower service should remain visible as occupancy, refusal, or delay rather than silently reducing the intended arrival schedule.

Keep call behavior separate from arrival timing

An open arrival model creates new attempts independently of previous completions. A closed model maintains a population of users or workers that often wait for one operation before beginning the next.

Both can be useful. Closed behavior can represent a caller taking turns inside one conversation, while open arrivals can represent independent people starting new calls during a campaign window.

A voice workload often needs both levels. The outer schedule creates calls, and each admitted call follows a bounded synthetic turn script whose timing depends on its own conversation state.

Do not count every model request as a new call. One call can create several model, synthesis, and tool operations, so report those populations separately when diagnosing which resource saturates.

The admission control guide names the resource reservation boundary. A load test should use the same unit when comparing active occupancy with the configured capacity limit.

Keep service demand visible. Prompt size, output length, language, tool use, and audio duration can change occupancy, so a burst of short greeting requests does not establish capacity for long tool-heavy conversations.

Use a small workload matrix with explicit weights when you need a mix. Retain the assignment of each scheduled call to its class so a rerun does not accidentally compare different service demand under the same nominal arrival rate.

Correlations can matter. If the burst also begins with cold language paths or a cache invalidation, model that condition explicitly instead of assuming that independent random choices reproduce it.

The cache compatibility guide explains why cached work depends on context. A warm-cache burst and a miss-heavy burst are different experiments even if both offer the same number of calls per second.

Inside a call, preserve turn ownership. An interruption can cancel one response and start another, so the workload script should not keep consuming obsolete output merely to maintain a fixed request count.

The self-correction guide provides a useful behavioral fixture. A synthetic caller can revise a value while work is pending, allowing the test to exercise cancellation and fresh admission under the same call identity.

Separate correctness from throughput in the script. A response that arrives quickly with the wrong generation's speech is a failure at the conversation boundary, even when the transport and model request both report success.

Protect the test boundary

Run load only against environments and services that the owner has authorized for that workload. Authorization includes provider quotas, test duration, maximum concurrency, permitted destinations, and the absence or explicit approval of business side effects.

The reference function refuses to run unless its caller supplies an explicit consent flag. That flag is a local guard for the example, not a substitute for the real approval and destination controls of a deployed harness.

Use synthetic identities and content. Avoid real phone numbers, customer recordings, private transcripts, credentials, and tool payloads in both the request corpus and the report.

A voice test can cause effects beyond compute. An outbound call, a callback request, a CRM update, or a human handoff can create cost or contact another person, so use the application's existing harmless test route or a dedicated synthetic adapter.

The workflow enforcement guide explains why signed requests and valid transport do not authorize business actions. Keep that separation when a load harness reaches a tool endpoint.

Bound the generator's active operations. A test that creates an unbounded task for every missed arrival can exhaust the test host and distort the workload before it reaches the application.

Bound each operation's duration too. The adapter must cooperate with cancellation and use its transport's own timeout behavior, because a local timeout cannot guarantee that remote work released its resources.

The batch cancellation guide explains that cleanup boundary. Stop new arrivals first, invalidate unneeded output, and reconcile already started work under the public runtime's supported cancellation route.

Define a stop condition before the run. It can include a maximum duration, a resource ceiling, an unexpected side effect, or a correctness failure that makes continued traffic unhelpful.

Keep the stop procedure observable. Record how many arrivals were not attempted after the stop and which active operations still needed cleanup, rather than allowing the interrupted test to resemble a completed low-load run.

The reference runs only in-process synthetic adapters. A live transport adapter is a separate integration task with its own destination allowlist, credentials, cleanup, and evidence boundaries.

Run a bounded synthetic replay

The replay accepts a sorted list of relative arrival offsets and an asynchronous adapter. It schedules work independently of completion, refuses excess generator concurrency, and returns immutable raw rows in scheduled identity order.

A generator-dropped row means no adapter call was made for that scheduled identity. A rejected row means the adapter ran and returned a service refusal, so the two saturation boundaries remain distinguishable.

Run the complete fence with Python 3.11 or later as python burst_replay_reference.py. The fixtures use only local coroutines and make no network calls.

import asyncio
from collections import Counter
from dataclasses import dataclass, FrozenInstanceError
import math


@dataclass(frozen=True, slots=True)
class Row:
    identity: int
    scheduled: float
    started: float | None
    finished: float | None
    outcome: str


def finite(value):
    if type(value) not in (int, float) or not math.isfinite(value):
        raise ValueError("expected finite number")
    return value


async def replay(offsets, adapter, *, consented, max_inflight=4, timeout=1,
                 collector=None):
    if consented is not True:
        raise PermissionError("explicit workload consent required")
    if type(max_inflight) is not int or max_inflight < 1 or finite(timeout) <= 0:
        raise ValueError("invalid generator limits")
    schedule = tuple(finite(offset) for offset in offsets)
    if any(offset < 0 for offset in schedule) or schedule != tuple(sorted(schedule)):
        raise ValueError("offsets must be sorted and nonnegative")
    if collector is not None and (type(collector) is not list or collector):
        raise ValueError("collector must be an empty caller-owned list")
    loop = asyncio.get_running_loop()
    origin = loop.time()
    rows, inflight, tasks, launched = {}, set(), [], set()

    async def execute(identity, due):
        started = loop.time() - origin
        try:
            outcome = await asyncio.wait_for(adapter(identity), timeout)
            if outcome not in ("completed", "rejected"):
                outcome = "adapter-error"
        except asyncio.CancelledError:
            outcome = "canceled"
            raise
        except TimeoutError:
            outcome = "timeout"
        except Exception:
            outcome = "adapter-error"
        finally:
            rows[identity] = Row(identity, due, started,
                                 loop.time() - origin, outcome)

    try:
        for identity, due in enumerate(schedule):
            delay = origin + due - loop.time()
            if delay > 0:
                await asyncio.sleep(delay)
            if len(inflight) >= max_inflight:
                rows[identity] = Row(identity, due, None, None, "generator-dropped")
                continue
            launched.add(identity)
            task = asyncio.create_task(execute(identity, due))
            inflight.add(task)
            tasks.append(task)
            task.add_done_callback(inflight.discard)
        await asyncio.gather(*tasks)
    finally:
        for task in tasks:
            if not task.done():
                task.cancel()
        await asyncio.gather(*tasks, return_exceptions=True)
        for identity, due in enumerate(schedule):
            if identity not in rows:
                outcome = "canceled-before-start" if identity in launched else "not-attempted"
                rows[identity] = Row(identity, due, None, None, outcome)
        ordered = tuple(rows[i] for i in range(len(schedule)))
        if collector is not None:
            collector.extend(ordered)
    return ordered


def summarize(rows):
    counts = Counter(row.outcome for row in rows)
    started = [row for row in rows if row.started is not None]
    return {"raw": tuple(rows), "scheduled": len(rows), "started": len(started),
            "counts": dict(counts),
            "start_lateness": tuple(row.started - row.scheduled for row in started),
            "service_durations": tuple(row.finished - row.started for row in started)}


@dataclass(frozen=True, slots=True)
class CounterSample:
    receiver: str
    completed: int


def counter_delta(before, after):
    for sample in (before, after):
        if (type(sample.receiver) is not str or not sample.receiver or
                type(sample.completed) is not int or sample.completed < 0):
            raise ValueError("invalid cumulative sample")
    if before.receiver != after.receiver:
        raise ValueError("receiver changed")
    if after.completed < before.completed:
        raise ValueError("counter reset")
    return {"before": before, "after": after,
            "completed_delta": after.completed - before.completed}


async def expect_error(error, action):
    try:
        await action()
    except error:
        return
    raise AssertionError("invalid replay accepted")


async def tests():
    calls = []
    async def ok(identity):
        calls.append(identity)
        return "completed"
    raw_schedule = [0, 0, 0]
    rows = await replay(raw_schedule, ok, consented=True, max_inflight=2)
    raw_schedule.append(9)
    report = summarize(rows)
    assert report["scheduled"] == 3 and report["started"] == 2
    assert report["counts"] == {"completed": 2, "generator-dropped": 1}
    assert calls == [0, 1] and report["raw"] == rows
    assert rows[2].started is None and rows[0].scheduled == 0
    try:
        rows[0].outcome = "changed"
    except FrozenInstanceError:
        pass
    else:
        raise AssertionError("row rebinding accepted")
    assert all(value >= 0 for value in report["start_lateness"])
    assert all(value >= 0 for value in report["service_durations"])
    async def refuse(_):
        return "rejected"
    async def broken(_):
        raise RuntimeError("synthetic adapter failure")
    async def invalid(_):
        return "unknown"
    async def blocked(_):
        await asyncio.Event().wait()
    assert (await replay([0], refuse, consented=True))[0].outcome == "rejected"
    assert (await replay([0], broken, consented=True))[0].outcome == "adapter-error"
    assert (await replay([0], invalid, consented=True))[0].outcome == "adapter-error"
    assert (await replay([0], blocked, consented=True, timeout=0.001))[0].outcome == "timeout"
    assert await replay([], ok, consented=True) == ()
    await expect_error(PermissionError, lambda: replay([0], ok, consented=False))
    for bad in ([-1], [1, 0], [float("nan")], [True]):
        await expect_error(ValueError, lambda bad=bad: replay(bad, ok, consented=True))
    await expect_error(ValueError, lambda: replay([0], ok, consented=True, max_inflight=0))
    await expect_error(ValueError, lambda: replay([0], ok, consented=True, timeout=0))
    # Cancel while future arrivals remain unscheduled; retain all identities.
    ledger, entered = [], asyncio.Event()
    async def pending(identity):
        if identity == 0:
            return "completed"
        entered.set()
        await asyncio.Event().wait()
    run = asyncio.create_task(replay([0, 0, 60], pending, consented=True,
                                     collector=ledger))
    await entered.wait()
    run.cancel()
    await expect_error(asyncio.CancelledError, lambda: run)
    assert [row.outcome for row in ledger] == ["completed", "canceled", "not-attempted"]
    assert summarize(ledger)["scheduled"] == 3
    # Cancel during final gather; all scheduled adapters remain accounted for.
    ledger, entered = [], asyncio.Event()
    run = asyncio.create_task(replay([0, 0], pending, consented=True, collector=ledger))
    await entered.wait()
    run.cancel()
    await expect_error(asyncio.CancelledError, lambda: run)
    assert [row.outcome for row in ledger] == ["completed", "canceled"]
    # A task canceled before its first step differs from an unattempted arrival.
    ledger = []
    run = asyncio.create_task(replay([0], pending, consented=True, collector=ledger))
    asyncio.get_running_loop().call_soon(run.cancel)
    await expect_error(asyncio.CancelledError, lambda: run)
    assert ledger[0].outcome == "canceled-before-start"
    assert ledger[0].started is None
    before, after = CounterSample("receiver-a", 0), CounterSample("receiver-a", 2)
    delta = counter_delta(before, after)
    assert delta["completed_delta"] == 2 and delta["before"] is before
    assert counter_delta(before, before)["completed_delta"] == 0
    for left, right in ((after, before), (before, CounterSample("receiver-b", 2)),
                        (before, CounterSample("receiver-a", None))):
        try:
            counter_delta(left, right)
        except ValueError:
            pass
        else:
            raise AssertionError("invalid counter delta accepted")
    print("burst replay, outcome accounting, and counter fixtures passed")


asyncio.run(tests())

Pass an empty caller-owned collector list when the run can be canceled. Cleanup fills it with one immutable row per scheduled identity before the cancellation propagates, preserving completed, canceled, canceled-before-start, and not-attempted outcomes. Read the collector after awaiting the replay task; do not mutate it while the run is active.

This local reference assumes one cancellation request and cooperative adapters. Repeated forced cancellation or process termination can interrupt cleanup, so a real experiment coordinator needs durable outcome storage outside this process.

The schedule is captured as an immutable tuple before the first asynchronous wait. The returned rows preserve scheduled identities and original timing values so a report can recompute start lateness and service durations.

The simultaneous-burst fixture has a deliberately small worker budget. It schedules three identities at the same offset, starts two local adapter calls, and records the remaining identity as generator-dropped; those counts are synthetic consequences of the configured cap, not a throughput measurement.

That fixture verifies the denominator before any performance interpretation. The returned tuple still contains one row for every scheduled identity, including the row with unavailable start and finish timestamps, so a consumer cannot accidentally mistake two completed rows for the entire offered population.

The refusal fixture reaches a different boundary. Its adapter runs and returns rejected, while the exception fixture produces adapter-error, and the timeout fixture starts work that never reaches its normal endpoint within the cooperative waiting budget.

Keep those cases separate when adapting the code. An HTTP refusal, a connection failure before the server sees the request, and an unknown outcome after a connection closes need distinct adapter evidence even if a higher-level report groups some of them for an additional summary.

The generator's cap is conservative within one event-loop scheduling turn. A simultaneous burst can exceed its worker budget before any adapter runs, which is intentional and tested; increase the budget only when the authorized test requires more offered concurrency.

The Python event-loop documentation defines the loop's monotonic timing interface. The reference uses that one local domain and does not subtract timestamps from another process or host.

The Python task documentation explains timeout and cancellation behavior. In particular, an adapter that suppresses cancellation can delay cleanup, so this example requires cooperative bounded adapters rather than claiming a hard remote execution deadline.

Distinguish two saturation points

The generator saturates when it cannot start the intended arrival schedule. The application saturates when offered work exceeds useful service capacity under its admission and waiting policy.

A generator-dropped arrival never reached the adapter. An application rejection reached the service boundary and received a refusal, so counting both as one generic error hides which system needs attention.

The public k6 dropped-iterations guide describes how unavailable virtual users can prevent scheduled work from starting. Its meaning is useful even when you use a different generator with its own names.

A real community question makes the confusion concrete. A Grafana forum thread about dropped and interrupted iterations asks how to eliminate both, illustrating why the report must separate generator capacity, request duration, and test shutdown.

Do not copy that thread's numerical results into your capacity recommendation. It describes another workload and environment, while your experiment needs its own scheduled counts and resource evidence.

Measure generator CPU, memory, connection limits, and scheduling lateness when using a live adapter. A host struggling to maintain timers can smear a sharp burst into a slower arrival wave even without reporting explicit drops.

The k6 virtual-user allocation guide explains that arrival-rate execution still needs enough workers. A configured rate is an intention until the generator demonstrates that it delivered the schedule.

Keep late starts visible even when they complete successfully. Service latency measured from actual start excludes the time the generator spent missing the intended schedule, so it cannot establish the response time for the original offered arrival.

The conceptual accounting figure separates generator admission from adapter outcome. The first gate asks whether the generator started the work; the later outcomes describe adapter evidence without assuming that every failed attempt reached the remote service.

If the generator saturates first, stop interpreting the run as a service capacity ceiling. Fix the authorized generator budget or reduce the test scope, then repeat the same schedule and compare actual start evidence.

Measure the full voice path

A call arrival is earlier than model execution. Connection setup, authentication, routing, admission, media negotiation, turn detection, generation, synthesis, and output delivery can each contribute delay or failure.

Choose the useful endpoint before starting the run. The latency benchmark guide distinguishes first model output from first received audio and caller-perceived response, so preserve those boundaries in the saturation report.

A synthetic in-process adapter measures only its coroutine boundary. It cannot establish WebRTC negotiation, SIP media behavior, browser audio-device readiness, or acoustic playback.

The SIP troubleshooting guide explains why a connected call does not prove working two-way audio. A live transport experiment should retain the media evidence needed for its declared endpoint instead of treating signaling success as a completed voice path.

The mobile browser latency guide adds device and browser conditions. Keep those cohorts separate from server-only tests because permission and output-device behavior can change the observed boundary.

Record queue age and active reservations at the constrained stage. A flat successful-response percentile can coexist with increasing refusals or an old backlog, so report those dimensions together.

Record interruptions and stale output separately from transport errors. The interruption guide defines the correctness boundary that a saturation test must preserve while the service is busy.

Retain cumulative counter identity when deriving rates. The OpenTelemetry metrics data model describes metric identity and temporality, which are relevant when joining samples across a run.

The reference's counter helper rejects a receiver change or counter reset and preserves both original samples. It keeps a valid zero delta separate from unavailable data rather than substituting zero for a missing observation.

Do not average worker averages without their denominators. Aggregate raw counts and duration sums under the declared population, or use compatible histogram data; a mean of means can change when worker traffic is uneven.

Build a saturation report

Start with the workload contract: schedule identifier, units, shape, duration, configuration mix, synthetic content policy, authorized target, generator cap, timeout, and measured response endpoint.

Then report delivery fidelity. Include scheduled arrivals, actual starts, generator drops, start lateness, and any stop condition that prevented the remaining schedule from being offered.

Report service dispositions against actual starts and against the full schedule where useful. The two denominators answer different questions, so label them rather than presenting one unexplained success percentage.

Use this template as a report structure. Unavailable means the boundary was not observed; it is not a zero result or a claim that nothing failed.

Checks mean the published synthetic reference retains the evidence. Crosses identify measurements that require a real application adapter and are not established by the local fixture.

The synthetic fixture can populate the schedule fields immediately: three scheduled identities, two actual starts, two completed local adapters, and one generator drop. The application queue-age and audio cells remain unavailable because the fixture never crossed either boundary.

Do not fill those cells from a different test run. A separately measured audio percentile belongs to its own workload and environment unless the report establishes the join, and combining it with this local fixture would create an apparent end-to-end result that no experiment observed.

Attach a disposition definition to each report column. Completed can mean an adapter returned its declared endpoint, while verified task success needs the separate business ledger; readers should not have to infer the difference from a footnote or a filename.

Use a time series around the burst when the question is recovery. Aggregate percentiles over a long quiet period can dilute the short saturation window, so retain the window boundaries and the workload offered inside each one.

Keep cohort counts beside those windows. A later interval with few admitted requests can appear faster because the service refused most arrivals, and the report should make that selection visible before comparing its response distribution with the earlier interval.

A useful evidence bundle contains the schedule, raw rows, configuration manifest, analysis code, and a short narrative of the first saturated boundary. It should be possible to rebuild every reported count and duration without running the workload again or accessing private customer content.

For latency, name the successful subset and sample count. A timeout did not reach the endpoint, so retain it as an outcome and a waiting lower bound rather than placing the timeout threshold into a successful latency distribution.

The k6 threshold documentation describes executable pass/fail criteria. Set criteria from the application's operating contract, including schedule fidelity and correctness, instead of borrowing an attractive percentile threshold from another workload.

Include the initial and final inventories. A run that ends with pending cleanup or queued work has different operational meaning from one that returned to its baseline after the burst.

Separate test validity from service acceptance. A run can fail validity because the generator missed too much of its schedule, while another valid run can reveal that the service violates its waiting contract; those outcomes require different fixes and should not share one unexplained red badge.

Write both sets of criteria before execution. Validity criteria cover schedule fidelity, complete accounting, permitted target, and available boundary evidence, while service criteria cover the application's declared refusal, delay, correctness, and recovery expectations under that workload.

A threshold is a decision rule, not a measured fact. Record the rule's owner and rationale, keep it stable for the comparison, and avoid adjusting it after seeing a disappointing run merely to label the experiment successful.

Keep the report reproducible. Retain the deployed revision, adapter version, workload file, safe raw rows, counter identities, and exact analysis script, while excluding customer payloads and credentials.

Test recovery and cancellation storms

A burst experiment should continue observing after arrivals stop. Queue drain time, cleanup delay, returned capacity, and the response to a later small probe reveal whether the service recovered or merely stopped receiving work.

The admission control guide gives the lease-release contract. A canceled request can lose output authority before it releases resources, so recovery accounting must not assume every cancel intent immediately creates a free slot.

Run a wave that causes requests to expire while waiting. Verify that the service exposes those expiries and does not later process them as fresh work when the queue becomes shorter.

Run a wave with interruptions. Verify that sibling requests retain ownership, stale output is suppressed, and a late callback cannot restore a terminal request.

The batch cancellation guide provides those negative fixtures. Under load, the same invariants matter more because delayed callbacks and shared execution become easier to overlap.

Run a cache-miss wave and a prepared-cache wave separately. A cache invalidation can concentrate dependency demand without a corresponding increase in call arrivals, so the report should name which state the test exercised.

Run a deployment transition only when it is within the authorized test scope. The shutdown guide separates closed admission from termination and helps define what existing calls should retain while workers drain.

Keep the recovery probe harmless and independent. A successful HTTP status from a health endpoint does not prove that the voice response boundary recovered, so use the declared representative path.

Repeated waves can expose a failure that a single spike misses. If each wave leaves a small amount of retained work, the next wave begins with less useful capacity even though the peak arrival shape is unchanged, so record the inventory immediately before each wave.

The same principle applies to retries. A retry wave can arrive after the original burst and overlap with its cleanup, so a workload that deliberately includes retries should preserve their relationship to the original identity and keep them separate from new callers.

Keep the recovery experiment bounded by its stated objective. Once it demonstrates leaked or unreconciled work, preserve the evidence and stop the run; sending a larger wave rarely explains the missing release boundary better than inspecting the existing request ledger.

A run that never recovers needs its pending inventory preserved. Stop adding traffic, reconcile the public request lifecycle, and report the hard failure rather than increasing the load until the generator or service collapses.

Choose the next capacity change

Use the first saturated boundary to choose the next experiment. Generator drops call for generator evidence; application refusals call for admission and capacity analysis; slow admitted work calls for service and dependency tracing.

The cold-start guide helps when the first wave suffers before new capacity becomes useful. The cache-key guide helps when miss behavior changes the service demand behind an unchanged schedule.

Before tuning, write a hypothesis with an observable consequence. If warm capacity is the suspected limit, the changed run should improve first-wave admission or queue age while preserving the same service behavior; if cleanup is the limit, the changed run should reduce retained leases after cancellations under the same offered schedule.

A smaller successful-request percentile alone is insufficient evidence for either hypothesis. It can result from refusing more work, shortening generated answers, changing the language mix, or starting fewer requests because the generator became saturated first.

Use the synthetic accounting tests as a release check for the analysis pipeline too. If a report transformation drops rows with missing start timestamps, it should fail the fixture because generator-dropped identities are part of the intended denominator.

Test receiver changes before joining metric windows. A worker replaced during the run can restart a cumulative counter, and the correct report should split or explicitly reconcile that boundary rather than producing a negative delta or a fictional surge in completions.

Keep zero and unavailable distinct in exported formats. A spreadsheet cell, JSON field, or chart transform that converts a missing observation to zero can silently undo the careful distinction retained by the raw rows and counter helper.

Change one material policy at a time. Increasing warm slots, changing queue age, and reducing output length together can improve a graph without showing which change caused the improvement or which useful behavior was lost.

Compare the same schedule and workload assignment after the change. Preserve outcome definitions and the response endpoint so the second run cannot appear better merely because it measured a shorter path.

The cost per successful task guide provides the economic denominator. More accepted requests are useful only when the application preserves the required task and conversation correctness.

Do not turn one synthetic run into a universal capacity claim. Report the observed workload, environment, boundary, and limitations, then use that evidence to choose a narrower next test or a bounded rollout.

---

Frequently asked questions

What is a bursty call arrival test?

It offers new call attempts according to an explicitly concentrated time schedule. The schedule can contain simultaneous arrivals, a short spike, or repeated waves, and should remain separate from each call's internal turn behavior and duration.

Why can a fixed virtual-user test hide saturation?

A completion-driven worker waits before starting its next operation, so slower responses can reduce offered work. That model can fit some user behavior, but it does not reproduce independent new calls arriving during a burst.

Is an open workload always better?

It answers independent-arrival questions. A closed model can represent a bounded caller population or sequential turns inside one call, so choose the model from the workload contract and use both levels when the application needs them.

Is a ramp the same as a spike?

No. A ramp gives the system time to adapt while demand increases, whereas a spike tests capacity available near the start of the event. Name the shape and preserve its exact schedule when comparing runs.

Why retain scheduled and actual start times?

Their difference exposes generator scheduling lateness. Measuring only from actual start hides the work the generator delayed, so a successful service-duration percentile can coexist with poor delivery of the intended arrival schedule.

What is a generator-dropped arrival?

It is scheduled work that the generator did not start because its own active-operation limit was reached. It never reached the adapter and must remain separate from a service rejection returned after an actual request began.

How many generator workers should we allocate?

Use the authorized schedule and operation occupancy, then verify actual start fidelity under that budget. A configured rate alone does not prove the generator delivered it, and increasing workers beyond the approved test scope is not an automatic next step.

Does the reference place phone calls?

No. Its published fixtures use local asynchronous functions and make no network calls. A live adapter needs a separately authorized target, destination controls, provider limits, harmless content, and cleanup behavior.

Why does the replay require consent?

It makes the workload boundary explicit before execution. The boolean guard is only a reference safety check; real authorization still comes from the environment owner and the application's approved testing process.

What does the generator timeout prove?

It bounds waiting under cooperative adapter behavior. It does not prove immediate remote cancellation or resource release, so a live adapter needs its transport timeout and a public cleanup contract.

Should timeouts appear as successful latency samples?

No. They did not reach the successful timing endpoint. Retain them as outcomes with their observed waiting evidence, while reporting latency only for the named subset that reached that endpoint.

Should we test first token or first audio?

Choose the boundary that answers the question and label it. First token, first received audio, endpoint playback, and verified business completion are different observations and cannot be compared as one generic response-time metric.

Why keep raw cumulative counters?

They let another engineer recompute deltas and detect resets or receiver changes. A derived rate without original values and identity can hide a process restart or turn unavailable data into a misleading zero.

Can we average worker latency averages?

Only with the correct denominators and compatible populations. Preserve raw counts and sums or compatible histograms so uneven traffic across workers does not distort a fleet-level mean or imply an unsupported percentile.

How should cache state enter the workload?

Label prepared-cache, miss-heavy, and invalidation scenarios separately. The same arrival schedule can create different dependency demand under those conditions, so an unlabeled cache change can confound the capacity comparison.

Why test cancellation during saturation?

Busy systems expose more overlap between waits, late callbacks, and shared execution. Verify that canceled requests lose output authority, siblings continue, and resource leases remain held until the declared cleanup boundary.

What should happen after arrivals stop?

Continue observing queue age, active resources, cleanup, and a harmless representative probe. Recovery is part of the experiment, and pending inventory should remain visible if the system does not return to its declared baseline.

Does a health check prove voice recovery?

Only to the extent that it exercises the declared voice boundary. A generic status endpoint or successful signaling connection does not establish useful media output or business task completion.

How do we compare a capacity change fairly?

Replay the same schedule and workload assignments with the same outcome definitions and response endpoint. Change one material policy at a time and report generator fidelity, refusals, latency subsets, and recovery alongside cost.

What does this guide measure about 60db Agent?

The reference measures local synthetic coroutine behavior and validates its own accounting logic. It does not establish a product capacity limit, provider benchmark, live carrier result, acoustic measurement, or private implementation detail.

Keep the missing arrivals visible

Begin with an independent arrival schedule and a complete ledger. Preserve generator drops, late starts, service refusals, timeouts, and pending cleanup so the test cannot look healthier by silently offering less work.

Run the complete synthetic fence before connecting a live adapter. Then obtain the environment-specific authorization, choose a harmless public boundary, and verify the generator delivered the workload it claims to have offered.

Before a live run, review the workload file with the environment owner. Confirm that every synthetic class uses an approved harmless route, that the generator cap and duration match the authorization, and that the stop procedure can reconcile active requests without discarding their outcome evidence.

After the run, review the raw ledger before polishing charts. Missing scheduled identities, reset counters, or a mislabeled response endpoint can invalidate a tidy summary, and repairing those evidence boundaries is more useful than drawing a smoother curve over incomplete data.

Use the first saturated boundary to choose one next change. Keep the same schedule for the comparison, retain raw evidence, and report the limits of the result when evaluating any voice application, including one built with 60db Agent.