- Engineering
Burst packet loss versus random packet loss
Build seeded exact-count loss fixtures and compare gap shape, recovery evidence, and task damage without inventing codec results.
- Author
Udit Goenka- Editorial date
- Reading time
- 25 min

TL;DR: A packet-loss rate does not describe how missing media is arranged. Compare scattered and consecutive losses under the same expected population and loss budget, retain the generated drop identities, and measure actual decoder and task outcomes separately from the synthetic impairment model.
Two tests drop the same number of audio packets. One produces short scattered gaps; the other removes a continuous portion of a critical instruction.
Their average loss rates match, but the listener does not receive the same evidence. Recovery can depend on adjacent packets, and a long missing run can remove a whole value that isolated losses leave partly recoverable.
Begin with a paired impairment fixture. Fix the expected packet population and dropped count, change the arrangement, and keep the decoder, source audio, deadline and task constant.
The media transport guide explains the protocol boundary. This article supplies a seeded generator, exact-count assertions, gap-run accounting and a protocol for testing the next boundary with a real codec adapter.
All fixture values are synthetic. No generated mask is a measured network distribution, no local shape statistic is a listening score, and no private implementation or production result from 60db Agent is described.
Table of contents
- Define the damaged media population
- Separate rate from run length
- Choose a named impairment model
- Run the seeded fixture generator
- Keep timing damage separate
- Connect the mask to a decoder
- Evaluate critical speech fields
- Preserve reproducible evidence
- Test difficult loss boundaries
- Run a controlled network trial
- Report the comparison honestly
- Specify the receiver outcome ledger
- Build paired trials that answer one question
- Frequently asked questions
- Start with one paired trace
Define the damaged media population
Define what one packet means in the experiment before counting loss. A network packet, RTP packet, codec frame, application message and played audio block can have different boundaries.
RFC 3550 defines RTP sequence and timing context. A test adapter must preserve enough stream identity to distinguish an omitted expected packet from a packet belonging to another stream or restart.
The reference generator uses normalized finite sequence positions. It does not parse RTP headers, extend wrapping sequence numbers, recover timestamps, or determine how much audio each real packet represents.
Those are explicit adapter responsibilities. Establish them before applying the generated mask, since dropping a network packet that contains several media units can damage a different population from dropping one normalized unit.
The chunk-size guide develops that distinction. If packet duration varies, save duration per identity so a packet-count comparison cannot silently become a claim about equal damaged speech duration.
Keep the expected population even when every packet is dropped. Counting only received packets makes total loss disappear from the denominator because the receiver has no ordinary arrival events to report.
A sender observation helps establish expected media, but its boundary still matters. Generated, accepted by a local writer, placed on the network, received, and admitted for playback are separate observations.
When a retry or redundancy mechanism creates more network traffic, preserve original media identity as well as network packet identity. Otherwise extra repair traffic can appear to improve the denominator while the same source content remains missing.
Use synthetic source content for deterministic tests. A harmless canary phrase can identify a missing critical field without publishing a real account value, caller recording, or task identifier.

The figure keeps damage generation separate from recovery. A mask says which identities were withheld; it does not establish how a particular decoder rendered the remaining media.
Separate rate from run length
Loss rate counts missing identities relative to an expected population. Run length describes how those missing identities cluster in sequence, which can change the available context for recovery and understanding.
A contiguous gap can remove the entire middle of a word or identifier. Scattered gaps can affect several different portions of speech, so neither arrangement is universally easier without the actual codec and workload.
Do not convert that observation into a ranking without a trial. The source signal, packetization, codec state, redundancy, deadline and listening task all influence the consequence of the same loss mask.
A gap-run histogram records the length and frequency of consecutive missing runs. Its sum weighted by run length should equal the number of unique dropped identities in the fixture.
The maximum run gives another inspectable feature. It should remain visible even when the average rate across the full call is small enough to look harmless.
A call-level average can dilute a short severe event with a long undamaged period. Keep the local damaged interval and its source position when investigating a failed critical value.
The jitter-buffer guide explains another way a gap can arise: media can arrive but miss its useful deadline. Missing-at-source, network loss and late discard deserve distinct evidence.
The WebRTC statistics specification defines concealment and related counters at their own boundary. Replacement samples do not by themselves identify whether the original cause was permanent loss or late arrival.
A developer discussion about robotic audio without apparent packet loss illustrates why the symptom needs a trace. Its historical implementation details should not be reused as a current browser guarantee.
Choose a named impairment model
Exact-count random placement chooses a fixed number of distinct positions from a finite population. It is useful for paired comparisons because both candidates have the same realized drop count.
A Bernoulli model makes an independent drop decision for each position under a configured probability. Its realized count can vary, so it answers a different question from an exact-count fixture.
A single contiguous burst is a deliberately simple shape stressor. More complex burst processes can model transitions between states, but their parameter meanings and realized traces still need to be retained.
The netem documentation distinguishes random and state-based loss options. Selecting a model name does not prove it represents your production network; qualification needs observed evidence for the intended conditions.
The public WebRTC simulated-network tests provide inspectable examples of separate random and burst fixtures. Their constants are synthetic test settings, not recommended loss budgets.
The table compares generator properties. A check means the property is supplied by the named construction; a cross means it is absent or needs separate evidence.
| Criterion | Exact-count placement | Bernoulli process | Single contiguous burst |
|---|---|---|---|
| Fixed realized drop count | ✓ Explicit budget | ✗ Random realization | ✓ Explicit budget |
| Independent decisions per packet | ✗ Sampling without replacement | ✓ Under declared model | ✗ Consecutive arrangement |
| Long consecutive damage guaranteed | ✗ Not guaranteed | ✗ Not guaranteed | ✓ For the declared burst |
| Reproduces actual network behavior | ✗ Needs validation | ✗ Needs validation | ✗ Needs validation |
| Predicts decoder quality alone | ✗ No | ✗ No | ✗ No |
Name the process in every exported fixture. Calling all three random loss makes a result difficult to reproduce and can imply a statistical interpretation that the generator never provided.

Run the seeded fixture generator
Save the full fence as loss_fixtures.py and run Python 3.11 or later. The program creates immutable drop masks, reports run lengths, and checks boundary cases without installing a codec or changing a network interface.
The trusted inputs are a finite population, an explicit budget or probability, a process name and a seed. A real adapter owns mapping normalized positions to media and executing the decoder or transport experiment.
import math
import random
from dataclasses import dataclass
@dataclass(frozen=True)
class Trace:
population: int
dropped: tuple
model: str
seed: int
def __post_init__(self):
if type(self.population) is not int or self.population < 1:
raise ValueError("positive population required")
if type(self.dropped) is not tuple:
raise ValueError("immutable mask required")
if any(type(x) is not int or not 0 <= x < self.population
for x in self.dropped):
raise ValueError("invalid drop identity")
if tuple(sorted(set(self.dropped))) != self.dropped:
raise ValueError("unique sorted identities required")
def exact(population, budget, seed, burst=False):
if (type(population) is not int or population < 1
or type(budget) is not int or not 0 <= budget <= population
or type(seed) is not int or type(burst) is not bool):
raise ValueError("invalid fixture parameters")
rng = random.Random(seed)
if burst and budget:
start = rng.randrange(population - budget + 1)
dropped = tuple(range(start, start + budget))
else:
dropped = tuple(sorted(rng.sample(range(population), budget)))
return Trace(population, dropped,
"single_burst" if burst else "exact_count_placement", seed)
def bernoulli(population, probability, seed):
if (type(population) is not int or population < 1
or type(probability) not in (int, float)
or not math.isfinite(probability) or not 0 <= probability <= 1
or type(seed) is not int):
raise ValueError("invalid Bernoulli parameters")
rng = random.Random(seed)
return Trace(population, tuple(i for i in range(population)
if rng.random() < probability),
"bernoulli", seed)
def runs(trace):
lengths = []
previous = None
for identity in trace.dropped:
if previous is not None and identity == previous + 1:
lengths[-1] += 1
else:
lengths.append(1)
previous = identity
return tuple(lengths)
def tests():
for population in (1, 2, 8, 31):
for budget in range(population + 1):
for seed in (0, 1, 7):
scattered = exact(population, budget, seed)
burst = exact(population, budget, seed, burst=True)
assert len(scattered.dropped) == len(burst.dropped) == budget
assert sum(runs(scattered)) == sum(runs(burst)) == budget
assert exact(population, budget, seed) == scattered
assert runs(burst) == ((budget,) if budget else ())
assert runs(Trace(8, (0, 2, 3, 7), "explicit", 0)) == (1, 2, 1)
assert runs(Trace(4, (0, 1, 2, 3), "explicit", 0)) == (4,)
assert runs(Trace(4, (), "explicit", 0)) == ()
assert exact(4, 4, 0, True).dropped == (0, 1, 2, 3)
assert bernoulli(4, 0, 0).dropped == ()
assert bernoulli(4, 1, 0).dropped == (0, 1, 2, 3)
for bad in (
lambda: exact(0, 0, 0), lambda: exact(4, 5, 0),
lambda: exact(4, -1, 0), lambda: exact(4, 1, True),
lambda: bernoulli(4, float("nan"), 0),
lambda: bernoulli(4, 1.1, 0),
lambda: Trace(4, (1, 1), "explicit", 0),
lambda: Trace(4, (4,), "explicit", 0),
lambda: Trace(4, [1], "explicit", 0),
):
try:
bad()
raise AssertionError("invalid fixture accepted")
except ValueError:
pass
print("synthetic loss-mask and gap-run fixtures passed")
if __name__ == "__main__":
tests()
population, budget, seed = 16, 4, 7
for trace in (exact(population, budget, seed),
exact(population, budget, seed, burst=True)):
print({
"evidence": "synthetic",
"fixture_id": "paired-loss-demo-v1",
"generator_version": "loss-mask-v1",
"model": trace.model,
"population": trace.population,
"budget": budget,
"seed": trace.seed,
"dropped_count": len(trace.dropped),
"dropped": trace.dropped,
"runs": runs(trace),
})
The paired exact-count constructors share a budget, not a claim that their probability distributions are the same. Their purpose is to isolate arrangement while preserving one realized count.
After the checks, the entry point prints two records for one synthetic workload: the exact-count placement and the single burst. Each record includes the fixture and generator versions, population, budget, seed, dropped tuple and run lengths. Recompute its drop count from the tuple and its total damaged positions from the sum of the run lengths. These records describe generated masks, not measured network loss or decoded speech quality.
A seed is useful metadata, but save the resulting tuple too. A future runtime or generator change should not force a reviewer to guess which actual packets the original experiment removed.
Keep timing damage separate
A drop mask identifies media deliberately removed from the fixture. It does not describe when surviving media arrives, how it is reordered, or whether the receiver can use it before the playout deadline.
Keep arrival timing as another explicit input. A trace can have no dropped identities and still cause replacement audio because the surviving packets arrive after the deadline.
Conversely, a missing packet can be recovered from later redundant data if the actual mechanism and deadline permit it. The generated drop count should remain the same while recovery outcomes are recorded separately.
The recovery mechanism guide develops those distinctions. Do not reduce missing source data, recovered payload, concealment and timely playback to one count called repaired packets.
For reordering tests, keep sequence position and arrival order distinct. A later sequence position can arrive first without being a loss, and a receiver may need to wait within policy before deciding whether the earlier position is unavailable.
A bounded wait can create a late-discard outcome. That is a receiver admission decision, so report its policy rather than claiming that the packet disappeared from the network.
The jitter-buffer replay supplies a small fixed-deadline model. Compose its timing evidence with this mask instead of pretending either model alone explains every audible gap.
Keep clock domains explicit. A packet capture timestamp, local monotonic timer and media timestamp can support different calculations; subtract only values with a declared common basis or justified mapping.
Connect the mask to a decoder
A real codec trial begins with a valid encoded source fixture. Verify its baseline decode before applying loss so a format mismatch cannot masquerade as poor recovery.
The Opus codec specification and Opus RTP payload format define relevant representation and parameter boundaries. Use the tested implementation's supported API and record its version.
Keep encoder and decoder configuration together. Negotiating a capability, emitting redundancy, making it available before a deadline, and using it in the decoder are different observations.
An older community question asks how to enable and prove Opus in-band recovery. Use that as a concrete question to answer with current adapter evidence, not as a current Chrome compatibility statement.
RFC 8854 distinguishes audio recovery mechanisms and their constraints. Its guidance does not establish how your particular stream behaved under a particular generated mask.
For each expected media identity, record primary data present, supporting recovery data present, decode mode, output disposition and deadline outcome where the adapter exposes them. Preserve unknown when the adapter cannot establish a fact.
Do not count packet-loss concealment as reconstruction of the original missing content. It creates replacement samples, which may be useful for continuity while still altering a critical field.
Keep actual decoded output for an authorized listening review. A successful decoder call can establish that output was produced without proving that it preserved the intended word or that the listener understood it.

Evaluate critical speech fields
Aggregate transcription error can hide whether the lost segment contained the one value the task needed. Identify critical fields in the fixture before listening or running recognition.
A synthetic phrase can contain a harmless color, day or fictional item code. Save the expected canonical value separately from the audio so the evaluator can distinguish a wrong value from a merely awkward pronunciation.
Use identical source audio across paired masks. If the burst covers a critical field in one trial while a different source recording moves the field elsewhere in the other, the experiment no longer isolates the arrangement alone.
Run more than one source position when the task requires coverage. A burst at the beginning, middle or end can damage different acoustic context and response conditions.
The multilingual evaluation guide helps preserve critical values across languages. Keep language, accent and speaking style as declared workload dimensions rather than merging them into an unexplained average.
A human listening review and an automatic recognizer answer different questions. Recognition can fail on understandable audio, and a plausible transcript can hide uncertainty or a mistaken critical value that needs clarification.
The spoken answer guide provides a listener-focused rubric. Include whether the agent asks for a needed repair instead of confidently acting on damaged evidence.
A controller that declines every impaired input can avoid some wrong actions while failing usefulness. Preserve both safety and task continuation outcomes in the evaluation.
The outcome metrics guide supplies that separation. A decoder-quality trial should not be reported as completed business-task success without observing the additional boundaries.
Preserve reproducible evidence
Save the fixture's model name, parameters, seed, generated mask and generator version. Include expected population and media duration so the same file cannot be interpreted under two incompatible denominators.
The source audio and codec settings belong in the same evidence bundle. A drop mask without the encoded fixture can describe a shape while leaving the listening result impossible to reproduce.
For timing, retain the original arrival observations and their clock domain. A derived label such as late is useful, but the timestamp and deadline explain why the label was correct.
For receiver metrics, keep stream and receiver identity, epoch, raw cumulative counters, units and availability. Same SSRC after a restart does not justify reusing a previous receiver baseline.
The voice observability guide describes boundary-oriented events. Capture the smallest evidence needed for reproduction without putting transcripts or account data into ordinary metric labels.
Use synthetic identities in public examples. Real addresses, phone numbers, task identifiers and source recordings can reveal a person or customer even when names have been removed.

A missing field should remain missing. Substituting zero for an unsupported recovery counter creates a false negative observation and can make one implementation look worse simply because it exposes less telemetry.
Test difficult loss boundaries
The zero-drop fixture proves the undamaged path and the generator's empty-mask behavior. It should not produce an empty-media outcome because the expected population still exists.
The all-drop fixture proves the opposite boundary. A pipeline must retain the full expected denominator even when it has no surviving input to decode or play.
A budget larger than the population is invalid. Silently clipping it changes the requested experiment and makes the saved parameters disagree with the realized mask.
Duplicate drop identities are invalid in the explicit mask representation. Counting them twice inflates the damage without removing additional media, so the immutable trace validates uniqueness and order.
A burst touching the final packet should not wrap to the beginning unless wraparound is explicitly part of a separately named process. The reference chooses a contiguous in-range segment and preserves its exact length.
If a later generator supports multiple bursts, define overlap handling. Overlapping intervals can produce fewer unique dropped identities than their requested lengths imply, which breaks a fixed realized-budget comparison unless accounted for.
Use an explicit mask to cover a known critical boundary. Random trials are useful for workload variation, while a deterministic regression case should keep the exact arrangement that previously exposed the defect.
The voice regression guide develops that practice. Preserve failed cases rather than rerolling a seed until the pipeline looks healthy.

Run a controlled network trial
A local mask generator is a safe starting point because it changes no live network state. A real transport trial needs an isolated environment and evidence that the intended impairment reached the tested direction.
Record whether loss is applied before encryption, at an RTP adapter, at an interface, or elsewhere. Those positions can affect repair traffic, feedback, congestion control and the population observed by the receiver.
A downstream-only impairment differs from a return-path impairment. Feedback can be delayed or lost too, so save which direction and traffic class the tool changed.
Do not apply an example impairment to a shared production interface. Use an isolated test route or namespace, capture the original configuration, and restore it when the trial ends.
Check the actual realized damage. A requested tool setting is configuration; the observed dropped identities and arrival sequence are evidence, and the two can differ under scheduling or tool limitations.
The latency benchmark guide explains why transport, device and warm/cold conditions belong in a report. Keep the same discipline when an impairment trial affects response timing.
Test application output age after a stall. The backpressure guide shows why old buffered speech should not be replayed merely because connectivity improves.
For reconnection, the network-change guide keeps transport recovery separate from response authority. A loss experiment must not silently retry a committed business action just because a call route failed.
Report the comparison honestly
Lead with the comparison performed. State whether the trial held a realized drop count fixed, sampled an independent probability, or used a state-based burst model.
Report the generated arrangement and its duration, not only its average rate. Include maximum run and the damaged source position when those explain a critical-field failure.
Keep impairment, decoder, listening and task outcomes in separate columns. A successful decode call is weaker evidence than correct critical-field recognition, and neither alone proves authorized task completion.
The cost-per-success guide adds the operational denominator. Repeated clarification and failed calls can matter even when the nominal codec output remains continuous.
This guide reports no codec ranking, MOS improvement, production loss rate or customer result. Its executable program verifies mask construction and gap accounting, while the described trials define what to measure next.
Specify the receiver outcome ledger
A receiver outcome ledger should preserve what happened to each expected media unit without turning uncertainty into a success or failure. Start with the normalized source identity and record which later observations are available for that identity.
The first question is whether primary encoded data reached the declared receiver boundary. A packet capture at another host can support a different fact, so keep the observation location beside the event rather than using one received flag for the entire path.
The second question is whether the data was timely. A packet can arrive intact after its intended use, and a late discard is different from the generator deliberately withholding the packet.
The third question is whether supporting recovery data existed and could be used. Keep this separate from negotiated capability because a compatible codec configuration does not prove the particular payload contained useful redundancy.
The fourth question is what the decoder did. It may decode primary data, recover from supporting encoded content, generate replacement samples, fail, or expose insufficient evidence to classify the mode.
The fifth question is whether the resulting output reached the declared playback boundary on time. Decoder success alone cannot answer that question when a consumer or output queue waits after decoding.
These questions can be columns in an existing fixture report. They do not require a new service or elaborate schema platform, but each column should have a name, owner and explicit unknown value.
A boolean recovered flag is often too broad. It can accidentally mean capability was negotiated, repair data was received, a decoder API returned samples, or the listener heard the intended value, depending on which component wrote it.
Name those observations separately. That makes it possible to compare mechanisms without awarding a recovery success merely because one implementation exposes a capability flag while another exposes actual decoder evidence.
Do not derive human comprehension from a complete output ledger. A fully played sequence can still obscure a critical field or contain a plausible replacement for a missing syllable, so task evaluation remains a separate record.
For cumulative receiver counters, retain the original values and the observation epoch. A restart can reuse an SSRC while resetting or continuing counters in a way that makes an old baseline unsafe.
Compare only compatible observations under the declared identity rule. If the epoch changes, begin another interval rather than allowing a coincidentally increasing counter to appear as uninterrupted evidence.
Keep a zero delta distinct from an unavailable field. Zero concealment under a supported observation interval is a fact about that interval; no concealment metric is a limitation of the evidence.
Store the realized impairment mask with the receiver ledger. That join lets an engineer investigate whether a missing identity was supplied by redundancy, replaced locally, or simply excluded from an incorrectly defined population.
A redundant packet can carry information for more than one media identity. Preserve the mapping the actual adapter can establish rather than counting one network packet as one repaired source unit by assumption.
If the mapping is unavailable, report the observable aggregate without inventing a per-unit claim. The report can still be useful when its boundary is explicit and the missing detail remains visible.
A listener's correction can also reveal a task-level failure after media recovery looked successful. Keep that response in the evaluation rather than treating a later successful clarification as proof the first impaired answer was correct.
The confirmation repair guide shows why a positive word followed by a correction remains an unfinished acceptance. Network damage should not cause the controller to commit a business action from a truncated affirmative prefix.
When an action outcome is unknown, reconcile with its owning system. The workflow enforcement guide keeps that authority separate from audio recovery and prevents an impairment test from turning a transport retry into a duplicate real-world effect.
Build paired trials that answer one question
Choose the question before generating a large corpus. A trial about the effect of consecutive gaps should hold the realized loss budget fixed, while a trial about a stochastic network model can intentionally allow its realized count to vary.
Use the exact-count constructors for the first question. Save both masks and assert their equal populations before any audio result is inspected, so an unfavorable result cannot be explained away by an unnoticed difference in total damage.
Then choose source positions deliberately. A consecutive gap over a critical number and a scattered mask outside that number are different task conditions even when their packet counts match.
To evaluate general effects, distribute those positions across a declared corpus and retain the position as an analysis dimension. To reproduce one defect, keep the precise mask and source position fixed.
The first approach explores a workload; the second locks a regression. Combining them without labels can make a deterministic failure disappear among many harmless cases or make one targeted failure look like an estimate of typical network behavior.
Keep the source fixture immutable during a paired comparison. Reencoding with different settings can change frame boundaries or recovery content, so save the actual encoded input rather than regenerating it from a text phrase each time.
Record the encoder and decoder versions as well as the negotiated parameters. A capability that appears in signaling may be unused under a particular mode, and the actual decode path needs evidence from the adapter.
Run a valid baseline through the same adapter. If the unmodified fixture already loses a critical value or violates timing, repair that cause before attributing later differences to the generated mask.
For a listening task, define the expected answer without showing it to the listener in advance. A reviewer who already knows the fictional code can fill in a missing sound from expectation and hide the defect you intended to measure.
Use a separate factual reviewer where needed. That reviewer can inspect source and canonical values, while the listening reviewer encounters the audio under conditions closer to the intended user experience.
Preserve individual disagreements. If listeners identify different values from the same impaired phrase, a single average pleasantness score would erase the ambiguity that the application needs to handle.
Include a clarification outcome in the task contract. A system that recognizes uncertainty and asks the caller to repeat a damaged field can be safer and more useful than one that guesses a plausible value immediately.
The spoken self-correction guide explains how a later repair should replace the accepted field. The earlier damaged interpretation should not remain hidden in a tool argument or retrieval filter after the caller corrects it.
Keep the evaluation denominators explicit. You can report the fraction of completed listening trials with a correct critical field, provided disconnected and incomplete trials remain visible alongside that conditional population.
Do not count a missing response as a correct refusal unless the controller produced the task's declared unavailable or clarification behavior. Silence can result from a decoder crash, a stalled queue or a disconnected route.
Separate those outcomes before comparing policies. A candidate that avoids wrong answers by failing to respond in every difficult case has not established a useful conversational improvement.
Capture latency at a named boundary when timing is part of the decision. Time to decoded samples, time to local output acceptance, and time to a useful heard answer are different observations and need different evidence.
A real network trial can add feedback and congestion-control effects that a local mask does not simulate. Treat disagreement between the two as a reason to inspect the additional boundary rather than forcing the live result to match the simplified model.
The head-of-line diagnosis guide supplies another counterexample. A pause may come from ordered complete-message delivery or a blocked consumer even when the generated packet mask contains no dropped media.
Before release, preserve the smallest set of fixtures that distinguishes the approved behavior from the known failure. Keep the more extensive workload protocol separately so future changes can run focused regression tests first and broader qualification when the affected contract warrants it.
The published generator leaves no background processes or network configuration behind. A later real trial should have the same operational discipline: record what it starts or changes, stop owned processes, restore the isolated interface and retain the evidence needed to understand the result.
Before archiving a paired trial, verify that the exported drop identities are sufficient to reconstruct the mask without the generator. A seed is useful for repeating the same implementation, while the explicit identities preserve the realized experiment across library versions or future generator changes. Store both, and retain the offered population and generator version beside them.
Ask a reviewer to distinguish the requested impairment from the receiver's observed outcome. The generator may request a drop that reaches a different interface than intended, or a receiver may see additional timing damage. Preserve those observations separately rather than modifying the requested mask to match the result. A disagreement is diagnostic evidence about the harness or environment.
For speech review, keep the impairment location visible to the analyst while preventing it from becoming a cue for the listener's answer. The listener should judge the actual output using the declared task procedure. The analyst can later align the judgment with damaged units, decoder behavior and deadlines. That separation reduces the temptation to fill in a missing word simply because the source transcript is known.
A comparison is ready to reuse when its source material, population, mask, timing model, adapter contract and scoring boundary can be reconstructed. It remains a synthetic trial until real traffic or a declared capture is measured, and its findings remain limited to the tested configuration. Those limits make the evidence portable without turning one pattern into a claim about every network.
Frequently asked questions
Why can equal loss rates sound different?
The same count of missing packets can be scattered or consecutive. Recovery and listening depend on which source content and adjacent support were removed, so an average rate cannot describe the complete damage pattern.
What is a packet-loss burst?
A burst is a consecutive run of missing identities under a declared sequence population. Its meaning needs packet duration and source position when the experiment intends to describe damaged audio rather than network counts alone.
Is random placement Bernoulli loss?
Exact-count random placement samples a fixed number of distinct positions. Bernoulli loss makes independent decisions under a probability and can produce a different realized count, so name the process explicitly.
How do I hold the loss budget fixed?
Choose the expected population and a valid dropped count before generating either mask. Assert that both realized masks contain exactly that many unique in-range identities, then compare arrangement separately.
Why retain the actual drop mask?
The mask records which media identities the experiment withheld. A seed and configuration are useful, but storing the realized mask protects reproduction against generator and runtime changes.
Does a seed guarantee cross-version reproducibility?
A seed helps reproduce a named generator under its supported runtime contract. Preserve the generated output and version rather than assuming every future implementation produces the identical sequence.
How do packet durations affect damage?
Equal packet counts represent equal missing duration only when their media durations are equal. Variable packetization needs per-identity duration accounting before a count comparison can support an audio-duration claim.
What does a gap-run histogram show?
It records consecutive missing-run lengths and their frequencies. The run lengths should sum to the unique drop count, while the maximum run remains visible beside the average loss rate.
How should an all-loss trace be reported?
Retain the expected population and report that no primary media survived. Do not remove the trial from the denominator merely because the receiver produced no normal arrival events.
Can a burst cross a fixture boundary?
It can only if the generator contract explicitly supports that behavior. The reference keeps one burst in range and never wraps its tail to the beginning.
Are late packets counted as lost?
Late arrival and permanent absence are different observations. Keep late discard separate from the generated drop mask so receiver deadlines do not become unexplained network-loss counts.
Does concealment recover original audio?
Concealment produces replacement samples under the decoder's behavior. Those samples can support continuity without reconstructing the original missing content or preserving a critical spoken value.
How should reordering be separated?
Save sequence position and arrival order independently. Reordering can be evaluated without removing any expected packet, then combined with a declared deadline to determine which arrivals remain usable.
Can one trace establish codec quality?
One trace establishes a bounded case under one source, configuration and decoder. A broader quality claim needs representative workloads, actual output evidence and an evaluation protocol with explicit omissions.
Should zero-loss fixtures be included?
Zero-loss fixtures verify the baseline media and decoder path before impairment. They help prevent a format or scheduling defect from being blamed on the loss generator.
What does netem change?
Netem applies supported network impairments at its configured boundary. The actual realized arrival and drop evidence still needs observation, and tests should isolate the route and restore configuration afterward.
Why test an early burst and a tail burst?
Startup and end-of-stream can expose different decoder and recovery behavior. A final missing unit may lack later supporting data, while an initial burst can affect state establishment.
Which raw counters should be saved?
Save supported cumulative loss, discard, concealment and recovery counters with receiver identity, epoch, units and availability. Compute deltas only across compatible observations and keep zero distinct from unavailable.
How do critical-field errors differ from WER?
A critical-field error describes a value needed for the task, while aggregate word error rate treats a broader transcript population. A small overall error can still remove or change the decisive field.
Where should a regression corpus start?
Start with an undamaged baseline and one paired exact-count scattered/burst fixture. Add the precise source position and mask that expose a real defect, then retain the complete reproduction bundle.
Start with one paired trace
Choose a synthetic source phrase and an exact loss budget. Run the generator, save both masks, and check their realized populations before connecting them to a real decoder adapter.
The next useful evidence is a comparison that preserves damage shape, decoder behavior and the listener's critical-field outcome separately. Use that record to build a regression case instead of promoting one average loss rate into a universal network-quality claim.
