Voice consistency across short and long streaming replies

TL;DR: Evaluate short and long replies with matched text families, frozen synthesis settings, and separate judgments for identity, level, cadence, and intelligibility. Version the audio artifacts and trial manifest so a new voice, changed adapter, or failed capture cannot silently rewrite the comparison.

---

The agent's brief acknowledgment sounds like the expected speaker. A longer explanation begins well, then changes in level, cadence, or apparent identity. A single average quality score cannot tell you which dimension failed or where it changed.

Start with one matched family: a short utterance, a longer utterance that includes the same critical phrase, and explicit opening and tail review regions. Keep synthesis settings fixed. Ask separate questions about speaker identity and understood content.

A public CosyVoice streaming issue reports perceived voice changes that appeared more often in long input than short input. That is a specific community observation from a named project and date, not evidence that every streaming system shares its cause or that any suggested workaround applies to your deployment.

The multilingual voice benchmark method supplies a broader evaluation boundary. This article narrows the work to a versioned listening corpus that can expose reply-length and position effects without inventing model internals.

The artifact below validates synthetic corpus records and reproducible trial ordering. No listening study or acoustic measurement is reported for 60db Agent. The proposed rubric remains a protocol until authorized listeners and captures produce real observations.

Table of contents

1. Define consistency as separate properties
2. Build matched text families
3. Version artifacts and trial identity
4. Inspect the streaming path before listening
5. Run a listening protocol that can fail
6. Analyze changes without hiding dimensions
7. Keep the corpus useful across releases
8. Frequently asked questions
9. Ship evidence for the affected reply

Define consistency as separate properties

Voice consistency is a set of observable properties across controlled conditions. It is not a single hidden quality that one score measures automatically. Define the properties you need to preserve before collecting audio or asking listeners for judgments.

Identity concerns whether the listener perceives the intended speaker or voice character. Level concerns the signal's amplitude and its perceived loudness under an explicit playback condition. Cadence concerns timing and grouping. Intelligibility concerns whether the listener can recover the spoken content.

Those dimensions can disagree. A reply can preserve recognizable identity while becoming too quiet. It can remain easy to understand while sounding like another voice. It can sound smooth while omitting a critical word.

The ITU-T P.800 recommendation provides a public reference for subjective transmission-quality methods. Use its existence as a methodological starting point, not a license to label an informal listening session with a standardized score it did not earn.

For each dimension, write a concrete question. "Does this sound like the reference voice?" addresses identity under a selected reference. "Which value did you hear?" addresses content. "Which sample did you prefer?" is a different outcome and should remain separate.

Keep natural variation distinct from a defect. A speaker can change stress in a question or slow down around an unfamiliar name without losing identity. A corpus should allow expected variation while identifying changes that damage the user task or violate the product's voice policy.

The confirmation prosody guide helps define one task-specific boundary. A clear critical value can justify a local pace change, provided that change is tested rather than silently accepted because it sounds expressive.

Define the comparison reference. It can be an authorized fixed clip, a documented voice preset, or a frozen baseline generation. Without a reference, participants may compare each candidate with their own imagined speaker, producing an answer that is hard to interpret.

Record the allowed variation in the reference itself. If the baseline contains a question, an acknowledgment, and an explanation, the listener can hear the intended expressive range. That is more informative than requiring every utterance to match one flat sentence.

Avoid making biometric claims from a listening preference task. Perceived identity, speaker verification, and legal identity are different boundaries. This guide evaluates a product voice's perceived consistency and does not certify a person's identity or authenticity.

For 60db Agent or another application, work from the public configuration and returned evidence. A voice name is a requested setting. It does not establish that every generated sample used identical processing or that a private implementation remained unchanged.

The session isolation contract covers an important alternative cause. A voice change caused by another call's mutable settings is an ownership defect, not a reason to tune the model's prosody.

Build matched text families

A matched family isolates the comparison you care about. Use a short acknowledgment, a short critical statement, and a longer explanation containing the same statement in a controlled position. Keep the canonical content and intended meaning visible in the corpus.

Do not compare an easy short greeting with a difficult long sequence of numbers and call the difference a length effect. Content difficulty, punctuation, language, and field type can explain the observation. Change one factor at a time or record the factorial design explicitly.

Use opening and tail regions in longer replies. A participant who hears only the first sentence can miss a later failure. Whole-reply listening is still necessary because cropping can remove context that affects perceived cadence and identity.

Include several linguistic roles. Acknowledgments, questions, lists, spelled names, numeric values, and explanatory prose put different demands on synthesis. The sentence-boundary guide explains why segmentation can change the unit the voice performs.

Version punctuation and whitespace exactly. A text cleanup that changes sentence boundaries can change delivery while leaving the visible words similar. Preserve the original UTF-8 text and a stable digest with every audio artifact.

When the application supports multiple languages, define separate matched families for qualified listeners. Do not translate one English phrase mechanically and assume equal difficulty. Local spelling, number naming, and grammatical structure can change the listening task.

Keep customer identifiers, account facts, and private transcripts out of the public corpus. Synthetic fields can reproduce length and pronunciation difficulty without exposing a real person. Clearly label them synthetic so a reader never treats a test value as an actual instruction.

Freeze the synthesis settings for the primary comparison. Include provider-facing voice, model or version when available, language, rate, output format, and any supported style controls. Record unavailable version information as unavailable rather than inventing a stable identifier.

SSML 1.1 supplies a public vocabulary for several speech controls. Provider and voice support still differs. A corpus record should store the exact selected interface and markup rather than an abstract label such as "normal prosody" that cannot reproduce the request.

The speaking-rate experiment helps keep sample-grid changes separate from linguistic pace. A candidate should not receive a different playback ratio merely because its file sounds slower during an informal review.

For a streaming comparison, preserve the submitted text segmentation and the received audio fragmentation as separate fields. One describes the synthesis input; the other describes delivery. Replaying only the final file cannot reproduce every streaming lifecycle problem.

Choose the corpus size based on the decision. A narrow regression can begin with the failing family and meaningful negatives. A broader voice selection needs enough content and qualified listeners to support its stated scope. More files alone do not make the comparison valid.

Before generating the full corpus, run a small dry pass through metadata validation and artifact storage. Confirm the manifest can identify every file, distinguish missing output, and preserve the exact requested settings. Fixing those boundaries early prevents an expensive collection of untraceable clips.

Version artifacts and trial identity

A corpus manifest should preserve immutable records for text, configuration, and audio identity. A file path alone is unstable: a later run can overwrite it while the report still points to the same name. Store a content digest and verify it before using the artifact.

Python's hashlib documentation describes the standard-library digest interface used below. A digest establishes artifact identity under the chosen algorithm; it does not establish that the audio says the intended words or preserves a voice.

The reference supports a bounded manifest for raw signed 16-bit little-endian PCM, mono or stereo, with a positive integer sample rate. Synthetic bytes are used in the tests. It performs no decoding, speech generation, playback, or perceptual scoring.

Save the full fence as voice-corpus.py and run Python 3.11 or later. The fixture verifies immutable snapshots, duplicate detection, frame alignment, digest changes, and reproducible ordering. It is a small manifest contract, not a production audio pipeline.

from dataclasses import dataclass, FrozenInstanceError
from hashlib import sha256
from random import Random

@dataclass(frozen=True)
class Clip:
    family: str
    length: str
    revision: int
    text: str
    settings: tuple
    rate: int
    channels: int
    pcm: bytes
    digest: str

    def __post_init__(self):
        if type(self.family) is not str or not self.family:
            raise ValueError("family required")
        if self.length not in ("short", "long"):
            raise ValueError("length class required")
        if type(self.revision) is not int or self.revision < 1:
            raise ValueError("positive revision required")
        if type(self.text) is not str or not self.text:
            raise ValueError("text required")
        if type(self.rate) is not int or self.rate <= 0:
            raise ValueError("positive integer rate required")
        if type(self.channels) is not int or self.channels not in (1, 2):
            raise ValueError("mono or stereo required")
        if type(self.pcm) is not bytes or not self.pcm:
            raise ValueError("owned nonempty PCM required")
        if len(self.pcm) % (2 * self.channels):
            raise ValueError("incomplete s16le frame")
        if type(self.settings) is not tuple:
            raise ValueError("immutable settings required")
        for pair in self.settings:
            if type(pair) is not tuple or len(pair) != 2:
                raise ValueError("immutable setting pairs required")
            if any(type(value) is not str for value in pair):
                raise ValueError("string setting pairs required")
        if len({k for k, _ in self.settings}) != len(self.settings):
            raise ValueError("duplicate setting key")
        if self.digest != sha256(self.pcm).hexdigest():
            raise ValueError("audio digest mismatch")

def clip(family, length, revision, text, settings, rate, channels, pcm):
    pairs = tuple(sorted((k, v) for k, v in settings.items()))
    owned = bytes(pcm)
    return Clip(family, length, revision, text, pairs, rate, channels,
                owned, sha256(owned).hexdigest())

def trials(clips, seed):
    owned = tuple(clips)
    keys = [(x.family, x.length, x.revision) for x in owned]
    if len(keys) != len(set(keys)):
        raise ValueError("duplicate clip identity")
    result = list(owned)
    Random(seed).shuffle(result)
    return tuple(result)

def rejects(error, fn):
    try:
        fn()
    except error:
        return
    raise AssertionError("expected " + error.__name__)

settings = {"voice": "synthetic-A", "language": "en"}
raw = bytearray(b"\x00\x00\x01\x00")
a = clip("field", "short", 1, "Synthetic field.", settings, 24000, 1, raw)
settings["voice"] = "synthetic-B"
raw[0] = 255
assert dict(a.settings)["voice"] == "synthetic-A"
assert a.pcm == b"\x00\x00\x01\x00"
b = clip("field", "long", 1, "Long synthetic field.",
         {"voice": "synthetic-A", "language": "en"}, 24000, 1, a.pcm)
assert trials([a, b], 17) == trials([a, b], 17)
assert set(trials([a, b], 17)) == {a, b}
assert trials([], 17) == ()
rejects(ValueError, lambda: trials([a, a], 17))
rejects(ValueError, lambda: clip("x", "short", 1, "x", {}, 0, 1, a.pcm))
rejects(ValueError, lambda: clip("x", "short", 1, "x", {}, 24000, 2, b"\x00\x00"))
rejects(ValueError, lambda: clip("x", "short", 1, "x", {}, 24000, 1, b""))
rejects(ValueError, lambda: Clip("x", "short", 1, "x", (), 24000, 1,
                                a.pcm, "wrong"))
rejects(ValueError, lambda: Clip("x", "short", 1, "x",
                                (["voice", "synthetic-A"],),
                                24000, 1, a.pcm, a.digest))
rejects(FrozenInstanceError, lambda: setattr(a, "pcm", b"changed"))
rejects(FrozenInstanceError, lambda: setattr(a, "settings", ()))
print("Synthetic corpus fixtures passed; no perceptual scores computed")

The factory copies mutable bytes and converts the supported string settings into immutable pairs. A caller's later dictionary or buffer mutation cannot alter the stored artifact. The direct record constructor also rejects unsupported mutable settings containers.

The audio digest does not include the text or settings. Those remain explicit fields with their own versioned manifest identity. If you need a single manifest digest, serialize a declared canonical representation and retain that representation, rather than assuming concatenated strings are unambiguous.

Python's random documentation explains separate generator instances and deterministic pseudorandom behavior. Record the actual trial order as well as the seed and runtime version. A seed is a reproduction aid, not a substitute for the presented assignment map.

A long and short clip can share an audio digest in this synthetic harness because the bytes are test data. A real corpus must attach the actual generated bytes to the correct text. A digest collision in the manifest is not a quality result, and identical audio under different text should trigger an investigation.

Do not overwrite a clip after listening begins. A corrected recording becomes a new revision, and the report records which trials used which version. Preserve the earlier artifact when it explains a failed run or a corrected collection defect.

Inspect the streaming path before listening

A voice can appear inconsistent because the playback path changed the signal. Before asking participants to judge identity, verify format, channel mapping, sample continuity, trimming, and output level policy. Otherwise the study can attribute an adapter defect to synthesis.

The boundary inspection guide provides exact PCM reassembly fixtures. Use them before adding fades or smoothing. A lost onset can change how a short acknowledgment sounds while leaving a longer phrase understandable from context.

Compare source and assembled captures under a fixed format. If resampling or another transform is intentional, retain the original and transformed artifacts and the processor settings. Byte equality is appropriate only for a path that promises exact preservation.

The Web Audio specification describes rendering and source scheduling behavior. Browser and device conditions still need empirical verification. A saved file that sounds stable offline does not establish that the live player scheduled every chunk correctly.

A public Readest issue discusses long pauses between TTS sentences and player lifecycle. It illustrates why an apparent cadence issue can involve the player. Treat its reported environment as historical evidence, not a current diagnosis for another application.

Record whether the actual selected voice changed. A fallback can be triggered by unsupported settings, capacity, or another provider response. Requested settings and observed metadata belong in separate fields, and absent provider metadata should remain unavailable rather than inferred from the sound.

The session configuration guide gives a concurrency test for another common cause. A long request can begin under one setting and read a changed shared setting later. That failure needs ownership repair before a listening study can isolate voice behavior.

Check the codec and container path too. RFC 7845 documents Ogg encapsulation for Opus, including timing-related handling. Do not splice arbitrary encoded fragments as though they were independent raw PCM blocks. Decode through the documented boundary before evaluating waveform properties.

A gain processor can make a tail seem like another speaker by changing level or noise behavior. Keep it fixed for an identity comparison, or make it the explicit treatment. Store the raw signal as well as any normalized listening copy.

A route change can alter the sound without changing the voice. Headphones, a phone speaker, and a Bluetooth device have different output paths. Record the route and repeat only the conditions the decision actually needs.

For live diagnostics, WebRTC statistics can provide transport and playout observations where supported. Retain receiver identity and original cumulative counters before deriving deltas. Those counters can explain delivery conditions; they do not score perceived identity or intelligibility.

If a capture is incomplete, mark it unusable or missing under a predeclared rule. Do not silently replace the long tail with a fresh generation while retaining the original trial identity. That destroys the relationship between the observation and the artifact.

Repeated generation deserves its own condition when the public interface is nondeterministic. A single favorable recording can conceal run-to-run variation. Preserve each generated artifact separately and state whether the comparison concerns one frozen artifact or a distribution of repeated generations under the same requested settings.

Do not use the same filename to hide those repeats. Include a run identity in the private manifest and retain the returned metadata that the provider exposes. If the provider offers a seed, record its documented scope without assuming it fixes every downstream codec, scheduling, or device behavior.

A useful diagnostic matrix separates text length from streaming segmentation. Keep a short text and a long text under one fixed segmentation rule, then run a separate segmentation comparison if that rule is suspect. Changing both at once prevents the observation from identifying which condition caused the apparent voice change.

The audio chunk sizing guide helps describe output delivery granularity, while the transport boundary guide identifies which protocol owns media framing. Neither grants permission to treat every received byte fragment as an independently decodable speech unit.

For a controlled file study, decode all conditions through the same documented path and store the listening copy's identity. For a live-path study, preserve the actual transport and player behavior. These answer different questions, so the report should not combine the cleanest file from one path with the latency observation from another.

Run a listening protocol that can fail

A listening protocol should permit every candidate to perform poorly. Use neutral labels, randomized presentation order, explicit uncertainty, and questions tied to the dimensions you defined. Avoid exposing provider names or labels such as "improved" during the blind task.

Begin with a short familiarization step using authorized reference clips. Explain what identity means in the study and show the permitted expressive range. Do not coach participants to prefer a particular accent, rate, or level unless that is the explicit product requirement being tested.

For identity, compare against the reference voice and record uncertainty. For content, ask a task question or collect a transcription under the language-specific rubric. For cadence, ask about timing and grouping without treating every pause as an error.

Level needs its own policy. If clips are normalized, record the method and retain originals. If level is a dimension under test, do not normalize it away before participants hear it. The raw-versus-listening distinction belongs in the manifest.

ITU-T P.808 is a public reference for crowdsourced subjective speech-quality evaluation. Remote listening adds environmental and participant-control concerns. A small internal study should report its actual method rather than claiming a standardized score without the required procedure.

Use whole-reply and region-focused tasks separately. Whole replies reveal whether a longer explanation remains coherent. Opening and tail excerpts help locate a perceived change, but their crop boundaries can alter the listening context and must be documented.

The confirmation listening method gives a useful content-first example. Ask what value was heard before asking which sample sounded better. This can expose an intelligibility failure that a preference question would hide.

Balance presentation order and avoid teaching the same answer repeatedly. A participant who hears the same value in several variants can recover it from memory. Use an assignment plan that preserves comparable difficulty without letting earlier trials answer later ones.

Recruit qualified listeners for each language. Record eligibility and the relevant language familiarity without publishing participant identities. If the team lacks qualified reviewers for a condition, mark it untested instead of allowing a confident guess to become a release claim.

Keep the number of ratings and the number of distinct participants separate. Repeated judgments from one listener do not create the same evidence as independent participants. The analysis should preserve the grouping needed to account for that structure.

A listening session can include catch or attention items, but define them before collecting results. Do not remove participants solely because they disliked the candidate. Exclusions should follow the recorded protocol and remain visible in the report.

For a diagnostic run, capture short notes around disagreements. A listener saying the tail sounded quieter supplies a hypothesis about level; it does not automatically establish a new speaker. Follow up with a controlled comparison rather than recoding the response to match an expected cause.

Analyze changes without hiding dimensions

Report each dimension under its own denominator. Identity uncertainty, content error, cadence complaints, and missing captures are different outcomes. A single average can conceal a tail-specific failure or treat an unavailable observation as a successful zero.

A comparison should preserve the matched family relationship. If one family contains difficult identifiers, compare variants within that family before combining them with easy acknowledgments. Keep reply length and reviewed position in the analysis rather than averaging them away.

The outcome metrics guide helps keep task completion distinct from media quality. A consistent voice can still speak the wrong value, while a less preferred voice can deliver a correct and usable answer.

In this criteria table, ✓ means the evidence directly supports the stated question under its scope. ✗ means it cannot establish that property alone. The entries compare evidence types, not products.

Preserve raw ratings and response categories in restricted records. Summary statistics should be recomputable from the same item and participant grouping. If the report changes its scoring rule, retain the earlier version and explain why.

Do not claim statistical significance from an unplanned handful of preferences. A small diagnostic can identify a failure worth fixing without estimating population-level superiority. State whether the result is a reproduction, an exploratory observation, or a planned comparative study.

When a long reply fails and a short one passes, inspect the earliest observed change. It may align with a synthesis unit, a rate change, a fallback, a codec transition, a queue gap, or content difficulty. The trace and corpus should let you compare those hypotheses.

The observability guide supplies the broader event vocabulary. Preserve generation and configuration revision so a listener's tail complaint can be matched to the actual request rather than a similarly named clip from another run.

Keep unmeasured conditions visible in the decision table. A successful English headphone comparison does not establish a multilingual telephone result. The deployment boundary follows the evidence you collected, not the marketing breadth of the voice preset.

Keep the corpus useful across releases

A corpus is useful when a future engineer can rerun the same decision. Keep text, settings, audio, trial assignment, scoring rubric, environment, and result identities together in a versioned record. A folder of clips with descriptive names is not enough.

A new provider model, voice version, decoder, output processor, or segmentation policy can invalidate earlier evidence. Decide which corpus families each change affects and rerun those conditions. Do not repeat an entire study merely because a comment changed, but do not reuse old listening results after a material audio change.

The regression testing guide helps choose narrow release checks. Preserve the failing family and meaningful negative cases, then broaden when the change affects shared behavior. Keep deterministic manifest validation cheap enough to run whenever artifacts are updated.

A negative manifest fixture should reject duplicate identities, changed bytes under an old digest, incomplete PCM frames, unsupported channel counts, and mutable value aliases. These failures establish that the record can protect the evidence you intend to review.

A negative listening fixture has another purpose. It can verify that the task and scoring method detect a known content error or a deliberately mismatched reference. Use authorized synthetic artifacts and label the purpose in the private study record, avoiding deceptive public claims about a provider.

Monitor the live application for symptoms that justify rerunning the corpus. Repetition requests, unexpected fallback selections, or tail-specific complaints can point to a condition that the current set misses. They are investigation leads, not automatic quality scores.

Keep the audio backpressure policy aligned with long-reply tests. A system that preserves every frame by allowing a growing queue can still fail conversational timing and interruption. Long-form consistency must include the actual delivery policy.

For 60db Agent, request reproducible observable behavior through public interfaces. Avoid inferring proprietary algorithms or architecture from the shape of one failure. The corpus can document what happened and what remained controlled without describing how a private engine works.

A release report should distinguish artifact validation, integrated-path checks, completed listening observations, and pending work. If only the synthetic harness ran, say so. That honest boundary lets a later study add evidence without rewriting a fabricated result.

A release comparison also needs a stable policy for failed generations. Missing audio, incomplete tails, unsupported settings, and canceled output are distinct outcomes. Record them before listening selection so the corpus does not quietly contain only successful samples from a candidate that failed often during collection.

If the corpus includes only retained successes, describe that selection explicitly. It can support a bounded question about those artifacts, but it cannot establish the reliability of the generation process. A reliability comparison needs the attempted-request denominator and the conditions under which failures occurred.

Keep file integrity and semantic integrity separate. A complete digest-matching artifact can repeat a phrase, omit a number, or say the wrong name. Use the task rubric and qualified review to identify those errors rather than assuming a successful file check means the text was spoken faithfully.

A critical-field family should preserve its canonical value independently from the generated text. If a normalizer changes a date or number, the manifest needs both the original approved value and the rendered form. The self-correction guide addresses why revised meaning must remain visible through later speech.

When a defect occurs only after an interruption, add an interrupted condition rather than editing the ordinary long clip to imitate it. The interruption recovery method distinguishes continuing, summarizing, and restarting. Each produces different content and can create a different listening comparison.

A corpus owner should periodically remove redundant items only after preserving the failure they protected. Deletion is useful when several clips test the same invariant, but the remaining set must still include the difficult field, reply position, language, and route that justified the original regression.

Document licensing and permitted use with the artifact store. A voice reference approved for internal evaluation may not be approved for public distribution or model training. The public article can share synthetic code and the manifest format while leaving restricted recordings in their authorized location.

At handoff, give the reviewer the exact manifest revision, the rendered text, the audio identities, and the questions asked of listeners. Avoid an informal playlist whose order and filenames can change. The smallest useful corpus is one another engineer can inspect without guessing which recording supports which claim.

A corpus handoff should also state which files are original captures and which are listening derivatives. Preserve a transformation record between them, including trimming, level adjustment, decoding, or resampling. Otherwise a reviewer can reproduce the source artifact's digest while hearing a different file in the actual trial.

The review record should name the dimension each figure explains. The manifest flow establishes traceability, the rubric separates judgments, and the position grid defines coverage. None depicts a measured improvement. Keeping those explanatory jobs explicit helps future editors update the article without turning conceptual artwork into unsupported result-shaped evidence.

Before the next release, choose the smallest affected family and run its deterministic checks again. If the change touches the audio itself, repeat the relevant listening condition. Store a new result record rather than editing the previous release's observations in place.

---

Frequently asked questions

What does voice consistency mean in this guide?

It means preserving specified observable properties across controlled reply conditions. Identity, level, cadence, and intelligibility are evaluated separately. A single preference score cannot establish all of them, and a requested voice name alone does not prove the resulting audio stayed consistent.

Why compare short and long replies separately?

They expose different conditions and can fail differently. A brief acknowledgment can lack context, while a longer reply can reveal tail or segmentation problems. Use matched text families and reviewed positions so content difficulty does not masquerade as a pure length effect.

Can a file digest prove the voice stayed the same?

It proves artifact identity under the selected digest algorithm. It does not establish spoken content, perceived speaker identity, naturalness, or intelligibility. Keep listening and task-response evidence separate from deterministic file checks in both the manifest and the report.

Why include opening and tail review regions?

A long reply can begin acceptably and change later. Region-focused review helps locate that behavior, while whole-reply listening preserves conversational context. Record crop boundaries because an excerpt can sound different when removed from the phrase that introduced it.

Should I normalize every clip before listening?

Only under the study's declared policy. Normalization can control a level confound, but it can also remove the level difference you intended to study. Retain original audio, normalized listening copies, and the exact transform settings so the comparison remains inspectable.

Does a more natural voice imply better intelligibility?

No general equivalence is established here. A listener can prefer a voice while misunderstanding a critical field. Ask a content task separately, preserve uncertainty, and analyze naturalness or preference under its own rubric rather than merging everything into one quality number.

Can a playback bug look like a voice change?

Yes. Missing onsets, gain changes, rate mistakes, route changes, or gaps can alter the perceived sound. Inspect source, assembled, and output boundaries before assigning the cause to synthesis. Keep the actual configuration and fallback evidence with the affected clip.

Is the published corpus script a speech generator?

No. It validates a narrow raw PCM manifest and reproducible trial ordering using synthetic bytes. It performs no decoding, playback, synthesis, or perceptual scoring. A real collection process must attach actual authorized artifacts to the corresponding text and settings.

Why store settings as immutable string pairs?

The reference owns the supported value shape and prevents ordinary caller dictionary mutation from changing an existing record. A richer application can use another explicitly validated immutable shape. Avoid retaining mutable provider options that can silently alter the meaning of a saved trial.

What happens if the provider does not expose a model version?

Record it as unavailable and preserve the request date, public model selection, returned metadata, and artifact identity. Do not invent a version. That uncertainty limits exact regeneration but does not prevent an honest comparison of the artifacts you actually captured.

Can I compare different text in each condition?

You can, but the design must account for content difficulty and meaning. A simple length comparison should use matched families and balanced assignments. Otherwise pronunciation, field type, or sentence structure can explain the difference attributed to streaming length.

Does the seed fully reproduce the listening study?

No. Retain the actual trial order, participant assignment, runtime version, presented artifact identities, and scoring rules. A seed reproduces a bounded scheduling operation under the recorded implementation; it cannot reconstruct missing participant responses or undocumented exclusions.

How should I handle uncertain identity judgments?

Keep them separate from same and different judgments under the predeclared rubric. Uncertainty is an observation, not missing data to replace with a favorable result. Inspect whether particular content, devices, or positions produce it before selecting a repair.

Can automatic speaker embeddings replace listening?

They can supply another bounded signal if their method and calibration fit the task. This article does not establish a threshold or a biometric guarantee. Validate any automated metric against qualified listening and task outcomes rather than treating an uncalibrated score as the final decision.

Should a fallback voice share the baseline identity label?

Record the actual selected voice and treat a fallback as another condition. A successful request does not establish perceptual equivalence. If the product permits a different fallback identity, state that policy and test whether the caller can still follow the interaction.

What if only one language has qualified reviewers?

Report evidence for that language and mark the others untested. A familiar-sounding sample is insufficient for pronunciation or intelligibility claims in an unfamiliar language. Expand the listening team or narrow the release claim instead of manufacturing certainty.

Do WebRTC statistics measure perceived voice quality?

They provide supported transport and playout observations under their documented definitions. They do not directly score identity, cadence, or comprehension. Retain original counters and receiver identity for reproduction, then use them to investigate delivery conditions around a listening failure.

How do I avoid exposing private speech in the corpus?

Use synthetic or licensed text and authorized voices. Remove direct and indirect identifiers from public artifacts, and keep restricted study records separate. A reproducible method can publish formats, code, and synthetic fixtures without publishing customer recordings or participant identities.

What should trigger a corpus rerun?

Material changes to voice selection, model, segmentation, rate processing, decoder, output path, or a reproduced quality symptom. Select the affected families and channels first. A narrow deterministic metadata change can use lighter verification than an audio-processing change.

What is the first useful consistency test?

Freeze one matched short-and-long family, capture the actual configuration and audio identities, and ask separate identity and content questions. Inspect the long tail explicitly. Preserve the result and limits so the next change can be compared against the same evidence.

Ship evidence for the affected reply

A useful voice-consistency decision identifies the property, the controlled condition, and the observation. Keep identity, level, cadence, and intelligibility separate. Preserve the exact artifacts rather than relying on a voice label or an average score.

Run the manifest harness, verify the streaming path, and freeze a small matched corpus. Then conduct the listening task under a recorded protocol that permits uncertainty and failure. Report only the dimensions and channels that the completed work supports.

Take one short acknowledgment and one longer reply from the same synthetic family into your next regression run. Review both the opening and tail, retain the configuration snapshot and audio digests, and attach each conclusion to the artifact that produced it. That gives future changes a stable comparison point.