- Guides
Voice agent outcome metrics: measure task success now
Voice agent outcome metrics with explicit denominators, verified business results, calibrated review, and a practical release scorecard.
- Author
Udit Goenka- Published
- Reading time
- 25 min

TL;DR: Measure verified task completion, incorrect actions, useful handoffs, abandonment, and caller effort using explicit denominators. Keep sentiment and transcript-derived quality separate from business evidence, preserve unknown outcomes, and join operational metrics to the same versioned call cohort before making release decisions.
A pleasant conversation can end with an appointment that never existed. A frustrated caller can still get the correct refund, and a short call can mean either fast resolution or an early disconnect.
Your scorecard must distinguish those cases. Google's SRE workbook recommends service indicators that reflect the user experience; for voice agents, that means connecting a conversation to the result the caller needed.
As of November 9, 2026, this guide proposes an evaluation contract rather than reporting a production improvement. It complements the real-call regression harness with the labels, joins, and denominators that make a release result interpretable.
Contents
- Define the outcome contract
- Choose a compact scorecard
- Build reliable outcome evidence
- Handle unknowns and delayed results
- Calibrate automated reviewers
- Compare cohorts without hiding failures
- Turn metrics into release decisions
- Implement a minimal reporting path
- Frequently asked questions
- Make the next decision auditable
Define the outcome contract
Task success is a verified result within a declared scope. The scope includes the caller's eligible request, the authorized action, the business system that confirms it, and the deadline after which the result is considered settled.
For a scheduling task, success can mean that the requested slot exists in the scheduling system with the confirmed details. For an information task, success can mean the caller received an answer that matches the approved knowledge source and did not require a subsequent correction.
These tasks require different evidence. A calendar record can establish a booking, while an answer-quality label requires a review rubric and access to the source that was valid during the call.
Write the contract before choosing a dashboard tool. Otherwise, a convenient field such as completed or positive tends to become the success label even when it describes a different observation.
A call status might only say that the session ended normally. A positive sentiment label describes modeled affect, and neither establishes that a requested business change occurred.
Define eligibility without deleting hard calls
Eligibility describes what the agent is authorized and configured to handle. It should be determined from documented scope and observable request facts, with a reason recorded for exclusions.
An appointment agent might be eligible to create a booking but ineligible to provide medical advice. That does not mean every caller who asks a difficult scheduling question can be removed from the success denominator.
Separate out-of-scope requests from in-scope failures. An unknown requested service, a tool outage, or a missing slot is part of the operating reality and needs an explicit outcome rather than a retrospective eligibility exception.
Track routing quality too. If many callers reach the wrong agent, the task-completion rate among eligible conversations can look healthy while the overall service is wasting caller time.
Report the intake funnel alongside eligible-task success. The funnel should preserve attempts that never connected, connected sessions with no useful exchange, conversations with an identifiable task, and those whose eligibility remains unknown.
Use the transport comparison to understand why a failed media setup may need a different diagnostic category from a misunderstood request, while both still affect end-to-end service delivery.
Decide what the unit represents
A call, a task, a caller, and a business case are different units. A single call can contain several tasks, and one unresolved case can generate several calls.
Choose a primary unit and name it in every rate. For an early deployment, call-level reporting is often easier to audit, but a multi-task assistant needs task-level evidence to avoid labeling a partly completed conversation as fully successful.
A repeat caller introduces another question. If a second call repairs the first call's mistake, count the first failure and the later recovery, then measure eventual case resolution separately.
Do not merge several attempts into a single success without retaining the attempts. The cost-per-success model needs those retries to explain the effort and expense required for the final result.

Calls, tasks, and eventual case resolution answer different questions and need named denominators.
Choose a compact scorecard
A useful scorecard has enough metrics to explain the service without asking every team to maintain dozens of overlapping labels. Start with business completion and harmful mistakes, then add the observations needed to diagnose why they change.
The table proposes a starting contract. ✓ means the signal can establish the stated fact with the named evidence; ✗ means that signal alone cannot establish verified task completion.
| Signal | Suggested denominator | Primary evidence | Establishes task completion alone? |
|---|---|---|---|
| Verified task completion | Eligible settled tasks | Business record or reviewed answer | ✓ within declared task scope |
| Incorrect action rate | Authorized-action opportunities or executed actions, named explicitly | Action ledger plus policy review | ✗ measures harmful behavior |
| Useful handoff rate | Handoff attempts | Destination accepted plus context evidence | ✗ unless handoff is the task |
| Abandonment before resolution | Connected unresolved calls | Disconnect and task state | ✗ measures an unresolved exit |
| Caller repair effort | Eligible reviewed conversations | Repeated corrections and clarifications | ✗ measures effort |
| Response delay distribution | Observed answerable turns | Speech and playback timing | ✗ measures experience |
| Positive sentiment | Labeled conversations or turns | Model-derived label | ✗ measures modeled affect |
| Normal call completion | Started calls | Session lifecycle status | ✗ measures session ending |
Keep the scorecard definitions close to the reporting query. A prose definition in a slide deck drifts when an analyst later filters out nulls or changes the business-time window.
OpenTelemetry semantic conventions provide shared names for operational observations. Your business outcome fields still need a task-specific schema because a general telemetry standard cannot determine what counts as a correctly fulfilled appointment.
Measure incorrect actions independently
A high task-completion rate can coexist with a low-frequency action error that matters more than several failed conversations. Track wrong writes separately and preserve severity instead of averaging them into a general quality score.
For example, an unauthorized booking and a harmless extra clarification have different consequences. Your policy owner should define severity and the action boundary before the evaluator labels calls.
Choose the denominator carefully. Incorrect actions divided by executed actions answers a different question from incorrect actions divided by all eligible calls, especially when a release changes how often the agent attempts a write.
Report both when they answer real operational questions, with clear names. Do not let the denominator switch quietly during a release comparison because that can reverse the apparent direction of change.
The workflow enforcement guide explains how to prevent writes before they occur. Outcome measurement verifies the behavior of that boundary and reveals failures that prompt-level quality scores can miss.
Define useful handoff from the caller's view
A transfer request is an attempt. A useful handoff requires evidence that the destination could take over and received the permitted context needed to avoid starting again.
Define which observations are available on your route. A system may report a transfer destination or a connected call leg without proving that a human read the summary.
Use separate states for transfer requested, destination accepted, context delivered, and handoff usefulness reviewed. If the last observation is unavailable, report that limitation rather than assigning every connected transfer a success label.
The warm handoff design makes those states explicit. It also covers fallback when the destination is busy or unreachable, which belongs in the same outcome record.
Count caller effort with a rubric
Repeated questions are useful only when you distinguish repetition from necessary confirmation. Reading a date back once can protect the caller, while asking for the same date after it was confirmed creates avoidable work.
Define a repair event as a caller correction or repeated provision of already accepted information caused by an agent failure. A reviewer should be able to point to the relevant exchange and explain why it qualifies.
Track repairs per eligible conversation and the share of conversations requiring any repair. Long calls create more opportunities for repairs, so inspect task type and complexity before comparing raw counts.
This signal is diagnostic. It helps identify memory loss, recognition problems, and tool-state confusion even when a later correction allows the task to finish successfully.
Build reliable outcome evidence
Business systems should confirm business actions. Use the scheduling, order, ticketing, or payment system as the source of truth for the corresponding result, while keeping the conversation's claim as a separate observation.
A tool response saying that a request was accepted can be weaker than a committed record. Document the exact state that your business API returns, and reconcile asynchronous completion before assigning final success.
The distinction also protects against fabricated confirmations. If the agent says a booking is complete while the ledger shows no successful write, the discrepancy should become a failure category with an actionable owner.
Do not reconstruct the entire business database inside the analytics platform. Store a restricted reference and the minimum verified fields required for evaluation, respecting retention and access rules.
Diagram source (Mermaid)
flowchart LR
C[Call observations] -->|Link task reference| J[Outcome join]
B[Business evidence] -->|Confirm final state| J
P[Versioned task policy] -->|Define success| J
J -->|Enough evidence| S[Settled outcome]
J -->|Evidence missing| U[Unknown queue]
S -->|Aggregate cohort| R[Release scorecard]
U -->|Reconcile later| J
R -->|Select review sample| H[Human audit]
H -->|Correct labels| J
This is a proposed reporting architecture. It does not imply that a particular managed platform exposes every event or that a transcript alone supplies the missing business evidence.
Join by durable action identity
Each authorized action needs an identifier that survives retries and delayed delivery. Link it to the logical task and call, then record the business system's final reference once available.
Avoid joining by phone number and approximate timestamp. Shared telephone numbers, retries, and clock differences can attach a valid business result to the wrong conversation.
The speculative tool-calling guide separates read-only work from committed writes. That separation also makes outcome evidence cleaner because a prefetched slot list cannot be mistaken for a completed booking.
If your application receives webhooks, verify them using the provider's current documented signature contract. Preserve event identity for deduplication, and treat an authenticated notification as evidence about its stated event rather than proof of every downstream business result.
RFC 9110 defines HTTP method semantics including idempotency. Application-level deduplication for business actions still needs its own contract when the operation or retry behavior does not provide the guarantee you need.
Retain policy and configuration versions
An outcome is meaningful only relative to the task policy in effect. Store the prompt or configuration version reference, tool schema version, evaluation rubric version, and relevant deployment cohort.
A release may intentionally change eligibility or confirmation rules. Without version references, a later evaluator can label old calls using requirements the deployed agent never had.
Keep the original label and the revised label when correcting a rubric error. The reporting view can use the latest approved classification while the audit trail explains why historical rates changed.
The complete call trace helps connect these versions to transport and model events. The outcome store should reference that trace instead of duplicating every low-level event and sensitive payload.
Use recordings only for a defined purpose
Audio can reveal interruption and pronunciation failures that a transcript hides. It can also contain more sensitive information than the narrow business fields needed for aggregate reporting.
Choose recording policy, access, and retention before using recordings for evaluation. Where recording is unavailable or inappropriate, identify which quality questions remain unobservable and use supported alternatives.
OpenTelemetry's sensitive-data guidance recommends minimizing and protecting sensitive telemetry. Apply that principle before export, because downstream redaction cannot undo content already sent to an unapproved destination.
An aggregate scorecard should rarely need raw names, addresses, payment details, or full transcripts. Keep restricted review material separate and publish only approved summaries that cannot be traced back to a small or recognizable group.
Handle unknowns and delayed results
Unknown is a first-class outcome. It means the evidence does not yet support success or failure under the declared contract, and it needs its own count and owner.
A pending asynchronous write, a missing event, or an unavailable recording can create an unknown. These causes deserve different remediation even if they share the same high-level label.
Do not turn null into false automatically. Also do not drop nulls and call the remaining set representative; either choice can distort results when missingness is related to failures.
Publish the eligible total, settled count, unknown count, and the settlement window. Readers can then distinguish a strong success rate from a score calculated over a small, unusually observable subset.

Work a synthetic denominator example
Suppose a synthetic evaluation contains 100 eligible tasks. Business evidence confirms 72 successes and 18 failures, while 10 tasks remain unknown at the reporting deadline.
The settled-task success rate is 72 divided by 90, or 80%. The confirmed-success share of all eligible tasks is 72 divided by 100, or 72%, and the unknown share is 10%.
All three numbers are useful. None is a production result, and the 80% value should never appear without its settled denominator and unknown share.
A worst-case interpretation treats the unknown tasks as unsuccessful, producing 72% for that hypothetical cohort. A best-case interpretation treats them as successful, producing 82%, which exposes how missing evidence limits the conclusion.
These bounds are bookkeeping, not a statistical confidence interval. A confidence interval addresses sampling uncertainty under a stated model, while these bounds address outcomes that have not been observed.
Set a settlement window
Some results become final after the voice call ends. A booking may confirm immediately, while a ticket resolution or a later cancellation may require a longer observation period.
Define the window by task and explain why it fits the business process. A daily report can contain a provisional cohort and a settled cohort rather than pretending that every call ending today already has a final outcome.
Track time to settlement as a reporting-health metric. A growing backlog of unknowns can indicate delayed webhooks, failed joins, or a business dependency problem even when the voice runtime appears healthy.
The shutdown and post-call guide helps keep this reporting continuation separate from live media. A call can release its microphone and telephone leg while outcome reconciliation continues under an explicit retention policy.

Keep unresolved cases visible and version the report when delayed evidence changes the classification.
Distinguish caller intent from abandonment
A hangup does not always mean abandonment. The caller may have received the answer, declined an optional action, or intentionally ended a task after learning that no suitable appointment exists.
Define abandonment as an unresolved caller exit under the task contract, and preserve unknown intent when evidence is insufficient. Avoid labeling every short call a failure or every normal disconnect a success.
Timing helps diagnose the exit without determining it alone. A disconnect during a long silent tool wait deserves a different review path from a disconnect after an explicit refusal or a confirmed result.
Use the latency measurement guide to capture the wait boundary. The resulting correlation can identify a useful investigation cohort without proving that delay caused every abandonment.
Calibrate automated reviewers
A model reviewer can apply a rubric at scale, but it needs validation against human-labeled examples and deterministic business evidence. Fluent explanations from the reviewer do not establish that its labels are correct.
Begin with a small, deliberately varied reference set. Include successful calls, known failures, interrupted confirmations, missing evidence, language switches, and ambiguous intent rather than sampling only clean conversations.
Have reviewers label independently before discussing disagreements. Preserve the evidence reference and the rule that led to the final label, then revise the rubric when different readers apply it inconsistently.
NIST's AI Risk Management Framework provides a context-dependent risk-management structure rather than a universal quality threshold. Your acceptance thresholds should follow the consequences of mislabeling in the task you operate.
Measure reviewer errors in both directions
A false success hides a failed task. A false failure creates unnecessary review work and can cause a team to reject a useful release.
Report the reviewer's confusion matrix on the reference set, preserving unknown as a permitted output. Accuracy alone can look favorable when one label dominates, so inspect precision and recall for the failure classes you most need to detect.
Severity matters. An evaluator that detects ordinary repetition but misses unauthorized actions is unsuitable as a safety gate even if its overall agreement is high.
Set escalation rules by evidence and consequence. Deterministic policy violations can fail a release directly, while ambiguous answer quality can enter a human-review queue with the necessary context.
Keep sentiment separate from task success
Sentiment can help prioritize review or understand caller experience, but a sentiment model infers affect from available signals. It cannot establish that the requested business action completed correctly.
A caller may sound cheerful before discovering a wrong appointment later. Another caller may remain annoyed about a policy even after the agent provides accurate information and completes the permitted task.
The 60db Agent documentation describes sentiment and transcript-derived quality alongside call status and cost fields. Treat those labels as modeled observations, preserve null or unknown values, and join them to your own business evidence before reporting verified success.
Do not relabel a positive sentiment field as customer satisfaction. A satisfaction survey, a modeled affect label, and a verified outcome answer different questions and require different collection methods.

Prevent evaluator leakage
Give the reviewer the evidence required for the assigned question. If it is judging whether the agent asked for confirmation, it may need the relevant conversation segment and policy, but not an unrelated customer profile or payment history.
Keep candidate release identity hidden when practical. A reviewer that knows which prompt is newer can learn an unwanted preference for the candidate instead of applying the rubric consistently.
Separate generation and evaluation configurations. Changes to the evaluator should be versioned and rechecked against the reference set, because changing both sides at once makes an apparent product improvement difficult to interpret.
The local versus managed boundary guide applies to reviewer providers too. Sending post-call content to an evaluator creates another processing path even when the live voice service already meets your approved data requirements.
Compare cohorts without hiding failures
An overall success rate is a weighted mixture of tasks. If the task mix changes, the rate can move even when each task type behaves the same.
Slice by a bounded set of decision-relevant dimensions: task type, language, transport, release version, and a small number of failure categories. Avoid adding every available customer field because it increases privacy risk and creates noisy, tiny groups.
The multilingual evaluation guide explains why language count is weaker evidence than per-language task completion. Keep recognition accuracy, entity correctness, and business success separate so a transcription improvement is not mistaken for an outcome gain.
Similarly, the SIP troubleshooting guide helps distinguish carrier-media failures from conversation-policy failures. A transport cohort provides a diagnostic route rather than an excuse to exclude inconvenient calls.
Control release comparisons
Where appropriate, assign comparable eligible traffic to a baseline and candidate using a documented allocation method. Preserve the assignment for a task so retries do not switch releases unpredictably.
If random allocation is unavailable, report the differences between cohorts. Time of day, campaign source, language, task complexity, and downstream system health can all change the workload.
Pair controlled test fixtures with production observation. Fixtures isolate known behaviors, while production cohorts reveal the mix and failure patterns that scripted cases do not cover.
The interruption-safe agent guide gives a concrete release example: a candidate may reduce stale speech after interruption while increasing premature cutoffs. Both observations need to appear before declaring an improvement.
Retain counts beside percentages
A percentage without its count hides how much evidence supports it. Publish the numerator and denominator for every rate, and show the settlement status of the cohort.
Small groups need careful interpretation and privacy protection. Suppress or combine groups when a result would reveal an individual, while retaining the private evidence needed for authorized investigation.
Do not average language-level or workspace-level percentages unless that averaging rule answers a defined question. A pooled task-weighted rate and an equally weighted cohort average can disagree because they represent different populations.
The same warning applies to latency quantiles. Prometheus's histogram guidance explains aggregation differences between histograms and summaries; choose instrumentation that supports the aggregation you intend rather than averaging already computed p95 values.
Connect delay to outcome cautiously
Delay is an experience metric with operational value. Join response-delay cohorts to abandonment and repair rates to identify where review is likely to pay off.
Do not call the association causal without an appropriate design. Complex requests can both take longer and fail more often, and a slow tool can affect several metrics at once.
Measure caller speech end to audible response separately from TTS first audio bytes. 60db's June 2026 TTS report reports about 197 milliseconds median first audio on a warm streaming connection across a 330-prompt Hindi/English corpus; it does not measure end-to-end task success or caller playback.
That boundary distinction prevents a component benchmark from becoming an unsupported business claim. Use your measured conversation and outcome evidence for the product decision.
Turn metrics into release decisions
A scorecard should name an action owner. If a metric changes and nobody knows whether to investigate, pause rollout, or correct data, the dashboard is an expensive archive.
Choose release criteria before examining the candidate results. Include verified task performance, severe incorrect actions, unknown-evidence rate, and the operational conditions required to trust the comparison.
Avoid one weighted quality score for everything. A small gain in politeness should not compensate mathematically for an unauthorized write, and a missing business join should not disappear inside an average.
Use explicit gates for unacceptable behavior and separate trends for improvement work. That makes a release decision explainable to engineering, operations, and the task owner without asking them to reverse-engineer a scoring formula.

A favorable quality average cannot compensate for an unauthorized action or another blocking invariant.
Tie alerts to actionable failure
Immediate paging belongs to conditions that require immediate response. A broken tool authorization boundary or widespread inability to establish media may qualify, while a slow-moving change in modeled sentiment may belong in a scheduled review.
Google's SLO alerting chapter describes alerts based on error-budget consumption across time windows. Apply that operating idea to well-defined voice service indicators rather than paging on every isolated model label.
A low-volume service may not produce stable short-window rates. Use counts, synthetic checks, and direct severe-event alerts where appropriate, documenting their limits instead of copying thresholds from a high-volume service.
Give each alert a runbook that identifies the evidence to inspect and the permitted response. The operator should know whether to disable writes, redirect calls, restore a prior configuration, or investigate reporting before touching the live voice path.
Use failure categories that lead somewhere
A useful failure category points to a system boundary or a policy owner. Examples include media setup, recognition of a critical entity, premature endpointing, unauthorized action, tool timeout, and transfer destination unavailable.
Keep the category set bounded and allow a reviewer to mark insufficient evidence. A large taxonomy with overlapping labels makes manual review slower and reduces agreement without necessarily improving the next engineering decision.
The turn detection guide can help resolve premature cutoffs, while the workflow guide addresses confirmation and authorization failures. Link those runbooks from the private scorecard rather than forcing the aggregate report to carry every diagnostic detail.
Review category usage after real incidents. Merge categories that do not lead to different actions, and split a category only when the distinction changes ownership or remediation.
Implement a minimal reporting path
Start with a durable call-to-task join and a versioned classification record. You can add a warehouse later if the current database and reporting job already answer the question reliably.
A minimal record needs a synthetic-safe call reference, task type, eligibility state, release version, settlement state, outcome label, evidence type, and evaluation version. Add a restricted business reference only where it is needed to reconcile the result.
Keep metrics labels bounded. Prometheus's naming guidance warns against unbounded label cardinality; individual call identifiers belong in protected event records or traces rather than time-series dimensions.
The reporting job should fail visibly when required evidence is malformed. Quietly converting malformed records to unsuccessful calls hides a data-quality problem, while quietly dropping them biases the scorecard.
Validate denominator arithmetic
The following vendor-neutral Python example checks only the accounting invariant. Its inputs are synthetic, and it makes no network requests or assumptions about a managed service API.
from fractions import Fraction
eligible = 100
success = 72
failure = 18
unknown = 10
assert min(eligible, success, failure, unknown) >= 0
assert success + failure + unknown == eligible
settled = success + failure
settled_rate = Fraction(success, settled) if settled else None
confirmed_share = Fraction(success, eligible) if eligible else None
unknown_share = Fraction(unknown, eligible) if eligible else None
assert settled_rate == Fraction(4, 5)
assert confirmed_share == Fraction(18, 25)
assert unknown_share == Fraction(1, 10)
Save the snippet as a local file and run it with Python 3. The assertions confirm that the categories partition the eligible tasks and that the illustrated rates match the stated fractions.
Production validation needs more than this arithmetic. Check duplicate task references, incompatible label versions, missing release assignment, and results attached after the settlement window, then expose those issues in a reporting-health view.
Audit one record from source to report
Select a synthetic task with a known final business state and follow it through the reporting path. Verify that its call reference, action reference, release assignment, eligibility decision, and outcome version survive each transformation without changing meaning.
Then replay the same event and confirm that the numerator does not increase. Deliver the final business evidence after the initial reporting deadline and check whether the task moves from provisional unknown to settled success under the declared window.
Exercise the negative path too. Supply a normal call completion with no successful business write and confirm that the report cannot call it a verified success merely because the session status looks favorable.
Finally, remove required evidence and confirm that the reporting-health signal changes. These checks test the joins and state transitions responsible for the scorecard, which are distinct from the arithmetic assertions in the small example.
Keep expected results in the existing regression fixture rather than creating another independent evaluation service. A report is easier to maintain when the same task contract drives its test, its classification rule, and the explanation a reviewer reads after a failed release.
Ship one scorecard before adding automation
Begin with one task and a manually reviewed sample joined to business evidence. Produce a short report with counts, unknowns, severe errors, and a few redacted failure categories.
Use that report to identify the next engineering action. If the report cannot distinguish a tool failure from an evaluator mistake, fix the evidence path before adding more charts or a more elaborate reviewer.
For 60db Agent, map documented call analytics and outcomes to your own task contract. Do not assume sentiment confirms success, that every call route exposes identical analytics, or that a managed dashboard replaces your business system's final state.
The implementation goal is a trustworthy decision loop. A modest scorecard with known limits is useful when each field has an owner, evidence, and a clear consequence for the next release.
Frequently asked questions
What is the main voice agent success metric?
Use verified task completion within an explicit eligibility and settlement contract. The evidence should come from the relevant business system or a calibrated answer-review process, with unknown results reported separately.
Does a completed call mean a successful task?
No. A completed call status describes the session lifecycle, while task success requires evidence that the caller's eligible request was fulfilled correctly; a normal hangup can follow a failed or unfinished task.
Can sentiment measure customer satisfaction?
Sentiment is a modeled interpretation of affect, and satisfaction requires its own collection method. Keep sentiment, survey responses, and verified task outcomes separate so a positive label cannot hide an incorrect business result.
Should unknown outcomes count as failures?
Report them explicitly before applying any conservative interpretation. Show settled-task success, confirmed-success share of all eligible tasks, and unknown share so readers can see how missing evidence affects the conclusion.
How do I define an eligible task?
Use the agent's declared authority and supported task scope, with observable reasons for exclusions. Preserve difficult in-scope requests and operational failures instead of removing them after seeing an unfavorable result.
Should I measure calls or tasks?
Choose the unit that matches your service and name it in every rate. Multi-task calls often need task-level reporting, while call-level reporting remains useful for connection, abandonment, duration, and overall service experience.
What evidence proves a booking happened?
Use a committed record in the scheduling system with the confirmed details and a durable action reference. A conversational confirmation or accepted tool request is weaker evidence when the business operation can still fail asynchronously.
How should I count repeated calls?
Retain each attempt and link attempts to the same case where you can do so safely. Report eventual resolution separately, because combining retries into one success hides the caller effort and cost required to finish.
What makes a handoff successful?
Define the required observations, such as destination acceptance and permitted context delivery. If usefulness is not directly observable, preserve that limitation and avoid treating a transfer request as proof that a human resolved the issue.
Can I use a model to score every call?
Yes, with a validated rubric, appropriate data permissions, and monitoring of reviewer errors. Deterministic business evidence should still own business facts, and ambiguous or severe cases need a defined human escalation path.
How do I calibrate an automated reviewer?
Compare it with independently reviewed reference examples covering success, failure, unknowns, and important edge cases. Inspect false successes and false failures by severity, then version the rubric and recheck it when the evaluator changes.
Is transcript accuracy a business outcome?
Transcript accuracy is a component metric. A small error in a date or identifier can matter more than several harmless word errors, so evaluate critical entities and final task completion alongside recognition quality.
Should I average p95 latency across cohorts?
No, averaging quantiles does not produce the pooled quantile. Aggregate appropriate individual observations or histogram data and preserve the cohort counts and instrumentation method used for the reported distribution.
What should a release gate include?
Use criteria agreed before the comparison, covering task performance, severe incorrect actions, unknown-evidence rate, and operational reliability. Keep unacceptable behaviors as explicit gates rather than letting a weighted quality average compensate for them.
How do I avoid metric cardinality problems?
Use bounded dimensions such as task type, transport, language, and release cohort where appropriate. Keep unique call identifiers and restricted business references in protected records or traces rather than time-series labels.
Do short calls mean the agent is efficient?
Only after you know what happened. Short duration can reflect fast resolution, a refused action, connection trouble, or early abandonment, so join duration with outcome state and caller intent before interpreting it.
How long should outcome settlement take?
Choose a window that matches the business process and publish it with the metric. Separate provisional from settled cohorts and monitor delayed evidence so a reporting backlog does not silently distort recent results.
Can I compare languages with one overall rate?
An overall rate is useful alongside per-language counts and results. Different task mixes, recognition behavior, and audio routes can hide failures in a pooled value, so retain the cohorts that affect a real deployment decision.
Which 60db Agent fields establish business success?
Documented call status, sentiment, quality, transfer, and cost fields provide useful observations, but the business result still needs the relevant system's evidence. Preserve null labels and verify which analytics are available for the call route and workspace you use.
What is the smallest useful first implementation?
Join one task type to its business evidence, review a varied sample, and publish counts for success, failure, and unknown. Add diagnostic metrics only when they help explain a failure or choose the next engineering action.
Make the next decision auditable
A voice agent scorecard should let another engineer reproduce the conclusion. They need the task contract, cohort definition, evidence sources, settlement window, label versions, and the counts behind each percentage.
Build that record before optimizing presentation. It will expose whether the service failed, the evidence arrived late, or the evaluator applied the wrong rule, and those cases demand different fixes.
Start with a documented 60db Agent workspace evaluation or your existing voice stack, then join the call observations to one real business task. Review the unknowns, inspect the severe failures, and decide the next change from evidence you can defend.
