The eval engine for high-stakes AI agents.
Score production voice and chat against versioned evals. Catch what is breaking, prove what happened, and keep call data inside your environment.
- Zero data egress
- Signed images
- SBOM per release
- SSO / SCIM
Unverified agents are an invisible compliance liability.
- Blindfolded eye, representing unmeasured behavior.
Agent behavior is assumed, not measured
Prompt tests and pre-launch evals show what an agent might do. They do not prove what happened on the live call that promised the wrong refund or skipped a required disclosure.
- Shield marking a data boundary.
PII-heavy calls can't leave your boundary
Call audio, transcripts, and credentials carry regulated data. Sending them to a third-party scoring API creates the egress risk reviewers reject, so the calls that need verification most go ungraded.
- Fractured grid of disconnected tools.
Metrics, rules, tests, and labels are disconnected
Latency dashboards, policy checks, LLM-judge evals, and human QA rarely share one grading coordinate. Teams cannot answer whether a call passed with a single, defensible record.
- Pencil with a strike, representing an editable record.
Editable audit trails are not evidence
When a QA correction overwrites the original score, you lose the history auditors ask for. Mutable results cannot show who graded what, when, and against which rubric.
One eval engine between your agents and your systems of record.
Marker scores each conversation, routes review, and writes immutable evidence inside your environment.
1. Sources: your live agents
Production voice and chat agents push transcripts through API upload or OpenTelemetry. Deep instrumentation is optional.
Supported ingest paths
- API transcript upload
- OpenTelemetry spans
- Audio transcription
2. Eval engine
Each transcript is scored against versioned evals: built-in metrics, deterministic rules, and LLM judges.
- Built-in metric Quality and safety signals
- Deterministic rule Repeatable pass or fail checks
- LLM judge Criteria-aware grading
3. Evidence: audit-grade
Every answer is immutable and revision-pinned. Corrections supersede rather than overwrite.
Deploy where your compliance requires.
Hosted, customer cloud, connected on-prem, or fully air-gapped.
- T0
Hosted
SaaS or dedicated, hosted on Marker cloud. The fastest path to a verification run.
Egress: Managed by Marker
- T1
Customer cloud / BYOC
Runs in your own VPC. Your data plane, your account, your region.
Egress: Customer-controlled
- T2
Connected on-prem
In your datacenter, reaching only customer-approved outbound endpoints.
Egress: Allow-listed
- T3
Air-gapped
No outbound internet, no phone-home. A signed offline bundle, verified in place.
Egress: None - enforced in code
Connects to how your agents and your data already work.
Push transcripts, connect identity, and deploy in your environment.
API key upload
Direct transcript upload
OpenTelemetry
OTLP span ingest
Audio transcription
Word timings and diarization
SAML 2.0
Enterprise SSO
OIDC
Bring your own IdP
Scoped API keys
Per org or single agent
Postgres
Primary state and durable jobs
Helm / Kubernetes
One install, on your own domain
From conversation to eval to audit trail.
Machine scoring and human review use the same questions, so results do not drift across tools.
Step 1 of 3
Capture
Production agents push live voice and chat through API upload or OpenTelemetry. Each transcript keeps a clear lifecycle, then becomes write-once once it is ready to score.
- pending
- processing
- ready
- failed
call_8f21.wav
word timings · diarization · transcribed on your GPUs
chat_5c09 (OTel)
every real call kept - nothing sampled out
Step 2 of 3
Verify
Versioned evals score each transcript with built-in metrics, deterministic rules, or LLM judges. Human review answers the same questions instead of living in a separate QA tool.
Evaluator
LLM judge
metric · rule · judge · human
Output shape
category
boolean · number · category
Assertion
pass / warn / fail
turns observation into a verdict
Step 3 of 3
Audit
Every answer is append-only and pinned to the eval revision that produced it. Corrections supersede instead of overwriting, so auditors can see who graded what and when.
- rev a1b2
Mark · disclosure given = false
machine answer · superseded
- rev a1b2
Label · disclosure given = true
human correction · current truth
Same grading coordinate · append-only · exportable for auditors.
See exactly what your agents are doing on your own infrastructure.
We’ll walk through a verification run on representative calls, in the deployment tier your procurement requires.