---
title: How to Evaluate Uncensored & Abliterated AI Models
description: Build repeatable model comparisons: refusal behavior, coding quality, tool use, reasoning settings, token costs and retained failures, with public GLM evidence.
canonical: https://routerhub.si/guides/evaluating-uncensored-models
---

Model guide
# How to Evaluate Uncensored & Abliterated AI Models
Uncensored model evaluation measures refusal rate, task success, formatting, latency and cost using repeatable prompts and settings.
By RouterHub · Updated 3 November 2026

## What should a reduced-refusal model evaluation measure?
Record refusal rate, task success, formatting, latency and actual cost as separate measurements. Use tasks representative of the application you plan to build.

## A repeatable comparison protocol

- Fix the task. Use the same inputs, output format, reasoning setting and output budget for each exact model ID.
- Separate refusal from failure. Record an explicit refusal separately from a timeout, empty response, invalid JSON, wrong answer or truncated generation.
- Keep every attempt. Repeat your tests and retain unsuccessful outputs, not just the best answer.
- Measure useful work. Compare task acceptance, instruction following, tool-call validity and cost per successful task, not token price alone.
- Re-test changes. A hosted alias or model update can change behavior even when your client configuration stays the same.

## Evaluate coding outputs in isolation
Generated code is untrusted. Run acceptance tests in a disposable sandbox without API keys, host credentials or network access. Pin the test suite, record a candidate hash, and keep the tests read-only. Use operating-system isolation in addition to a virtual environment.

## Report cost per accepted result
For ordinary uncached text usage, estimate cost as `input tokens × input price / 1,000,000 + output tokens × output price / 1,000,000`. Reported cache reads and writes use their own published rates. The final Usage record, not time spent waiting or a headline token limit, is the billing reference.
Example: at $0.20 per million input tokens and $0.60 per million output tokens, 10,000 uncached input tokens and 2,000 output tokens cost an estimated $0.0032. This is an illustrative calculation, not a quoted price for a particular model.
For model selection, divide the total cost of all measured attempts by the number of acceptable results. A lower token rate is not automatically a lower cost per completed task.

## What RouterHub’s published evaluation shows
RouterHub’s [public coding evaluations on GitHub](https://github.com/routerhubSI/routerhub-model-evals) include prompts, raw outputs, settings, candidate hashes and sandbox-scored tests. The GLM results record both the default-effort attempts and the low-effort cohort, so you can compare runs with their original settings.
The published Flask upload task reported three low-effort attempts for each of three GLM entries passing all 17 tests. A separate default-effort abliterated cohort returned no code within its budget. Compare these cohorts separately, using their recorded settings.

- [Read results and run settings Read the source, examples and project notes on GitHub.](https://github.com/routerhubSI/routerhub-model-evals/blob/main/RESULTS.md)
- [Read the candidate sandbox procedure Read the source, examples and project notes on GitHub.](https://github.com/routerhubSI/routerhub-model-evals/blob/main/docs/SANDBOX.md)

## Choose a model after testing

- [Uncensored API models Browse uncensored models and compare prices.](https://routerhub.si/models/uncensored)
- [Abliterated API models Browse abliterated models and compare prices.](https://routerhub.si/models/abliterated)
- [Variant definitions and comparisons Compare the techniques and model variants.](https://routerhub.si/guides/uncensored-vs-abliterated)