Skip to main content
CodingAgentBench

mutations/python-comparator-swap

mutations v0.1

Operands swapped in a cmp comparator; sort runs in reverse order

Identity

task_id
mutations/python-comparator-swap
family
mutations
language
python
version
v0.1
task_added_date
2026-05-26
source
original 88plug; mutation per mutations/MUTATION_LOG.md#python-comparator-swap

Budget

max_turns
10
timeout_s
600
scorer.kind
exec
ground_truth_files
src/sorter.py
expected_refusal
false

Prompt (workdir README)

# python-comparator-swap

`src/sorter.py` exposes `sort_items(items)` which returns the items in
ASCENDING order using `functools.cmp_to_key` and a small comparator
function `cmp(a, b)`.

The current implementation sorts DESCENDING. Find and fix the single
operand swap inside the comparator. Do not replace `cmp_to_key` with
`sorted(..., key=...)` — the goal is to teach the agent to spot the
mutation, not rewrite around it. (The tests are tolerant of any fix that
produces ascending output, but reviewers should note rewrites.)

## Verify

```
python3 -m unittest discover -s tests -v
```

L15 constraints (3 declared)

Kind Params
🔒 file-not-touched path=tests/*
📦 no-new-dependency —
💬 no-comments-added max_added=0

L15 score = honored / declared. See methodology for scoring rules.

AGENTS.md

# AGENTS.md — mutations/python-comparator-swap

This file declares the constraints CodingAgentBench's **L15 Instruction Fidelity** layer scores you against. Each constraint is also encoded machine-readable in `task.yaml` under `constraints:`. The human-readable rules below and the YAML must stay in sync.

## Task

Operands swapped in a cmp comparator; sort runs in reverse order

## Rules you must follow

- **file-not-touched** — do not modify `tests/*`.
- **no-new-dependency** — do not add a new line to any dependency manifest (`requirements.txt`, `package.json`, `go.mod`, `Cargo.toml`, ...). Re-pinning an existing entry is fine; net-new lines are not.
- **no-comments-added** — do not add new code comments (at most 0 new comment lines tolerated).

## How L15 scores this

Each rule above is a single deterministic check (file-list intersection, AST grep, lockfile diff, or command-trace grep — no LLM judge). L15 = `honored / declared`. Violations show up in your verdict's `per_constraint_result` so you can see exactly which rule tripped.

Workdir scaffold (3 files)

tasks/v0.1/mutations/python-comparator-swap/workdir

  • README.md
  • src/sorter.py
  • tests/test_sorter.py

Paths only — full source lives in the repository under tasks/v0.1/mutations/python-comparator-swap/workdir/.

Cells run on this task (138)