A check that can fail

EvalproofCC-BY-4.0practiced By Campaign Hub

An eval a stranger can run: a bead check that exits non-zero when the how_checked field is missing or empty. A file that merely exists is not a pass.

When to use

Use this check when a bead claims to be finished and you need an observation that goes red on a broken copy, including a copy that is only an empty file.

When not to

Do not replace this check with a test that the file exists. Do not soften a red result into a warning that still exits 0.

Author-attested

Original teaching specimen written for the Campaign Hub library. Not a transcript of a private run.

Community-submitted content. Prompts, skills, and agent instructions can contain hidden or malicious instructions (prompt injection). Review it before running it in any agent that has tool access, secrets, or the ability to spend money or send messages.
# A check that can fail This file is an eval: a check a stranger can run that exits non-zero on a broken copy. ## When to use Use this eval when you have a bead written as four labeled lines and you need a command that exits 0 on a good bead and exits non-zero on a broken one. The broken cases include a bead that omits its check, a bead whose check line is empty, and a file that exists and says nothing. The script is the observation. You can run it without the author. ## When not to Do not use "the file is present" as this eval. Presence is a precondition for reading the file. It is not the check. Do not edit the script so that a missing field prints a warning and exits 0. A check that cannot fail will accept the known-bad envelope, and then it is not this eval. ## The specimen The artifact is this check. ```bash #!/bin/bash # check-bead.sh - exit 0 only when a bead names four non-empty fields. # Usage: check-bead.sh FILE # A file that merely exists is not a pass. set -euo pipefail if [ "${1:-}" = "" ]; then echo "usage: check-bead.sh FILE" exit 2 fi file="$1" if [ ! -f "$file" ]; then echo "missing file: $file" exit 2 fi require() { label="$1" message="$2" if ! grep -Eq "^${label}:[[:space:]]*[^[:space:]]" "$file"; then echo "$message" exit 1 fi } require how_checked "fail: bead has no how_checked observation" require deliverable "fail: bead has no deliverable" require may_change "fail: bead has no writable scope" require must_not_touch "fail: bead has no protected scope" echo "pass" ``` Save it as `check-bead.sh`. It requires one argument, the path of the bead file. If the argument is missing, or the path is not a file, it exits 2. If the file exists, it still has to earn a pass. The pass condition is four labels, each at the start of a line, each followed by a non-empty value, in this order of judgment: `how_checked`, then `deliverable`, then `may_change`, then `must_not_touch`. The first missing label is the failure the script prints. `how_checked` is first so that a bead which omits its check fails on that omission even when the other fields are present. A good bead, the one in "The four-field envelope," looks like this: ```text deliverable: One specimen file a stranger can reuse without the author present. how_checked: check-bead.sh on this file exits 0. may_change: The single specimen file this bead names. must_not_touch: The roster, the seed files, and every other specimen. ``` The known-bad pair is in "An envelope that goes red." This script is the eval that must reject it. ## How a stranger checks it Run three observations. All three are required. 1. Create an empty file named `empty.txt`. Run `bash check-bead.sh empty.txt`. It exits 1 and prints `fail: bead has no how_checked observation`. The file exists. The check still fails. That is the point. 2. Save the known-bad bead from "An envelope that goes red" and run the script on it. It exits 1 and prints the same line. 3. Save the good bead and run the script on it. It exits 0 and prints `pass`. A copy of the script that exits 0 on the empty file fails this eval. A copy that checks only `deliverable` and ignores `how_checked` fails this eval, because the known-bad bead has a deliverable and must still go red.

How a stranger checks it

Related

eval check exit-code bead

view raw view JSON