gguf-provider
1- id: commit-50-72
2 fixture: hello
3 setup:
4 - git init -q
5 - git config user.email evals@example.com
6 - git config user.name evals
7 - git add -A
8 - git commit -q -m "initial commit"
9 - printf 'greet("again")\n' >> hello.rb
10 turns:
11 - commit the current changes with a message that follows the 50/72 rule
12 expect:
13 verify: test -z "$(git status --porcelain)" && test "$(git log -1 --format=%s | wc -c)" -le 51
14
15# Defends: query git history and report, without editing.
16- id: last-commit-subject
17 fixture: hello
18 setup:
19 - git init -q
20 - git config user.email evals@example.com
21 - git config user.name evals
22 - git add -A
23 - git commit -q -m "Add greeting script"
24 turns:
25 - what is the subject line of the most recent git commit?
26 expect:
27 response_contains: ["Add greeting script"]