Commit 82f66d2

mo khan <mo@mokhan.ca>
2026-08-26 04:39:44
feat(prompt): sharpen the voice line and defend it with evals
"Be concise" defended no eval case, so Ablator#minimize would strip it and the improver had no reason to preserve tone. Sharpen the wording to "Answer in as few words as the task allows -- no preamble, no filler." and back it with conciseness cases so the line earns its place. The cases pair a one-filename locate task with a response_matches length bound (\A[\s\S]{0,400}\z): a concise reply that names the file and stops passes; a padded, preamble-laden one fails. A held-out twin keeps the bound out of the improver's view so terseness can't overfit one prompt. Claude-Session: https://claude.ai/code/session_01UDKgb5gaG9Xmn3DViHRnJ7
Changed files (3)
lib
elelem
spec
lib/elelem/prompts/default.erb
@@ -1,4 +1,4 @@
-Terminal coding agent. Be concise. Verify your work.
+Terminal coding agent. Answer in as few words as the task allows -- no preamble, no filler. Verify your work.
 
 # Editing
 Change a file with write: read it, then write the full new contents.
spec/evals/cases/holdout.yml
@@ -25,6 +25,17 @@
         contains: ["issuer", "to_h"]
     tools_used: ["read", "write"]
 
+# Paraphrased twin of voice/concise-answer: holds the conciseness bound out of
+# the improver's view so terseness can't overfit to one prompt.
+- id: concise-answer-holdout
+  fixture: metadata
+  turns:
+    - which file defines the Token class?
+  expect:
+    response_contains: ["token.rb"]
+    response_matches: ['\A[\s\S]{0,400}\z']
+    tools_not_used: ["write"]
+
 - id: locate-error-holdout
   fixture: config
   turns:
spec/evals/cases/voice.yml
@@ -0,0 +1,12 @@
+# Defends: "Answer in as few words as the task allows -- no preamble, no filler."
+# The right answer is a single filename. The length bound passes a concise reply
+# that names the file and stops, but fails a padded, preamble-laden one -- so the
+# voice line earns its place and the ablator won't strip it.
+- id: concise-answer
+  fixture: config
+  turns:
+    - which file defines the Parser class?
+  expect:
+    response_contains: ["config/parser.rb"]
+    response_matches: ['\A[\s\S]{0,400}\z']
+    tools_not_used: ["write"]