fix: scope scoring bug in prompt_score + add 10 unit tests#129
Open
TerminalGravity wants to merge 2 commits intomainfrom
Open
fix: scope scoring bug in prompt_score + add 10 unit tests#129TerminalGravity wants to merge 2 commits intomainfrom
TerminalGravity wants to merge 2 commits intomainfrom
Conversation
- Fix: long prompts no longer auto-score 25/25 on scope (was rewarding verbosity regardless of whether scope was actually bounded) - Long prompts now get 20/25 with feedback to add explicit bounds - Export scorePrompt for testability - Add 10 tests covering all scoring dimensions, grade boundaries, and edge cases (vague prompts, questions, broad vs bounded scope)
Tests cover: - Specificity scoring (file paths, backtick identifiers, generic mentions) - Scope scoring (bounded vs broad keywords) - Actionability scoring (specific vs vague verbs) - Done condition scoring (outcome words, questions) - Grade assignment (A+ threshold) - Feedback messages (praise for perfect prompts) - Score consistency (total = sum of sub-scores)
TerminalGravity
commented
Mar 6, 2026
Collaborator
Author
TerminalGravity
left a comment
TerminalGravity
left a comment
There was a problem hiding this comment.
Nice catch on the scope scoring — giving 25/25 just for prompt length was wrong. Fix is clean, test coverage is solid. One thought: might be worth adding a test for a prompt that's long and has explicit scope bounds to confirm those still get full marks. Not blocking though.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
scorePromptgave 25/25 scope to any prompt >100 chars, regardless of whether scope was actually bounded. A rambling vague prompt got perfect scope just for being long.only,just,this).scorePromptcovering all 4 scoring dimensions, grade boundaries, edge cases.scorePromptis now exported for direct testing.Tests
All 53 tests pass (10 new + 43 existing).