fix(formatter): show full error message for test failures (#690)#729
fix(formatter): show full error message for test failures (#690)#729ousamabenyounes wants to merge 1 commit intortk-ai:developfrom
Conversation
format_compact() was truncating error messages to 2 lines via .take(2). Playwright (and vitest, pytest, cargo test) errors contain the critical expected/received diff and call log starting at line 3+, so agents saw only the error type with no actionable debug information. Fix: iterate all lines of error_message for each failure instead of taking the first 2. Summary line stays compact; only the per-failure detail is preserved in full. Affects all test runners using TestResult (playwright, vitest, cargo, pytest, dotnet) — any test failure is now fully visible. Closes rtk-ai#690 Signed-off-by: Ousama Ben Younes <benyounes.ousama@gmail.com>
|
Hello Ousama ! Thanks for contributing again :) The truncation fix is of course correct, but PR 690 also asked for pass-through mechanism like others commands, One thing missing : playwright_cmd.rs is the only test runner that doesn't call tee_and_hint, which is RTK mechanism for pass-through on failure, used by vitest, pytest, cargo, go, etc. That was asked in #690, could you maybe fix this little as well so we resolve the full issue ? Should be easy to add and straght-forward to merge after |
|
If no news from you, i'll merge what you have done, it solve a part of the issue, but i'll not be able to close the issue if not totally resolved. Thanks again for solving issues |
|
|
Problem
format_compact()truncated all test failure error messages to 2 lines via.take(2). Playwright errors look like:Agents saw only
"Error: expect(locator).toHaveText(expected)"— no diff, no call log, nothing actionable. This caused the 170 tool calls / 3.6M token loop described in #690.Fix
Replace
.take(2).join(" ")with a full line-by-line iteration oferror_message. The summary line (PASS (28) FAIL (2)) stays compact. Only the per-failure detail is now fully preserved.Impact
Affects all test runners using
TestResult: Playwright, Vitest, Cargo test, Pytest, .NET — any test failure is now fully visible in compact mode.Tests
4 new unit tests in
formatter.rs:test_compact_shows_full_error_message— asserts expected/received diff and call log are presenttest_compact_summary_line_is_concise— summary stays on one linetest_compact_all_pass_is_one_line— success output stays compacttest_compact_single_line_error_no_trailing_noise— single-line errors still workCloses #690