Skip to content

fix(formatter): show full error message for test failures (#690)#729

Open
ousamabenyounes wants to merge 1 commit intortk-ai:developfrom
ousamabenyounes:fix/test-failure-error-message-truncation
Open

fix(formatter): show full error message for test failures (#690)#729
ousamabenyounes wants to merge 1 commit intortk-ai:developfrom
ousamabenyounes:fix/test-failure-error-message-truncation

Conversation

@ousamabenyounes
Copy link
Contributor

Problem

format_compact() truncated all test failure error messages to 2 lines via .take(2). Playwright errors look like:

Error: expect(locator).toHaveText(expected)   ← line 1 (visible)
                                               ← line 2 (visible, empty)
Expected: 'Submit'                             ← cut here
Received: 'Loading'                            ← cut here
Call log:                                      ← cut here
  - waiting for getByRole('button', ...)       ← cut here

Agents saw only "Error: expect(locator).toHaveText(expected)" — no diff, no call log, nothing actionable. This caused the 170 tool calls / 3.6M token loop described in #690.

Fix

Replace .take(2).join(" ") with a full line-by-line iteration of error_message. The summary line (PASS (28) FAIL (2)) stays compact. Only the per-failure detail is now fully preserved.

// Before
let error_preview: String = failure.error_message.lines().take(2).collect::<Vec<_>>().join(" ");
lines.push(format!("   {}", error_preview));

// After
for line in failure.error_message.lines() {
    lines.push(format!("   {}", line));
}

Impact

Affects all test runners using TestResult: Playwright, Vitest, Cargo test, Pytest, .NET — any test failure is now fully visible in compact mode.

Tests

4 new unit tests in formatter.rs:

  • test_compact_shows_full_error_message — asserts expected/received diff and call log are present
  • test_compact_summary_line_is_concise — summary stays on one line
  • test_compact_all_pass_is_one_line — success output stays compact
  • test_compact_single_line_error_no_trailing_noise — single-line errors still work
cargo test parser::formatter::tests
✓ 4 passed
cargo test
✓ 983 passed, 0 failed

Closes #690

format_compact() was truncating error messages to 2 lines via .take(2).
Playwright (and vitest, pytest, cargo test) errors contain the critical
expected/received diff and call log starting at line 3+, so agents saw
only the error type with no actionable debug information.

Fix: iterate all lines of error_message for each failure instead of
taking the first 2. Summary line stays compact; only the per-failure
detail is preserved in full.

Affects all test runners using TestResult (playwright, vitest, cargo,
pytest, dotnet) — any test failure is now fully visible.

Closes rtk-ai#690

Signed-off-by: Ousama Ben Younes <benyounes.ousama@gmail.com>
@aeppling
Copy link
Contributor

Hello Ousama !

Thanks for contributing again :)

The truncation fix is of course correct, but PR 690 also asked for pass-through mechanism like others commands,

One thing missing : playwright_cmd.rs is the only test runner that doesn't call tee_and_hint, which is RTK mechanism for pass-through on failure, used by vitest, pytest, cargo, go, etc.

That was asked in #690, could you maybe fix this little as well so we resolve the full issue ?

Should be easy to add and straght-forward to merge after

@aeppling aeppling self-assigned this Mar 19, 2026
@aeppling
Copy link
Contributor

If no news from you, i'll merge what you have done, it solve a part of the issue, but i'll not be able to close the issue if not totally resolved.

Thanks again for solving issues

@CLAassistant
Copy link

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants