Skip to content

tests: introduce doctest infra + migrate llcommon/llmath/llcorehttp subsets (ctest green) [relates to #4445]#4852

Open
mayconbekkers wants to merge 12 commits intosecondlife:developfrom
mayconbekkers:feat/doctest-poc-clean
Open

tests: introduce doctest infra + migrate llcommon/llmath/llcorehttp subsets (ctest green) [relates to #4445]#4852
mayconbekkers wants to merge 12 commits intosecondlife:developfrom
mayconbekkers:feat/doctest-poc-clean

Conversation

@mayconbekkers
Copy link

Relates to #4445.

This PR adds a header-only doctest setup and migrates a first wave of unit tests while keeping the legacy TUT harness for suites not yet ported. The goal is to land a minimal, working baseline that the team can iterate on module by module.
This affects only unit-test targets behind LL_TESTS=ON; viewer runtime and packages are unchanged.

What’s included

  • New doctest targets: llcommon_doctest, llmath_doctest, llcorehttp_doctest
  • Shared helpers (LL_CHECK_* for floats, ranges, buffers, wide strings) + small Windows wide-string shim
  • Deterministic llcorehttp fakes (zero-latency transport, monotonic clock, queued per-handle responses, redirects/retries/cancels) to keep tests network/IO-free
  • Suites green locally:
    • llcommon_doctest: 100 cases
    • llmath_doctest: 93 cases
    • llcorehttp_doctest: 34 cases
  • Quickstart doc: docs/testing/doctest_quickstart.md

How to run locally

# Enable tests
autobuild configure -c RelWithDebInfoOS -- -DLL_TESTS=ON

# Build (VS generator)
"C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake\bin\cmake.exe" --build build-vc170-64 --config RelWithDebInfo --target llcommon_doctest llmath_doctest llcorehttp_doctest -- /p:BuildProjectReferences=false

# Run
"C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake\bin\ctest.exe" -C RelWithDebInfo -R "(llcommon_doctest|llmath_doctest|llcorehttp_doctest)" -V --test-dir build-vc170-64

Scope / non-goals

Does not change viewer runtime or packaging.
TUT remains for non-migrated suites; removal will happen after follow-ups when coverage is sufficient.

Proposed follow-ups

Continue migrating remaining suites per module (llcommon -> llmath -> llcorehttp -> newview/test).
Once coverage is high enough, remove Tut.cmake glue and clean up LLAddBuildTest.cmake usage.

@github-actions
Copy link

github-actions bot commented Oct 17, 2025

All contributors have signed the CLA ✍️ ✅
Posted by the CLA Assistant Lite bot.

@mayconbekkers
Copy link
Author

I have read the CLA Document and I hereby sign the CLA

@Ansariel
Copy link
Contributor

What are the changes to .gitignore and .gitattributes for? They seem in no relation to doctest and counterproductive,

@mayconbekkers mayconbekkers force-pushed the feat/doctest-poc-clean branch 3 times, most recently from 5fb6af0 to dddc098 Compare October 20, 2025 23:34
@mayconbekkers
Copy link
Author

@Ansariel Thanks for the note! I’ve dropped the unrelated .gitignore/.gitattributes edits so this PR stays focused on doctest.
Also added the standard $LicenseInfo headers to the new test files and a third-party MIT notice for indra/extern/doctest/doctest.h.
pre-commit run -a passes locally; once workflows are approved for this fork, CI should reflect that.
Happy to adjust anything else if the maintainers prefer.

@akleshchev akleshchev requested review from Geenz and Copilot November 17, 2025 13:54
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR introduces a doctest-based unit testing framework to complement the existing TUT infrastructure, migrating initial test suites for llcommon, llmath, and llcorehttp. The changes keep the current TUT framework intact while establishing new doctest targets for gradual migration.

Key Changes

  • Added header-only doctest framework with shared test helpers (LL_CHECK_* macros for floating-point, range, and buffer comparisons)
  • Migrated 227 test cases across three modules: llcommon_doctest (100 cases), llmath_doctest (93 cases), llcorehttp_doctest (34 cases)
  • Created deterministic HTTP test fakes (zero-latency transport, monotonic clock, queued responses) for network-free testing

Reviewed Changes

Copilot reviewed 66 out of 68 changed files in this pull request and generated no comments.

Show a summary per file
File Description
tools/testing/gen_tut_to_doctest.py Python script to auto-generate doctest stubs from TUT sources
indra/test/{doctest_main.cpp,ll_doctest_helpers.h,tut_compat_doctest.h} Shared doctest infrastructure and TUT compatibility layer
indra/llmath/tests_doctest/*.cpp Migrated vector/matrix/quaternion math tests to doctest
indra/llcorehttp/tests_doctest/{http_fakes.h,http_fakes.cpp} Deterministic HTTP testing infrastructure
indra/llcorehttp/tests_doctest/*_test_doctest.cpp Migrated HTTP component tests with fake transport
indra/llcommon/tests_doctest/*_test_doctest.cpp Auto-generated TODO stubs for future llcommon migration
indra/viewer_components/login/tests/lllogin_doctest.cpp New doctest-based login workflow tests
indra/*/CMakeLists.txt Build configuration for new doctest targets


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@mayconbekkers
Copy link
Author

Hey @akleshchev

Thanks a lot for taking a look and for requesting the review here.

Just to clarify: this PR is only meant to be a first step for a doctest-based test setup, not the full migration. I wanted to get early feedback before going any further.

I also saw on Discord that Signal is stepping away, and I really appreciate that @Geenz is keeping the open source program moving forward. If this approach to the doctest infra, the CMake wiring and the way I’m migrating these first tests looks reasonable from your side, I’m happy to continue in follow-up PRs and adjust things to match what you’d like to see in the test suite.

No rush at all, I know there’s a lot happening right now. Just wanted to make the intent clear and let you know I’m around to iterate on this.

@akleshchev akleshchev linked an issue Nov 24, 2025 that may be closed by this pull request
@akleshchev
Copy link
Contributor

this PR is only meant to be a first step for a doctest-based test setup, not the full migration

That's obvious from the todo) We do need a replacement, but with signal away I don't know who is responsible for the commisions, will ask around.

P.S. You probably can make an AI do the bulk of the work here

@kylelinden kylelinden requested review from SntaxLinden and removed request for kylelinden November 25, 2025 20:41
Copy link

@SntaxLinden SntaxLinden left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Approved.

@mayconbekkers
Copy link
Author

Quick follow-up on the doctest infra:

  • I shared a common lltest_harness between the TUT runner and doctest_main (APR, logging with LLReplayLog, LLTrace).
  • I stabilized the doctest targets on MSVC by wiring:
    • llcommon_doctest (tuple/workqueue subset),
    • llmath_doctest (v2/v3/v4/llquaternion),
    • llcorehttp_doctest (HTTP fakes + header helpers),
    • login_doctest (login workflow with XML-RPC fakes).
  • I added a small docs/testing/doctest_quickstart.md to document how to enable and run the doctest targets locally on Windows.

Locally (RelWithDebInfoOS, LL_TESTS=ON) I ran:

  • llcommon_doctest – 7 test cases / 24 assertions
  • llmath_doctest – 102 test cases / 253 assertions
  • llcorehttp_doctest – 22 test cases / 105 assertions
  • login_doctest – 3 test cases / 5 assertions

All green (Status: SUCCESS).

@akleshchev
Copy link
Contributor

akleshchev commented Mar 23, 2026

All green (Status: SUCCESS).

I assume it's ready for a merge?

@Geenz Please check this. Specifically if it's fine to include that header in viewer repo directly intead of something like a package.

P.S. No idea why precommit got stuck, trying to fix it.

daatsuka added a commit to daatsuka/viewer that referenced this pull request Mar 23, 2026
PR secondlife#4852 (feat/doctest-poc-clean) introduces a comprehensive doctest
migration with a fundamentally different architecture: TUT is kept
running alongside doctest via a compatibility layer, with the header
vendored at indra/extern/doctest/ and new test targets in tests_doctest/
subdirectories.

Our branch attempted a full TUT replacement in the core test harness,
which conflicts on 4 files: Doctest.cmake, LLAddBuildTest.cmake,
test/CMakeLists.txt, and test.cpp. The approaches are incompatible.

Revert all conflicting changes to develop baseline, keeping only the
platform license file additions (licenses-linux.txt, licenses-mac.txt,
licenses-win32.txt) which complement secondlife#4852's indra/extern/doctest/LICENSE.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@mayconbekkers
Copy link
Author

Finished the llmath follow-up migration in small commits. llmath_doctest is green locally: 278 test cases, 1178 assertions, 0 failures. This batch only rewraps additional llmath suites; no new infra changes.

@akleshchev akleshchev requested a review from Copilot March 24, 2026 16:14
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 88 out of 90 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (8)

indra/llmath/tests_doctest/alignment_test_doctest.cpp:1

  • std::cout is used but this file does not include <iostream>. Add #include <iostream> (or remove the debug prints) to avoid relying on indirect includes and potential compile failures on some toolchains.
    indra/llmath/tests_doctest/alignment_test_doctest.cpp:1
  • Correct the spelling in the assertion message from 'LLAligment' to 'LLAlignment' to keep messages consistent and searchable.
    tools/testing/gen_tut_to_doctest.py:1
  • extract_block() returns a slice even when it never finds a matching closing brace (e.g., malformed input or regex mismatch). In that case end_index remains start_index, producing an empty/incorrect body and an invalid end_index + 1. Consider explicitly detecting depth != 0 at loop exit and failing fast (e.g., raise ValueError/SystemExit) so the generator doesn't emit misleading stubs.
    tools/testing/gen_tut_to_doctest.py:1
  • The generator hard-codes the doctest suite name to llcommon, which makes it easy to accidentally generate incorrect suites for other modules. Make the suite name configurable (CLI arg like --suite) or infer it from src_path/the destination directory so generated files consistently land in the correct suite.
    indra/test/lltest_harness.h:1
  • lltest_init_logging() is documented as installing the fatal handler for the TUT runner, but the implementation in lltest_harness.cpp doesn't set LLError::setFatalFunction(...). Either update this comment to match current behavior, or move the fatal-handler installation into lltest_init_logging() so callers don't need to remember to do it separately.
    indra/test/lltest_harness.cpp:1
  • The master thread recorder is heap-allocated and never released, which makes the harness own a permanent leak (especially visible for long-lived test runs or when embedding tests). Prefer a static std::unique_ptr<LLTrace::ThreadRecorder> (or a function-local static value) and register cleanup via std::atexit() if the underlying API requires explicit teardown.
    indra/test/lltest_harness.cpp:1
  • The master thread recorder is heap-allocated and never released, which makes the harness own a permanent leak (especially visible for long-lived test runs or when embedding tests). Prefer a static std::unique_ptr<LLTrace::ThreadRecorder> (or a function-local static value) and register cleanup via std::atexit() if the underlying API requires explicit teardown.
    indra/llcorehttp/tests_doctest/http_fakes.cpp:1
  • FakeBufferArray::assign() appends into the existing BufferArray without clearing it first. If the same FakeBufferArray instance is reused (multiple assign() calls), the resulting body will contain accumulated data rather than the last assigned payload. If the intended semantics are 'replace', clear/reset the buffer (e.g., drop+recreate, or otherwise truncate) before appending.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.


set(CMAKE_MODULE_PATH "${CMAKE_SOURCE_DIR}/cmake" "${CMAKE_CURRENT_BINARY_DIR}")

enable_testing()
Copy link

Copilot AI Mar 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR description says changes only affect unit-test targets behind LL_TESTS=ON, but enable_testing() is now unconditional at the indra root. If the intent is to keep non-test builds unchanged, gate this behind if(LL_TESTS) (or equivalent) and rely on per-module enable_testing()/add_test() only when tests are enabled.

Copilot uses AI. Check for mistakes.
@@ -0,0 +1,39 @@
# doctest quickstart
Copy link

Copilot AI Mar 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The file appears to start with a UTF-8 BOM (the invisible character before #). BOMs can cause noisy diffs and occasional tooling quirks (some linters/Markdown processors treat it as text). Consider removing the BOM so the first character is #.

Suggested change
# doctest quickstart
# doctest quickstart

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Replace tut with doctest

5 participants