diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index 680756d9..d2584f2a 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -9,6 +9,12 @@ repos: - id: check-merge-conflict # Prevents merge conflict markers - id: check-toml # Validates TOML syntax + - repo: https://github.com/igorshubovych/markdownlint-cli + rev: v0.39.0 + hooks: + - id: markdownlint + args: [--config, .markdownlint-config.yaml] + - repo: local hooks: - id: hatch-lint diff --git a/docs/concepts/index.rst b/docs/concepts/index.rst index fca15908..fa1e3226 100644 --- a/docs/concepts/index.rst +++ b/docs/concepts/index.rst @@ -1,8 +1,34 @@ Concepts ======== +Understanding how fromager works under the hood. + +Key Topics +---------- + +These guides explain the fundamental concepts and design principles behind fromager: + +* **Bootstrap vs Build Modes** - Understand the difference between recursive discovery (bootstrap) and single-package builds (build) +* **Dependency Types** - Learn about build-system, build-backend, build-sdist, and install dependencies + .. toctree:: :maxdepth: 1 bootstrap-vs-build dependencies + +Related Practical Guides +------------------------- + +Apply these concepts with task-oriented guides: + +* :doc:`/using` - Bootstrap and build mode usage +* :doc:`/how-tos/repeatable-builds` - Use build graphs for consistent builds +* :doc:`/how-tos/graph-commands/index` - Analyze dependency graphs +* :doc:`/customization` - Customize the build process + +Reference Material +------------------ + +* :doc:`/reference/glossary` - Definitions of key terms +* :doc:`/reference/files` - Build order and graph file formats diff --git a/docs/conf.py b/docs/conf.py index c1629367..38a0f380 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -58,7 +58,7 @@ # sphinxcontrib.spelling settings # File references a function object. Spell checker complaints about typo in # random object id. -spelling_exclude_patterns = ["config-reference.rst"] +spelling_exclude_patterns = ["reference/config-reference.rst"] # -- Options for HTML output ------------------------------------------------- # https://www.sphinx-doc.org/en/master/usage/configuration.html#options-for-html-output diff --git a/docs/customization.md b/docs/customization.md index a6af38ee..6d6a1245 100644 --- a/docs/customization.md +++ b/docs/customization.md @@ -323,7 +323,7 @@ requires = ["setuptools>=68.0.0", "torch", "triton"] ## Override plugins -Override plugins are documented in [the reference guide](hooks.rst). +Override plugins are documented in [the reference guide](reference/hooks.rst). ## Canonical distribution names @@ -474,3 +474,22 @@ def mycommand( ) -> None: ... ``` + +## See Also + +**Reference Documentation:** + +- [Configuration Reference](reference/config-reference.rst) - Complete settings schema +- [Hooks Reference](reference/hooks.rst) - Override plugin documentation +- [CLI Reference](reference/cli.rst) - Command-line options + +**How-To Guides:** + +- [Override pyproject.toml](how-tos/pyproject-overrides.rst) - Modify build configuration +- [Build from Git](how-tos/build-from-git-repo.rst) - Use custom source repositories +- [Multiple Versions](how-tos/multiple-versions.rst) - Handle version conflicts + +**Concepts:** + +- [Bootstrap vs Build](concepts/bootstrap-vs-build.rst) - Understand build modes +- [Dependencies](concepts/dependencies.rst) - Dependency types and resolution diff --git a/docs/getting-started.rst b/docs/getting-started.rst index 481f3ab0..47d579ca 100644 --- a/docs/getting-started.rst +++ b/docs/getting-started.rst @@ -118,3 +118,24 @@ The output below is redacted for brevity. Missing sections are replaced with ``. As each dependency is built, fromager will show output from the build process and progress information. At the end of the build, fromager shows the list of packages that were built and how long each step took. + +Next Steps +---------- + +Now that you understand the basic workflow and debugging process: + +**Improve Your Workflow:** + +* :doc:`how-tos/containers` - Run builds in containers for better isolation +* :doc:`how-tos/bootstrap-constraints` - Pin versions for reproducible builds +* :doc:`how-tos/repeatable-builds` - Use previous build graphs for consistency + +**Customize Builds:** + +* :doc:`customization` - Add patches, override settings, configure variants +* :doc:`how-tos/pyproject-overrides` - Override pyproject.toml settings + +**Learn How It Works:** + +* :doc:`concepts/index` - Understand bootstrap mode, build isolation, and more +* :doc:`reference/glossary` - Definitions of key terms diff --git a/docs/how-tos/index.rst b/docs/how-tos/index.rst index 1baea4e0..fd0465b0 100644 --- a/docs/how-tos/index.rst +++ b/docs/how-tos/index.rst @@ -1,9 +1,67 @@ How-tos ======= +Task-oriented guides for common workflows and customization scenarios. + +Popular Guides +-------------- + +Quick links to frequently used guides: + +* :doc:`containers` - Running fromager in containers (recommended approach) +* :doc:`bootstrap-constraints` - Pin versions for reproducible builds +* :doc:`graph-commands/using-graph-why` - Debug unexpected dependencies +* :doc:`repeatable-builds` - Ensure build reproducibility + +Getting Started +--------------- + +Essential guides for initial setup and first builds. + +.. toctree:: + :maxdepth: 1 + + containers + bootstrap-constraints + +Building Packages +----------------- + +Guides for building packages from various sources and configurations. + +.. toctree:: + :maxdepth: 1 + + build-from-git-repo + repeatable-builds + parallel + build-web-server + +Build Configuration +------------------- + +Customize builds with overrides, variants, and version handling. + .. toctree:: :maxdepth: 1 - :glob: - * - */index + pyproject-overrides + multiple-versions + pre-release-versions + +Analyzing Builds +---------------- + +Understand and debug dependency graphs and build issues. + +.. toctree:: + :maxdepth: 1 + + graph-commands/index + +See Also +-------- + +* :doc:`/customization` - Comprehensive guide to customization options +* :doc:`/reference/config-reference` - Configuration reference +* :doc:`/reference/hooks` - Override plugins and hooks diff --git a/docs/index.rst b/docs/index.rst index f0bd4e5e..2adacd20 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -23,21 +23,58 @@ behavior that works for most PEP-517 compatible packages, but support overriding all of the actions for special cases, without encoding those special cases directly into fromager. +Getting Started +--------------- + +Quick introduction and detailed walkthrough for new users. + .. toctree:: :maxdepth: 2 quickstart.rst - using.md getting-started.rst + +Guides +------ + +Task-oriented guides for common workflows and customization. + +.. toctree:: + :maxdepth: 2 + + how-tos/index.rst customization.md + using.md + +Concepts & Explanation +---------------------- + +Understanding how fromager works and technical deep-dives. + +.. toctree:: + :maxdepth: 2 + concepts/index.rst - how-tos/index.rst - hooks.rst - files.md http-retry.md - config-reference.rst - cli.rst - glossary.rst + +Reference +--------- + +Technical reference for CLI, configuration, files, and terminology. + +.. toctree:: + :maxdepth: 2 + + reference/index.rst + +Development +----------- + +Contributing to fromager. + +.. toctree:: + :maxdepth: 2 + develop.md What's with the name? diff --git a/docs/quickstart.rst b/docs/quickstart.rst index a4ea3228..1aef1f43 100644 --- a/docs/quickstart.rst +++ b/docs/quickstart.rst @@ -59,7 +59,7 @@ figured out the build and runtime dependencies, built each package in the correct order, and created wheels in ``wheels-repo/downloads/``. For a detailed explanation of the output files and directories, see -:doc:`files`. +:doc:`reference/files`. Pinning Versions with Constraints --------------------------------- @@ -76,8 +76,21 @@ The ``-c`` option ensures fromager uses exactly the versions you specify. Next Steps ---------- -Now that you've seen fromager work with a simple package, you might want to: +Now that you've seen fromager work with a simple package, explore: -* Learn to debug build failures with a more complex example in :doc:`getting-started` -* Customize builds with settings, patches, and variants in :doc:`customization` -* Check specific guides in :doc:`how-tos/index` +**Learn More:** + +* :doc:`getting-started` - Detailed walkthrough with debugging examples +* :doc:`concepts/index` - Understand how fromager works + +**Common Tasks:** + +* :doc:`how-tos/containers` - Run fromager in containers (recommended) +* :doc:`how-tos/bootstrap-constraints` - Pin versions for reproducible builds +* :doc:`how-tos/repeatable-builds` - Ensure consistent builds across environments + +**Customize Builds:** + +* :doc:`customization` - Comprehensive customization guide +* :doc:`reference/config-reference` - Configuration reference +* :doc:`how-tos/index` - Task-oriented guides diff --git a/docs/cli.rst b/docs/reference/cli.rst similarity index 100% rename from docs/cli.rst rename to docs/reference/cli.rst diff --git a/docs/config-reference.rst b/docs/reference/config-reference.rst similarity index 100% rename from docs/config-reference.rst rename to docs/reference/config-reference.rst diff --git a/docs/files.md b/docs/reference/files.md similarity index 100% rename from docs/files.md rename to docs/reference/files.md diff --git a/docs/glossary.rst b/docs/reference/glossary.rst similarity index 96% rename from docs/glossary.rst rename to docs/reference/glossary.rst index 371f8cd5..a04c6e6b 100644 --- a/docs/glossary.rst +++ b/docs/reference/glossary.rst @@ -40,7 +40,7 @@ This glossary defines key terms used throughout Fromager's documentation and cod The sequence in which packages must be built, determined by analyzing the :term:`dependency graph`. Packages are ordered bottom-up (topological sort) so that each package's dependencies are built before the package itself. - Stored in ``build-order.json``. See :doc:`/files` for the file format. + Stored in ``build-order.json``. See :doc:`/reference/files` for the file format. build sequence A command (``build-sequence``) that processes a pre-determined :term:`build order` @@ -94,7 +94,7 @@ This glossary defines key terms used throughout Fromager's documentation and cod Version specifications that control package :term:`resolution`. Provided via a ``constraints.txt`` file, they ensure specific versions are used or avoided during builds. Unlike :term:`requirements `, constraints only - apply when a package is already needed. See :doc:`/files` for format details. + apply when a package is already needed. See :doc:`/reference/files` for format details. cyclic dependency A circular dependency where packages depend on each other, forming a loop @@ -112,7 +112,7 @@ This glossary defines key terms used throughout Fromager's documentation and cod A directed graph representing all packages and their relationships discovered during :term:`bootstrap`. Nodes represent resolved package versions, and edges capture the :term:`requirement` specifications and dependency types (toplevel, - install, build-system, etc.). Stored in ``graph.json``. See :doc:`/files` and + install, build-system, etc.). Stored in ``graph.json``. See :doc:`/reference/files` and :doc:`/how-tos/graph-commands/index`. distribution name @@ -124,7 +124,7 @@ This glossary defines key terms used throughout Fromager's documentation and cod An extension point in fromager that allows customization of specific operations. Hooks include ``post_build``, ``prebuilt_wheel``, and ``post_bootstrap``. Multiple plugins can register for the same hook. - See :doc:`/hooks` and :doc:`/customization`. + See :doc:`/reference/hooks` and :doc:`/customization`. install dependency A runtime dependency of a package, extracted from the built :term:`wheel`'s @@ -153,7 +153,7 @@ This glossary defines key terms used throughout Fromager's documentation and cod A Python module registered as an entry point that provides custom implementations of fromager operations for specific packages. Unlike :term:`hooks `, overrides replace the default behavior entirely. Plugins can customize source acquisition, - dependency resolution, building, and more. See :doc:`/hooks`. + dependency resolution, building, and more. See :doc:`/reference/hooks`. package index A server providing package metadata and downloads, following the @@ -234,7 +234,7 @@ This glossary defines key terms used throughout Fromager's documentation and cod A strategy class that supplies version :term:`candidates ` during :term:`resolution`. The default provider queries :term:`PyPI`, but custom providers can resolve versions from GitHub tags, GitLab tags, or other sources. - See :doc:`/hooks`. + See :doc:`/reference/hooks`. sdist-only mode A :term:`bootstrap` mode (``--sdist-only``) that builds :term:`source @@ -246,7 +246,7 @@ This glossary defines key terms used throughout Fromager's documentation and cod Configuration options that customize package building. Can be global (in ``overrides/settings.yaml``) or per-package (in ``overrides/settings/.yaml``). Settings control environment variables, source URLs, :term:`variants `, - and more. See :doc:`/customization` and :doc:`/config-reference`. + and more. See :doc:`/customization` and :doc:`/reference/config-reference`. Simple API The :pep:`503` specification for :term:`package index` directory layout. @@ -299,4 +299,4 @@ This glossary defines key terms used throughout Fromager's documentation and cod work directory The directory (default: ``work-dir/``) where fromager stores working files during builds. Contains :term:`build order`, :term:`dependency graph`, - constraints, logs, and per-package build artifacts. See :doc:`/files`. + constraints, logs, and per-package build artifacts. See :doc:`/reference/files`. diff --git a/docs/hooks.rst b/docs/reference/hooks.rst similarity index 100% rename from docs/hooks.rst rename to docs/reference/hooks.rst diff --git a/docs/reference/index.rst b/docs/reference/index.rst new file mode 100644 index 00000000..e811e294 --- /dev/null +++ b/docs/reference/index.rst @@ -0,0 +1,47 @@ +Reference +========= + +This section contains detailed technical reference documentation for Fromager. + +Command Line Interface +---------------------- + +Complete documentation of all fromager CLI commands and options. + +.. toctree:: + :maxdepth: 2 + + cli.rst + +Configuration +------------- + +Comprehensive configuration reference for customizing package builds. + +.. toctree:: + :maxdepth: 2 + + config-reference.rst + hooks.rst + +Files and Formats +----------------- + +Documentation of input/output files and data formats used by fromager. + +.. toctree:: + :maxdepth: 2 + + files.md + glossary.rst + +Quick Reference +--------------- + +Looking for something specific? + +* **CLI Commands**: See :doc:`cli` for the complete command reference +* **Package Settings**: See :doc:`config-reference` for per-package and global settings +* **Override Plugins**: See :doc:`hooks` for customization hooks and extension points +* **File Formats**: See :doc:`files` for details on ``build-order.json``, ``graph.json``, and output directories +* **Terminology**: See :doc:`glossary` for definitions of key terms diff --git a/docs/using.md b/docs/using.md index 100ce1ec..404a6b08 100644 --- a/docs/using.md +++ b/docs/using.md @@ -252,3 +252,22 @@ The `step build-wheel` command creates a wheel using the build environment and prepared source, compiling any extensions using the appropriate override environment settings (refer to [customization](customization.md) for details about overrides). + +## See Also + +**Understanding Build Modes:** + +* [Bootstrap vs Build](concepts/bootstrap-vs-build.rst) - Detailed comparison of build modes +* [Dependencies](concepts/dependencies.rst) - How dependency resolution works + +**How-To Guides:** + +* [Bootstrap with Constraints](how-tos/bootstrap-constraints.rst) - Pin versions for reproducible builds +* [Repeatable Builds](how-tos/repeatable-builds.rst) - Use build graphs for consistency +* [Parallel Builds](how-tos/parallel.rst) - Speed up builds with parallelization +* [Build Web Server](how-tos/build-web-server.rst) - Serve built packages + +**Reference:** + +* [CLI Reference](reference/cli.rst) - Complete command documentation +* [Files Reference](reference/files.md) - Build order and graph file formats