Skip to content

linux_toolchain

Directory actions

More options

Directory actions

More options

Latest commit

 

History

History

linux_toolchain

Readme.md

Explicitly declare autodetected C++ toolchain when compiling on Linux

Goal

Show how to explicitly declare the C++ toolchain that is automatically detected by Bazel and used to compile by default on Linux.

About the example

In this example we will see how we can explicitly declare a C++ toolchain in Bazel taking as a reference the toolchain automatically generated.

Starting with Bazel 8 the only way to use toolchains is using platforms. Before, one could use --cpu, --crosstool_top and --compiler but it was complex and lacking flexibility. You might still find the command line options used for the old mechanism in the documentation but they are marked as no-op and will be deleted in future releases. In this example we will show how to use toolchains building with platforms.

Bazel toolchains

In order to have an easy and fast start, Bazel provies an automatic detection of toolchains. It detects which compilers are available in your system and configures automatically the C++ toolchains. In case of C++, a toolchain is composed mainly by compiler, linker, compiler flags, linker flags, system include directories, and library directories.

Obtain the default generted toolchain

The first think that we need to do, is to know which toolchain Bazel is using to compile C++ and where to find it. To do that we can build any target specifying the option --toolchain_resolution_debug=.*.

Note: If you follow the instructions with the code in this repository, you first need to comment out this line from MODULE.bazel and this one from .bazelrc.

bazel build --toolchain_resolution_debug=.* //:hello_world

When doing that we will see in the logs which toolchain Bazel decided to use. Not all toolchains that appear in the log are used, we need to pay special attention to the part that says Selected in the logs starting with ToolchainResolution:.

INFO: ToolchainResolution: Target platform @@platforms//host:host: Selected execution platform @@platforms//host:host,
INFO: ToolchainResolution: Performing resolution of @@bazel_tools//tools/cpp:toolchain_type for target platform @@platforms//host:host
      ToolchainResolution:   Rejected toolchain @@rules_cc++cc_configure_extension+local_config_cc//:cc-compiler-armeabi-v7a; mismatching values: armv7, android
      ToolchainResolution:   Toolchain @@rules_cc++cc_configure_extension+local_config_cc//:cc-compiler-k8 is compatible with target platform, searching for execution platforms:
      ToolchainResolution:     Compatible execution platform @@platforms//host:host
      ToolchainResolution:   All execution platforms have been assigned a @@bazel_tools//tools/cpp:toolchain_type toolchain, stopping
      ToolchainResolution: Recap of selected @@bazel_tools//tools/cpp:toolchain_type toolchains for target platform @@platforms//host:host:
      ToolchainResolution:   Selected @@rules_cc++cc_configure_extension+local_config_cc//:cc-compiler-k8 to run on execution platform @@platforms//host:host
INFO: ToolchainResolution: Target platform @@platforms//host:host: Selected execution platform @@platforms//host:host, type @@bazel_tools//tools/cpp:toolchain_type -> toolchain @@rules_cc++cc_configure_extension+local_config_cc//:cc-compiler-k8
INFO: ToolchainResolution: Target platform @@platforms//host:host: Selected execution platform @@platforms//host:host,
INFO: ToolchainResolution: Target platform @@platforms//host:host: Selected execution platform @@platforms//host:host,

In this case the relevant line is the one that contains Selected @@rules_cc++cc_configure_extension+local_config_cc//:cc-compiler-k8 to run on execution platform @@platforms//host:host and it is telling us that the toolchain selected is @@rules_cc++cc_configure_extension+local_config_cc//:cc-compiler-k8.

We need to keep in mind that the toolchain selected might depend on what is installed on the computer and what operating system you are using. In my case I'm using Ubuntu 24.04 with gcc 13. Also the content of the generated toolchain definition depends on the version of rules_cc that you are using in the MODULE.bazel file.

Now that we know what toolchain has been used, the next step is to take it as a reference to create our own toolchain. The good part of the autogenerated toolchain is that exist in the file system inside the bazel generated folders and we can take it almost as is.

Let's take it:

  • Create a folder called toolchain that is where we will store our toolchain.
mkdir toolchain
  • Copy the content of bazel-linux_toolchain/external/rules_cc++cc_configure_extension+local_config_cc except the REPO.bazel file, inside the toolchain folder that you just created. Make sure that are real copies and it does not contain any soft link.
cp -rL bazel-linux_toolchain/external/rules_cc++cc_configure_extension+local_config_cc/* toolchain/
rm toolchain/REPO.bazel

Make sure that the autogenerated toolchain is not used

Before proceeding with the configuration of our new toolchain that we just copied, we want to make sure that Bazel does not use the autogenerated one. To do that we can define the environment variable BAZEL_DO_NOT_DETECT_CPP_TOOLCHAIN=1.

Let's see what happens when we build our target with this option:

BAZEL_DO_NOT_DETECT_CPP_TOOLCHAIN=1 bazel build //:hello_world --toolchain_resolution_debug=.*
INFO: Invocation ID: 1b8e5c94-585f-41e0-ac4e-1c031f3278ea
INFO: ToolchainResolution: Performing resolution of @@bazel_tools//tools/cpp:toolchain_type for target platform @@platforms//host:host
      ToolchainResolution: No @@bazel_tools//tools/cpp:toolchain_type toolchain found for target platform @@platforms//host:host.
INFO: ToolchainResolution: Target platform @@platforms//host:host: Selected execution platform @@platforms//host:host,
ERROR: /home/limdor/.cache/bazel/_bazel_limdor/95b3a5978611a9c3f93e7e0a053332b3/external/rules_cc+/cc/BUILD:145:19: in cc_toolchain_alias rule @@rules_cc+//cc:current_cc_toolchain:
Traceback (most recent call last):
        File "/virtual_builtins_bzl/common/cc/cc_toolchain_alias.bzl", line 26, column 48, in _impl
        File "/virtual_builtins_bzl/common/cc/cc_helper.bzl", line 165, column 13, in _find_cpp_toolchain
Error in fail: Unable to find a CC toolchain using toolchain resolution. Target: @@rules_cc+//cc:current_cc_toolchain, Platform: @@bazel_tools//tools:host_platform, Exec platform: @@bazel_tools//tools:host_platform
ERROR: /home/limdor/.cache/bazel/_bazel_limdor/95b3a5978611a9c3f93e7e0a053332b3/external/rules_cc+/cc/BUILD:145:19: Analysis of target '@@rules_cc+//cc:current_cc_toolchain' (config: ca20f1d) failed
ERROR: Analysis of target '//:hello_world' failed; build aborted: Analysis failed
INFO: Elapsed time: 0.437s, Critical Path: 0.02s
INFO: 1 process: 1 internal.
ERROR: Build did NOT complete successfully

The relevant line here is Unable to find a CC toolchain using toolchain resolution. Target: @@rules_cc+//cc:current_cc_toolchain, Platform: @@bazel_tools//tools:host_platform, Exec platform: @@bazel_tools//tools:host_platform. As we wanted, Bazel is not able to compile anymore, because it cannot find the C++ toolchain. That is good because in this way we are sure that it does not take the autogenerated toolchain.

Configuring our explicitly declared toolchain

If we look in the BUILD file that we just copied inside the toolchain folder, we will see it contains multiple cc_toolchain and a cc_toolchain_suite target. To be able to use the toolchain building with platforms, we need to add an additional target called toolchain that contains the following information:

  • The toolchain to be used, needs to point to the cc_toolchain target, in this case cc-compiler-k8
  • The type of the toolchain, in this case C++
  • The constraints of the platform where the toolchain will be executed
  • The constraints of the platform that the toolchain is targeting. Meaning the platform where the code will be executed. In case that we would do cross-compilation then exec_compatible_with and target_compatible_with would be different.

The target that we need to add is as follows:

toolchain(
    name = "my_linux_toolchain",
    exec_compatible_with = [
        "@platforms//os:linux",
        "@platforms//cpu:x86_64",
    ],
    target_compatible_with = [
        "@platforms//os:linux",
        "@platforms//cpu:x86_64",
    ],
    toolchain = ":cc-compiler-k8",
    toolchain_type = "@bazel_tools//tools/cpp:toolchain_type",
)

And now we are ready for the last step, that is to inform Bazel that the toolchain is available. To do that we need to register the toolchain in the MODULE.bazel file.

register_toolchains("//toolchain:my_linux_toolchain")

Declaring the platform to be used

As we saw before, we might have two different platforms, one where the toolchain gets executed, and another one that is the target platform. If we want to be correct we should define the target platform and indicate it when we compile. If we do not do that, Bazel will assume that the target platform is the same like the host.

To declare the platform is pretty simple, we just need to create a platform target containing the constrains of our platform. In this case it is Linux on a x64 cpu.

To keep it structured, we put this target in a BUILD file in a platform folder:

platform(
    name = "linux_x64",
    constraint_values = [
        "@platforms//os:linux",
        "@platforms//cpu:x86_64",
    ],
    visibility = ["//visibility:public"],
)

Compiling with the explicitly declared toolchain

Now that the toolchain is registered, and the platform is declared, we just need to compile specifying the target platform and Bazel will know which toolchain to use.

BAZEL_DO_NOT_DETECT_CPP_TOOLCHAIN=1 bazel build //:hello_world --toolchain_resolution_debug=.* --platforms=//platform:linux_x64
INFO: Invocation ID: f1d65b61-8998-4f58-aff6-be4e282f52b8
WARNING: Build option --platforms has changed, discarding analysis cache (this can be expensive, see https://bazel.build/advanced/performance/iteration-speed).
INFO: ToolchainResolution: Target platform //platform:linux_x64: Selected execution platform @@platforms//host:host,
INFO: ToolchainResolution: Performing resolution of @@bazel_tools//tools/cpp:toolchain_type for target platform //platform:linux_x64
      ToolchainResolution:   Toolchain //toolchain:cc-compiler-k8 is compatible with target platform, searching for execution platforms:
      ToolchainResolution:     Compatible execution platform @@platforms//host:host
      ToolchainResolution:   All execution platforms have been assigned a @@bazel_tools//tools/cpp:toolchain_type toolchain, stopping
      ToolchainResolution: Recap of selected @@bazel_tools//tools/cpp:toolchain_type toolchains for target platform //platform:linux_x64:
      ToolchainResolution:   Selected //toolchain:cc-compiler-k8 to run on execution platform @@platforms//host:host
INFO: ToolchainResolution: Target platform //platform:linux_x64: Selected execution platform @@platforms//host:host, type @@bazel_tools//tools/cpp:toolchain_type -> toolchain //toolchain:cc-compiler-k8
INFO: ToolchainResolution: Target platform //platform:linux_x64: Selected execution platform @@platforms//host:host,
INFO: ToolchainResolution: Target platform @@platforms//host:host: Selected execution platform @@platforms//host:host,
INFO: Analyzed target //:hello_world (75 packages loaded, 497 targets configured).
INFO: Found 1 target...
Target //:hello_world up-to-date:
  bazel-bin/hello_world
INFO: Elapsed time: 1.996s, Critical Path: 0.75s
INFO: 4 processes: 3 action cache hit, 2 internal, 2 linux-sandbox.
INFO: Build completed successfully, 4 total actions

Because the part of the command line BAZEL_DO_NOT_DETECT_CPP_TOOLCHAIN=1 is something that we always want to do to make sure that we do not use anymore autmatically generated toolchains, we can extract it in the .bazelrc file.

And now we can build just doing:

bazel build //:hello_world --platforms=//platform:linux_x64

Cleaning up the toolchain and next steps

At this point everything is working fine but there is some cleanup that we can do. The automatic generated toolchain, contains indeed more than one toolchain but we do not use them all. Also contains a target called cc_toolchain_suite that is used in case of using --cpu and --crosstool_top, but that is not our case. Because of that, we should clean our toolchain folder and remove all unused targets like the arm targets (armeabi_cc_toolchain_config.bzl), cc_toolchain_suite, and any other target that is not referenced.

Because the automatic generated toolchain assumes the possibility that the compiler could be gcc or clang, we can also remove all clang branches from cc_toolchain_config.bzl. That might be different in your system if the selected compiler was not gcc.

Last but not least, I would also recommend that you apply buildifier to the files inside the toolchain folder. The automatically generated files are usually not properly formated.

Wrapping up

Now you have an explicitly declared toolchain for Linux. The next step is to go over the files and understand them. Then you can adapt the toolchain to your needs, adding features or changing compiler flags. Once you understand them you will see that you can keep cleaning up and remove unneeded features.

One of the next changes that is worth to mention, is the freezing of the compiler version. If you look at the default generated toolchain you will see that some of the tool paths are /usr/bin/cpp, /usr/bin/gcc, and /usr/bin/gcov. The problem of using this path, is that does not say anything about the version of the compiler to be used. If you have a single version installed this will always be the same, but if you want to make sure that the version used is the one that you want, you should change the path for /usr/bin/cpp-13, /usr/bin/gcc-13, and /usr/bin/gcov-13.

Related links

Related Bazel github issues