From 42a62e905d8f134deb2047bec2b1217be77c30ee Mon Sep 17 00:00:00 2001 From: Natalie Arellano Date: Fri, 15 Apr 2022 16:17:13 -0400 Subject: [PATCH 01/13] Platform api changes to support phase 1 of Dockerfiles implementation Signed-off-by: Natalie Arellano --- platform.md | 150 ++++++++++++++++++++++++++++++++++++++++++---------- 1 file changed, 121 insertions(+), 29 deletions(-) diff --git a/platform.md b/platform.md index 4cbf40cf..0e295d5f 100644 --- a/platform.md +++ b/platform.md @@ -42,25 +42,30 @@ Examples of a platform might include: - [Inputs](#inputs-2) - [Outputs](#outputs-2) - [Layer Restoration](#layer-restoration) - - [`builder`](#builder) + - [`generator`](#generator) - [Inputs](#inputs-3) - [Outputs](#outputs-3) - - [`exporter`](#exporter) + - [`builder`](#builder) - [Inputs](#inputs-4) - [Outputs](#outputs-4) - - [`creator`](#creator) + - [`exporter`](#exporter) - [Inputs](#inputs-5) - [Outputs](#outputs-5) - - [`rebaser`](#rebaser) + - [`creator`](#creator) - [Inputs](#inputs-6) - [Outputs](#outputs-6) - - [`launcher`](#launcher) + - [`rebaser`](#rebaser) - [Inputs](#inputs-7) - [Outputs](#outputs-7) + - [`launcher`](#launcher) + - [Inputs](#inputs-8) + - [Outputs](#outputs-8) - [Run Image Resolution](#run-image-resolution) - [Registry Authentication](#registry-authentication) - [Buildpacks](#buildpacks) - [Buildpacks Directory Layout](#buildpacks-directory-layout) + - [Image Extensions](#image-extensions) + - [Image Extensions Directory Layout](#image-extensions-directory-layout) - [Security Considerations](#security-considerations) - [Additional Guidance](#additional-guidance) - [Environment](#environment) @@ -73,6 +78,7 @@ Examples of a platform might include: - [Data Format](#data-format) - [Files](#files) - [`analyzed.toml` (TOML)](#analyzedtoml-toml) + - [`extender.toml` (TOML)](#extendertoml-toml) - [`group.toml` (TOML)](#grouptoml-toml) - [`metadata.toml` (TOML)](#metadatatoml-toml) - [`order.toml` (TOML)](#ordertoml-toml) @@ -122,6 +128,8 @@ A **launcher layer** refers to a layer in the app OCI image containing the **lau The **launcher** refers to a lifecycle executable packaged in the **app image** for the purpose of executing processes at runtime. +An **image extension** refers to software compliant with the [Image Extension Interface Specification](image-extension.md). Image extensions participate in detection and execute before the buildpack build process. + #### Additional Terminology An **image reference** refers to either a **tag reference** or **digest reference**. @@ -334,13 +342,13 @@ Usage: | `/dev/stderr` | Logs (warnings, errors) | `` | Analysis metadata (see [`analyzed.toml`](#analyzedtoml-toml) -| Exit Code | Result| -|-----------|-------| -| `0` | Success -| `11` | Platform API incompatibility error -| `12` | Buildpack API incompatibility error -| `1-10`, `13-99` | Generic lifecycle errors -| `30-39` | Analysis-specific lifecycle errors +| Exit Code | Result| +|-----------------|-------| +| `0` | Success +| `11` | Platform API incompatibility error +| `12` | Buildpack API incompatibility error +| `1-10`, `13-19` | Generic lifecycle errors +| `30-39` | Analysis-specific lifecycle errors - The lifecycle MUST write [analysis metadata](#analyzedtoml-toml) to ``, where: - `image` MUST describe the ``, if accessible @@ -354,6 +362,7 @@ Usage: /cnb/lifecycle/detector \ [-app ] \ [-buildpacks ] \ + [-exts ] \ [-group ] \ [-layers ] \ [-log-level ] \ @@ -363,16 +372,17 @@ Usage: ``` ##### Inputs -| Input | Environment Variable | Default Value | Description -|---------------|-------------------------|--------------------------------------------------------|------- -| `` | `CNB_APP_DIR` | `/workspace` | Path to application directory -| `` | `CNB_BUILDPACKS_DIR` | `/cnb/buildpacks` | Path to buildpacks directory (see [Buildpacks Directory Layout](#buildpacks-directory-layout)) -| `` | `CNB_GROUP_PATH` | `/group.toml` | Path to output group definition -| `` | `CNB_LAYERS_DIR` | `/layers` | Path to layers directory -| `` | `CNB_LOG_LEVEL` | `info` | Log Level -| `` | `CNB_ORDER_PATH` | `/order.toml` if present, or `/cnb/order.toml` | Path resolution for order definition (see [`order.toml`](#ordertoml-toml)) -| `` | `CNB_PLAN_PATH` | `/plan.toml` | Path to output resolved build plan -| `` | `CNB_PLATFORM_DIR` | `/platform` | Path to platform directory +| Input | Environment Variable | Default Value | Description | +|----------------|----------------------|--------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------| +| `` | `CNB_APP_DIR` | `/workspace` | Path to application directory | +| `` | `CNB_BUILDPACKS_DIR` | `/cnb/buildpacks` | Path to buildpacks directory (see [Buildpacks Directory Layout](#buildpacks-directory-layout)) | +| `` | `CNB_IMAGE_EXTS_DIR` | `/cnb/exts` | Path to image extensions directory (see [Image Extensions Directory Layout](#image-extensions-directory-layout) | +| `` | `CNB_GROUP_PATH` | `/group.toml` | Path to output group definition | +| `` | `CNB_LAYERS_DIR` | `/layers` | Path to layers directory | +| `` | `CNB_LOG_LEVEL` | `info` | Log Level | +| `` | `CNB_ORDER_PATH` | `/order.toml` if present, or `/cnb/order.toml` | Path resolution for order definition (see [`order.toml`](#ordertoml-toml)) | +| `` | `CNB_PLAN_PATH` | `/plan.toml` | Path to output resolved build plan | +| `` | `CNB_PLATFORM_DIR` | `/platform` | Path to platform directory | ##### Outputs | Output | Description @@ -436,13 +446,13 @@ Usage: | `//.sbom.` | Files containing the Software Bill of Materials for each analyzed layer (see [Buildpack Interface Specification](buildpack.md)) | `///*`. | Restored layer contents -| Exit Code | Result| -|-----------|-------| -| `0` | Success -| `11` | Platform API incompatibility error -| `12` | Buildpack API incompatibility error -| `1-10`, `13-99` | Generic lifecycle errors -| `40-49` | Restoration-specific lifecycle errors +| Exit Code | Result| +|-----------------|-------| +| `0` | Success +| `11` | Platform API incompatibility error +| `12` | Buildpack API incompatibility error +| `1-10`, `13-19` | Generic lifecycle errors +| `40-49` | Restoration-specific lifecycle errors - For each buildpack in ``, if persistent metadata for that buildpack exists in the analysis metadata, lifecycle MUST write a toml representation of the persistent metadata to `//store.toml` - **If** `` is `true` the lifecycle MUST NOT perform layer restoration. @@ -451,6 +461,60 @@ Usage: ##### Layer Restoration lifeycle MUST use the provided `cache-dir` or `cache-image` to retrieve cache contents. The [rules](https://github.com/buildpacks/spec/blob/main/buildpack.md#layer-types) for restoration MUST be followed when determining how and when to store cache layers. +#### `generator` +The platform MAY execute `generator` in the **build environment** for Linux builds + +Usage: +``` +/cnb/lifecycle/generator \ + [-analyzed ] \ + [-app ] \ + [-extender ] \ + [-exts ] \ + [-group ] \ + [-log-level ] \ + [-output ] \ + [-plan ] \ + [-platform ] +``` + +##### Inputs +| Input | Env | Default Value | Description | +|---------------|----------------------|--------------------------|-----------------------------------------------------------------------------------------------------------------| +| `` | `CNB_ANALYZED_PATH` | `/analyzed.toml` | Path to analysis metadata (see [`analyzed.toml`](#analyzedtoml-toml) | +| `` | `CNB_APP_DIR` | `/workspace` | Path to application directory | +| `` | `CNB_IMAGE_EXTS_DIR` | `/cnb/exts` | Path to image extensions directory (see [Image Extensions Directory Layout](#image-extensions-directory-layout) | +| `` | `CNB_GROUP_PATH` | `/group.toml` | Path to group definition (see [`group.toml`](#grouptoml-toml)) | +| `` | `CNB_LOG_LEVEL` | `info` | Log Level | +| `` | `CNB_EXT_OUTPUT_DIR` | `/exts` | Path to output directory | +| `` | `CNB_PLAN_PATH` | `/plan.toml` | Path to resolved build plan (see [`plan.toml`](#plantoml-toml)) | +| `` | `CNB_PLATFORM_DIR` | `/platform` | Path to platform directory | + +##### Outputs +| Output | Description | +|----------------------------------------------|-------------------------------------------------------------------------------| +| [exit status] | (see Exit Code table below for values) | +| `/dev/stdout` | Logs (info) | +| `/dev/stderr` | Logs (warnings, errors) | +| `//` | Generated Dockerfiles (see [Image Extension Specfication](image-extension.md) | +| `/config/metadata.toml` | Build metadata (see [`metadata.toml`](#metadatatoml-toml)) | + +| Exit Code | Result | +|-----------------|--------------------------------------| +| `0` | Success | +| `11` | Platform API incompatibility error | +| `12` | Buildpack API incompatibility error | +| `1-10`, `13-19` | Generic lifecycle errors | +| `91` | Image extension build error | +| `90`, `92-99` | Generation-specific lifecycle errors | + +- The lifecycle SHALL execute all image extensions in the order defined in `` according to the process outlined in the [Buildpack Interface Specification](buildpack.md). +- The lifecycle SHALL replace the `run-image` reference in `` with the selected run image reference. The selected run image reference SHALL be the base image referenced in the Dockerfile output by the last image extension in the group. +- The lifecycle SHALL add all invoked image extensions to`/config/metadata.toml`. +- The lifecycle SHALL record all Dockerfiles in ``. +- The lifecycle SHALL copy all output Dockerfiles to ``. +- The lifecycle SHALL filter the build plan with dependencies provided by image extensions. + #### `builder` The platform MUST execute `builder` in the **build environment** @@ -816,6 +880,15 @@ The buildpacks directory MUST contain unarchived buildpacks such that: - Each top-level directory is a buildpack ID. - Each second-level directory is a buildpack version. +## Image Extensions + +### Image Extensions Directory Layout + +The image extensions directory MUST contain unarchived image extensions such that: + +- Each top-level directory is an image extension ID. +- Each second-level directory is an image extension version. + ## Security Considerations The platform SHOULD run each phase of the lifecycle in an isolated container to prevent untrusted app and buildpack code from accessing storage credentials needed during the export and analysis phases. @@ -907,6 +980,14 @@ Where: - `run-image.reference` MUST be either a digest reference to an image in a docker registry or the ID of an image in a docker daemon - `previous-image.metadata` MUST be the TOML representation of the layer [metadata label](#iobuildpackslifecyclemetadata-json) +#### `extender.toml` (TOML) + +```toml +[[dockerfiles]] + extension_id = "" + path = "" +``` + #### `group.toml` (TOML) ```toml @@ -915,6 +996,7 @@ id = "" version = "" api = "" homepage = "" +extension = false ``` Where: @@ -931,6 +1013,11 @@ version = "" api = "" optional = false +[[image-extensions]] +id = "" +version = "" +api = "" + [[processes]] type = "" command = "" @@ -955,6 +1042,11 @@ Where: id = "" version = "" optional = false + +[[order-ext]] +[[order-ext.group]] +id = "" +version = "" ``` Where: From e669a537b79593956217d00d210ed19e50beea5c Mon Sep 17 00:00:00 2001 From: Natalie Arellano Date: Mon, 2 May 2022 12:44:01 -0400 Subject: [PATCH 02/13] Add extensions to build metadata Signed-off-by: Natalie Arellano --- platform.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/platform.md b/platform.md index 0e295d5f..b074f578 100644 --- a/platform.md +++ b/platform.md @@ -1149,6 +1149,13 @@ Where: "homepage": "" } ], + "extensions": [ + { + "id": "", + "version": "", + "homepage": "" + } + ], "launcher": { "version": "", "source": { From 9867543440373b98b382f37c713f42ae8ee70e1b Mon Sep 17 00:00:00 2001 From: Natalie Arellano Date: Wed, 15 Jun 2022 17:14:23 -0400 Subject: [PATCH 03/13] Updates from implementation Signed-off-by: Natalie Arellano --- platform.md | 119 ++++++++++++++++++---------------------------------- 1 file changed, 40 insertions(+), 79 deletions(-) diff --git a/platform.md b/platform.md index b074f578..340cad99 100644 --- a/platform.md +++ b/platform.md @@ -42,9 +42,6 @@ Examples of a platform might include: - [Inputs](#inputs-2) - [Outputs](#outputs-2) - [Layer Restoration](#layer-restoration) - - [`generator`](#generator) - - [Inputs](#inputs-3) - - [Outputs](#outputs-3) - [`builder`](#builder) - [Inputs](#inputs-4) - [Outputs](#outputs-4) @@ -78,7 +75,7 @@ Examples of a platform might include: - [Data Format](#data-format) - [Files](#files) - [`analyzed.toml` (TOML)](#analyzedtoml-toml) - - [`extender.toml` (TOML)](#extendertoml-toml) + - [`generated.toml` (TOML)](#generatedtoml-toml) - [`group.toml` (TOML)](#grouptoml-toml) - [`metadata.toml` (TOML)](#metadatatoml-toml) - [`order.toml` (TOML)](#ordertoml-toml) @@ -361,12 +358,15 @@ Usage: ``` /cnb/lifecycle/detector \ [-app ] \ + [-analyzed ] \ [-buildpacks ] \ - [-exts ] \ + [-extensions ] \ + [-generated ] \ [-group ] \ [-layers ] \ [-log-level ] \ [-order ] \ + [-output-dir ] \ [-plan ] \ [-platform ] ``` @@ -374,39 +374,53 @@ Usage: ##### Inputs | Input | Environment Variable | Default Value | Description | |----------------|----------------------|--------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------| +| `` | `CNB_ANALYZED_PATH` | `/analyzed.toml` | Path to output analysis metadata (see [`analyzed.toml`](#analyzedtoml-toml) | | `` | `CNB_APP_DIR` | `/workspace` | Path to application directory | | `` | `CNB_BUILDPACKS_DIR` | `/cnb/buildpacks` | Path to buildpacks directory (see [Buildpacks Directory Layout](#buildpacks-directory-layout)) | -| `` | `CNB_IMAGE_EXTS_DIR` | `/cnb/exts` | Path to image extensions directory (see [Image Extensions Directory Layout](#image-extensions-directory-layout) | +| `` | `CNB_EXTENSIONS_DIR` | `/cnb/extensions` | Path to image extensions directory (see [Image Extensions Directory Layout](#image-extensions-directory-layout) | +| `` | `CNB_GENERATED_PATH` | `/generated.toml` | Path to output metadata about generated Dockerfiles from image extensions | | `` | `CNB_GROUP_PATH` | `/group.toml` | Path to output group definition | | `` | `CNB_LAYERS_DIR` | `/layers` | Path to layers directory | | `` | `CNB_LOG_LEVEL` | `info` | Log Level | | `` | `CNB_ORDER_PATH` | `/order.toml` if present, or `/cnb/order.toml` | Path resolution for order definition (see [`order.toml`](#ordertoml-toml)) | +| `` | `CNB_OUTPUT_DIR` | ` | Path to output directory | | `` | `CNB_PLAN_PATH` | `/plan.toml` | Path to output resolved build plan | | `` | `CNB_PLATFORM_DIR` | `/platform` | Path to platform directory | ##### Outputs -| Output | Description -|--------------------|---------------------------------------------- -| [exit status] | (see Exit Code table below for values) -| `/dev/stdout` | Logs (info) -| `/dev/stderr` | Logs (warnings, errors) -| `` | Detected buildpack group (see [`group.toml`](#grouptoml-toml)) -| `` | Resolved Build Plan (see [`plan.toml`](#plantoml-toml)) - -| Exit Code | Result| -|-----------|-------| -| `0` | Success -| `11` | Platform API incompatibility error -| `12` | Buildpack API incompatibility error -| `1-10`, `13-19` | Generic lifecycle errors -| `20` | All buildpacks groups have failed to detect w/o error -| `21` | All buildpack groups have failed to detect and at least one buildpack has errored -| `22-29` | Detection-specific lifecycle errors +| Output | Description | +|--------------------------------------------------|----------------------------------------------------------------------------------------------------------| +| [exit status] | (see Exit Code table below for values) | +| `/dev/stdout` | Logs (info) | +| `/dev/stderr` | Logs (warnings, errors) | +| `` | Detected buildpack group (see [`group.toml`](#grouptoml-toml)) | +| `` | Resolved Build Plan (see [`plan.toml`](#plantoml-toml)) | +| `` | Metadata about generated Dockerfiles from image extensions (see [`generated.toml`](#generatedtoml-toml)) | +| `` | Updated to include the run image obtained from applying generated Dockerfiles | +| `//` | Generated Dockerfiles (see [Image Extension Specfication](image-extension.md) | + +| Exit Code | Result | +|-----------------|-----------------------------------------------------------------------------------| +| `0` | Success | +| `11` | Platform API incompatibility error | +| `12` | Buildpack API incompatibility error | +| `1-10`, `13-19` | Generic lifecycle errors | +| `20` | All buildpacks groups have failed to detect w/o error | +| `21` | All buildpack groups have failed to detect and at least one buildpack has errored | +| `22-29` | Detection-specific lifecycle errors | +| `80-89` | Generation-specific lifecycle errors | The lifecycle: - SHALL detect a single group from `` and write it to `` using the [detection process](buildpack.md#phase-1-detection) outlined in the Buildpack Interface Specification - SHALL write the resolved build plan from the detected group to `` +When image extensions are present in the order (**experimental**), the lifecycle: +- SHALL execute all image extensions in the order defined in `` according to the process outlined in the [Buildpack Interface Specification](buildpack.md). +- SHALL record all generated run.Dockerfiles in ``. +- SHALL copy all generated run.Dockerfiles to ``/generated. +- SHALL replace the `run-image` reference in `` with the selected run image reference. The selected run image reference SHALL be the base image referenced in the Dockerfile output by the last image extension in the group. +- SHALL filter the build plan with dependencies provided by image extensions. + #### `restorer` Usage: ``` @@ -461,60 +475,6 @@ Usage: ##### Layer Restoration lifeycle MUST use the provided `cache-dir` or `cache-image` to retrieve cache contents. The [rules](https://github.com/buildpacks/spec/blob/main/buildpack.md#layer-types) for restoration MUST be followed when determining how and when to store cache layers. -#### `generator` -The platform MAY execute `generator` in the **build environment** for Linux builds - -Usage: -``` -/cnb/lifecycle/generator \ - [-analyzed ] \ - [-app ] \ - [-extender ] \ - [-exts ] \ - [-group ] \ - [-log-level ] \ - [-output ] \ - [-plan ] \ - [-platform ] -``` - -##### Inputs -| Input | Env | Default Value | Description | -|---------------|----------------------|--------------------------|-----------------------------------------------------------------------------------------------------------------| -| `` | `CNB_ANALYZED_PATH` | `/analyzed.toml` | Path to analysis metadata (see [`analyzed.toml`](#analyzedtoml-toml) | -| `` | `CNB_APP_DIR` | `/workspace` | Path to application directory | -| `` | `CNB_IMAGE_EXTS_DIR` | `/cnb/exts` | Path to image extensions directory (see [Image Extensions Directory Layout](#image-extensions-directory-layout) | -| `` | `CNB_GROUP_PATH` | `/group.toml` | Path to group definition (see [`group.toml`](#grouptoml-toml)) | -| `` | `CNB_LOG_LEVEL` | `info` | Log Level | -| `` | `CNB_EXT_OUTPUT_DIR` | `/exts` | Path to output directory | -| `` | `CNB_PLAN_PATH` | `/plan.toml` | Path to resolved build plan (see [`plan.toml`](#plantoml-toml)) | -| `` | `CNB_PLATFORM_DIR` | `/platform` | Path to platform directory | - -##### Outputs -| Output | Description | -|----------------------------------------------|-------------------------------------------------------------------------------| -| [exit status] | (see Exit Code table below for values) | -| `/dev/stdout` | Logs (info) | -| `/dev/stderr` | Logs (warnings, errors) | -| `//` | Generated Dockerfiles (see [Image Extension Specfication](image-extension.md) | -| `/config/metadata.toml` | Build metadata (see [`metadata.toml`](#metadatatoml-toml)) | - -| Exit Code | Result | -|-----------------|--------------------------------------| -| `0` | Success | -| `11` | Platform API incompatibility error | -| `12` | Buildpack API incompatibility error | -| `1-10`, `13-19` | Generic lifecycle errors | -| `91` | Image extension build error | -| `90`, `92-99` | Generation-specific lifecycle errors | - -- The lifecycle SHALL execute all image extensions in the order defined in `` according to the process outlined in the [Buildpack Interface Specification](buildpack.md). -- The lifecycle SHALL replace the `run-image` reference in `` with the selected run image reference. The selected run image reference SHALL be the base image referenced in the Dockerfile output by the last image extension in the group. -- The lifecycle SHALL add all invoked image extensions to`/config/metadata.toml`. -- The lifecycle SHALL record all Dockerfiles in ``. -- The lifecycle SHALL copy all output Dockerfiles to ``. -- The lifecycle SHALL filter the build plan with dependencies provided by image extensions. - #### `builder` The platform MUST execute `builder` in the **build environment** @@ -980,11 +940,12 @@ Where: - `run-image.reference` MUST be either a digest reference to an image in a docker registry or the ID of an image in a docker daemon - `previous-image.metadata` MUST be the TOML representation of the layer [metadata label](#iobuildpackslifecyclemetadata-json) -#### `extender.toml` (TOML) +#### `generated.toml` (TOML) ```toml [[dockerfiles]] extension_id = "" + kind = "run" path = "" ``` @@ -1013,7 +974,7 @@ version = "" api = "" optional = false -[[image-extensions]] +[[extensions]] id = "" version = "" api = "" From da18b780cfdced338d9ff0a44a47b7aed6d83462 Mon Sep 17 00:00:00 2001 From: Natalie Arellano Date: Wed, 15 Jun 2022 17:18:51 -0400 Subject: [PATCH 04/13] Add extension to build plan Signed-off-by: Natalie Arellano --- platform.md | 1 + 1 file changed, 1 insertion(+) diff --git a/platform.md b/platform.md index 340cad99..f5869e7a 100644 --- a/platform.md +++ b/platform.md @@ -1022,6 +1022,7 @@ Where: [[entries.providers]] id = "" version = "buildpack Version" + extension = false [[entries.requires]] name = "" From b8e6b139f2ab199aac93592f6487519cd5017448 Mon Sep 17 00:00:00 2001 From: Natalie Arellano Date: Fri, 24 Jun 2022 15:29:55 -0400 Subject: [PATCH 05/13] Updates per recent Working Group discussions Signed-off-by: Natalie Arellano --- platform.md | 50 +++++++++++++++++++++----------------------------- 1 file changed, 21 insertions(+), 29 deletions(-) diff --git a/platform.md b/platform.md index f5869e7a..d3a19115 100644 --- a/platform.md +++ b/platform.md @@ -75,7 +75,6 @@ Examples of a platform might include: - [Data Format](#data-format) - [Files](#files) - [`analyzed.toml` (TOML)](#analyzedtoml-toml) - - [`generated.toml` (TOML)](#generatedtoml-toml) - [`group.toml` (TOML)](#grouptoml-toml) - [`metadata.toml` (TOML)](#metadatatoml-toml) - [`order.toml` (TOML)](#ordertoml-toml) @@ -361,7 +360,6 @@ Usage: [-analyzed ] \ [-buildpacks ] \ [-extensions ] \ - [-generated ] \ [-group ] \ [-layers ] \ [-log-level ] \ @@ -378,26 +376,24 @@ Usage: | `` | `CNB_APP_DIR` | `/workspace` | Path to application directory | | `` | `CNB_BUILDPACKS_DIR` | `/cnb/buildpacks` | Path to buildpacks directory (see [Buildpacks Directory Layout](#buildpacks-directory-layout)) | | `` | `CNB_EXTENSIONS_DIR` | `/cnb/extensions` | Path to image extensions directory (see [Image Extensions Directory Layout](#image-extensions-directory-layout) | -| `` | `CNB_GENERATED_PATH` | `/generated.toml` | Path to output metadata about generated Dockerfiles from image extensions | | `` | `CNB_GROUP_PATH` | `/group.toml` | Path to output group definition | | `` | `CNB_LAYERS_DIR` | `/layers` | Path to layers directory | | `` | `CNB_LOG_LEVEL` | `info` | Log Level | | `` | `CNB_ORDER_PATH` | `/order.toml` if present, or `/cnb/order.toml` | Path resolution for order definition (see [`order.toml`](#ordertoml-toml)) | -| `` | `CNB_OUTPUT_DIR` | ` | Path to output directory | +| `` | `CNB_OUTPUT_DIR` | ` | Path to output directory for generated Dockerfiles | | `` | `CNB_PLAN_PATH` | `/plan.toml` | Path to output resolved build plan | | `` | `CNB_PLATFORM_DIR` | `/platform` | Path to platform directory | ##### Outputs -| Output | Description | -|--------------------------------------------------|----------------------------------------------------------------------------------------------------------| -| [exit status] | (see Exit Code table below for values) | -| `/dev/stdout` | Logs (info) | -| `/dev/stderr` | Logs (warnings, errors) | -| `` | Detected buildpack group (see [`group.toml`](#grouptoml-toml)) | -| `` | Resolved Build Plan (see [`plan.toml`](#plantoml-toml)) | -| `` | Metadata about generated Dockerfiles from image extensions (see [`generated.toml`](#generatedtoml-toml)) | -| `` | Updated to include the run image obtained from applying generated Dockerfiles | -| `//` | Generated Dockerfiles (see [Image Extension Specfication](image-extension.md) | +| Output | Description | +|----------------------------------------------------------|----------------------------------------------------------------------------------------------------------| +| [exit status] | (see Exit Code table below for values) | +| `/dev/stdout` | Logs (info) | +| `/dev/stderr` | Logs (warnings, errors) | +| `` | Detected buildpack group (see [`group.toml`](#grouptoml-toml)) | +| `` | Resolved Build Plan (see [`plan.toml`](#plantoml-toml)) | +| `` | Updated to include the run image obtained from applying generated Dockerfiles | +| `/generated/run//Dockerfile` | Generated Dockerfiles (see [Image Extension Specfication](image-extension.md) | | Exit Code | Result | |-----------------|-----------------------------------------------------------------------------------| @@ -408,7 +404,8 @@ Usage: | `20` | All buildpacks groups have failed to detect w/o error | | `21` | All buildpack groups have failed to detect and at least one buildpack has errored | | `22-29` | Detection-specific lifecycle errors | -| `80-89` | Generation-specific lifecycle errors | +| `81` | Extension generate error | +| `82-89` | Generation-specific lifecycle errors | The lifecycle: - SHALL detect a single group from `` and write it to `` using the [detection process](buildpack.md#phase-1-detection) outlined in the Buildpack Interface Specification @@ -416,8 +413,7 @@ The lifecycle: When image extensions are present in the order (**experimental**), the lifecycle: - SHALL execute all image extensions in the order defined in `` according to the process outlined in the [Buildpack Interface Specification](buildpack.md). -- SHALL record all generated run.Dockerfiles in ``. -- SHALL copy all generated run.Dockerfiles to ``/generated. +- SHALL copy all generated run.Dockerfiles to `/generated/run//Dockerfile`. - SHALL replace the `run-image` reference in `` with the selected run image reference. The selected run image reference SHALL be the base image referenced in the Dockerfile output by the last image extension in the group. - SHALL filter the build plan with dependencies provided by image extensions. @@ -940,15 +936,6 @@ Where: - `run-image.reference` MUST be either a digest reference to an image in a docker registry or the ID of an image in a docker daemon - `previous-image.metadata` MUST be the TOML representation of the layer [metadata label](#iobuildpackslifecyclemetadata-json) -#### `generated.toml` (TOML) - -```toml -[[dockerfiles]] - extension_id = "" - kind = "run" - path = "" -``` - #### `group.toml` (TOML) ```toml @@ -957,7 +944,12 @@ id = "" version = "" api = "" homepage = "" -extension = false + +[[group-extensions]] +id = "" +version = "" +api = "" +homepage = "" ``` Where: @@ -1004,8 +996,8 @@ id = "" version = "" optional = false -[[order-ext]] -[[order-ext.group]] +[[order-extensions]] +[[order-extensions.group]] id = "" version = "" ``` From 1d79fcb2bb825aabc25ad1639f4a5c7a712b69c3 Mon Sep 17 00:00:00 2001 From: Natalie Arellano Date: Fri, 24 Jun 2022 15:48:59 -0400 Subject: [PATCH 06/13] Update with information about experimental features Signed-off-by: Natalie Arellano --- platform.md | 56 +++++++++++++++++++++++++++++++++-------------------- 1 file changed, 35 insertions(+), 21 deletions(-) diff --git a/platform.md b/platform.md index d3a19115..65f6a7c0 100644 --- a/platform.md +++ b/platform.md @@ -43,22 +43,23 @@ Examples of a platform might include: - [Outputs](#outputs-2) - [Layer Restoration](#layer-restoration) - [`builder`](#builder) + - [Inputs](#inputs-3) + - [Outputs](#outputs-3) + - [`exporter`](#exporter) - [Inputs](#inputs-4) - [Outputs](#outputs-4) - - [`exporter`](#exporter) + - [`creator`](#creator) - [Inputs](#inputs-5) - [Outputs](#outputs-5) - - [`creator`](#creator) + - [`rebaser`](#rebaser) - [Inputs](#inputs-6) - [Outputs](#outputs-6) - - [`rebaser`](#rebaser) + - [`launcher`](#launcher) - [Inputs](#inputs-7) - [Outputs](#outputs-7) - - [`launcher`](#launcher) - - [Inputs](#inputs-8) - - [Outputs](#outputs-8) - [Run Image Resolution](#run-image-resolution) - [Registry Authentication](#registry-authentication) + - [Experimental Features](#experimental-features) - [Buildpacks](#buildpacks) - [Buildpacks Directory Layout](#buildpacks-directory-layout) - [Image Extensions](#image-extensions) @@ -228,9 +229,10 @@ Mixin authors MUST ensure that mixins do not affect the [ABI-compatibility](http During build, platforms MUST use the same set of mixins for the run image as were used in the build image (excluding mixins that have a stage specifier). ## Lifecycle Interface + ### Platform API Compatibility -The platform SHOULD set `CNB_PLATFORM_API=` in the lifecycle's execution environment +The platform SHOULD set `CNB_PLATFORM_API=` in the lifecycle's execution environment. If `CNB_PLATFORM_API` is set in the lifecycle's execution environment, the lifecycle: - MUST either conform to the matching version of this specification or @@ -370,19 +372,19 @@ Usage: ``` ##### Inputs -| Input | Environment Variable | Default Value | Description | -|----------------|----------------------|--------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------| -| `` | `CNB_ANALYZED_PATH` | `/analyzed.toml` | Path to output analysis metadata (see [`analyzed.toml`](#analyzedtoml-toml) | -| `` | `CNB_APP_DIR` | `/workspace` | Path to application directory | -| `` | `CNB_BUILDPACKS_DIR` | `/cnb/buildpacks` | Path to buildpacks directory (see [Buildpacks Directory Layout](#buildpacks-directory-layout)) | -| `` | `CNB_EXTENSIONS_DIR` | `/cnb/extensions` | Path to image extensions directory (see [Image Extensions Directory Layout](#image-extensions-directory-layout) | -| `` | `CNB_GROUP_PATH` | `/group.toml` | Path to output group definition | -| `` | `CNB_LAYERS_DIR` | `/layers` | Path to layers directory | -| `` | `CNB_LOG_LEVEL` | `info` | Log Level | -| `` | `CNB_ORDER_PATH` | `/order.toml` if present, or `/cnb/order.toml` | Path resolution for order definition (see [`order.toml`](#ordertoml-toml)) | -| `` | `CNB_OUTPUT_DIR` | ` | Path to output directory for generated Dockerfiles | -| `` | `CNB_PLAN_PATH` | `/plan.toml` | Path to output resolved build plan | -| `` | `CNB_PLATFORM_DIR` | `/platform` | Path to platform directory | +| Input | Environment Variable | Default Value | Description | +|----------------|----------------------|--------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `` | `CNB_ANALYZED_PATH` | `/analyzed.toml` | (**[experimental](#experimental-features)**) Path to output analysis metadata (see [`analyzed.toml`](#analyzedtoml-toml) | +| `` | `CNB_APP_DIR` | `/workspace` | Path to application directory | +| `` | `CNB_BUILDPACKS_DIR` | `/cnb/buildpacks` | Path to buildpacks directory (see [Buildpacks Directory Layout](#buildpacks-directory-layout)) | +| `` | `CNB_EXTENSIONS_DIR` | `/cnb/extensions` | (**[experimental](#experimental-features)**) Path to image extensions directory (see [Image Extensions Directory Layout](#image-extensions-directory-layout) | +| `` | `CNB_GROUP_PATH` | `/group.toml` | Path to output group definition | +| `` | `CNB_LAYERS_DIR` | `/layers` | Path to layers directory | +| `` | `CNB_LOG_LEVEL` | `info` | Log Level | +| `` | `CNB_ORDER_PATH` | `/order.toml` if present, or `/cnb/order.toml` | Path resolution for order definition (see [`order.toml`](#ordertoml-toml)) | +| `` | `CNB_OUTPUT_DIR` | ` | (**[experimental](#experimental-features)**) Path to output directory for generated Dockerfiles | +| `` | `CNB_PLAN_PATH` | `/plan.toml` | Path to output resolved build plan | +| `` | `CNB_PLATFORM_DIR` | `/platform` | Path to platform directory | ##### Outputs | Output | Description | @@ -411,7 +413,7 @@ The lifecycle: - SHALL detect a single group from `` and write it to `` using the [detection process](buildpack.md#phase-1-detection) outlined in the Buildpack Interface Specification - SHALL write the resolved build plan from the detected group to `` -When image extensions are present in the order (**experimental**), the lifecycle: +When image extensions are present in the order (**[experimental](#experimental-features)**), the lifecycle: - SHALL execute all image extensions in the order defined in `` according to the process outlined in the [Buildpack Interface Specification](buildpack.md). - SHALL copy all generated run.Dockerfiles to `/generated/run//Dockerfile`. - SHALL replace the `run-image` reference in `` with the selected run image reference. The selected run image reference SHALL be the base image referenced in the Dockerfile output by the last image extension in the group. @@ -827,6 +829,18 @@ The lifecycle MAY provide other mechanisms by which a platform can supply regist The lifecycle MUST attempt to authenticate anonymously if no matching credentials are found. +### Experimental Features + +Where noted, certain features are considered experimental and susceptible to change in a future API version. + +The platform SHOULD set `CNB_PLATFORM_EXPERIMENTAL_MODE=` in the lifecycle's execution environment to control the behavior of experimental features. + +When an experimental feature is invoked, the lifecycle: +- SHALL warn and continue if `CNB_PLATFORM_EXPERIMENTAL_MODE` is unset +- SHALL warn and continue if `CNB_PLATFORM_EXPERIMENTAL_MODE=warn` +- SHALL fail if `CNB_PLATFORM_EXPERIMENTAL_MODE=error` +- SHALL continue without warning if `CNB_PLATFORM_EXPERIMENTAL_MODE=silent` + ## Buildpacks ### Buildpacks Directory Layout From 251c7a0a2a118376fe537d18ecf0bf9d18e5b345 Mon Sep 17 00:00:00 2001 From: Natalie Arellano Date: Wed, 29 Jun 2022 14:01:19 -0400 Subject: [PATCH 07/13] Fix generate exit code The 80 range is for the launcher. Signed-off-by: Natalie Arellano --- platform.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/platform.md b/platform.md index 65f6a7c0..7c69c1cd 100644 --- a/platform.md +++ b/platform.md @@ -406,8 +406,8 @@ Usage: | `20` | All buildpacks groups have failed to detect w/o error | | `21` | All buildpack groups have failed to detect and at least one buildpack has errored | | `22-29` | Detection-specific lifecycle errors | -| `81` | Extension generate error | -| `82-89` | Generation-specific lifecycle errors | +| `91` | Extension generate error | +| `92-99` | Generation-specific lifecycle errors | The lifecycle: - SHALL detect a single group from `` and write it to `` using the [detection process](buildpack.md#phase-1-detection) outlined in the Buildpack Interface Specification From b900760f4db7c816c0e2c616115a72b73bfce21a Mon Sep 17 00:00:00 2001 From: Natalie Arellano Date: Thu, 7 Jul 2022 12:06:09 -0400 Subject: [PATCH 08/13] Update env var name Signed-off-by: Natalie Arellano --- platform.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/platform.md b/platform.md index 7c69c1cd..2e28c845 100644 --- a/platform.md +++ b/platform.md @@ -833,13 +833,13 @@ The lifecycle MUST attempt to authenticate anonymously if no matching credential Where noted, certain features are considered experimental and susceptible to change in a future API version. -The platform SHOULD set `CNB_PLATFORM_EXPERIMENTAL_MODE=` in the lifecycle's execution environment to control the behavior of experimental features. +The platform SHOULD set `CNB_EXPERIMENTAL_MODE=` in the lifecycle's execution environment to control the behavior of experimental features. When an experimental feature is invoked, the lifecycle: -- SHALL warn and continue if `CNB_PLATFORM_EXPERIMENTAL_MODE` is unset -- SHALL warn and continue if `CNB_PLATFORM_EXPERIMENTAL_MODE=warn` -- SHALL fail if `CNB_PLATFORM_EXPERIMENTAL_MODE=error` -- SHALL continue without warning if `CNB_PLATFORM_EXPERIMENTAL_MODE=silent` +- SHALL warn and continue if `CNB_EXPERIMENTAL_MODE` is unset +- SHALL warn and continue if `CNB_EXPERIMENTAL_MODE=warn` +- SHALL fail if `CNB_EXPERIMENTAL_MODE=error` +- SHALL continue without warning if `CNB_EXPERIMENTAL_MODE=silent` ## Buildpacks From 642eaf4ca9a9444de394bd8536278c9d211f7b18 Mon Sep 17 00:00:00 2001 From: Natalie Arellano Date: Tue, 12 Jul 2022 11:57:03 -0400 Subject: [PATCH 09/13] Update platform.md Co-authored-by: Javier Romero Signed-off-by: Natalie Arellano --- platform.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/platform.md b/platform.md index 2e28c845..5ef7d801 100644 --- a/platform.md +++ b/platform.md @@ -382,7 +382,7 @@ Usage: | `` | `CNB_LAYERS_DIR` | `/layers` | Path to layers directory | | `` | `CNB_LOG_LEVEL` | `info` | Log Level | | `` | `CNB_ORDER_PATH` | `/order.toml` if present, or `/cnb/order.toml` | Path resolution for order definition (see [`order.toml`](#ordertoml-toml)) | -| `` | `CNB_OUTPUT_DIR` | ` | (**[experimental](#experimental-features)**) Path to output directory for generated Dockerfiles | +| `` | `CNB_OUTPUT_DIR` | `` | (**[experimental](#experimental-features)**) Path to output directory for generated Dockerfiles | | `` | `CNB_PLAN_PATH` | `/plan.toml` | Path to output resolved build plan | | `` | `CNB_PLATFORM_DIR` | `/platform` | Path to platform directory | From e9b350aa489cbd43ce3129c8fdbde3087df5ec51 Mon Sep 17 00:00:00 2001 From: Natalie Arellano Date: Tue, 12 Jul 2022 11:57:18 -0400 Subject: [PATCH 10/13] Update platform.md Co-authored-by: Javier Romero Signed-off-by: Natalie Arellano --- platform.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/platform.md b/platform.md index 5ef7d801..c9ff7cd7 100644 --- a/platform.md +++ b/platform.md @@ -395,7 +395,7 @@ Usage: | `` | Detected buildpack group (see [`group.toml`](#grouptoml-toml)) | | `` | Resolved Build Plan (see [`plan.toml`](#plantoml-toml)) | | `` | Updated to include the run image obtained from applying generated Dockerfiles | -| `/generated/run//Dockerfile` | Generated Dockerfiles (see [Image Extension Specfication](image-extension.md) | +| `/generated/run//Dockerfile` | Generated Dockerfiles (see [Image Extension Specfication](image-extension.md)) | | Exit Code | Result | |-----------------|-----------------------------------------------------------------------------------| From be44080aeed1f8907b720796dd87a35a97e06abf Mon Sep 17 00:00:00 2001 From: Natalie Arellano Date: Tue, 12 Jul 2022 11:57:48 -0400 Subject: [PATCH 11/13] Update platform.md Co-authored-by: Javier Romero Signed-off-by: Natalie Arellano --- platform.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/platform.md b/platform.md index c9ff7cd7..9ccf7723 100644 --- a/platform.md +++ b/platform.md @@ -1026,8 +1026,8 @@ Where: [[entries]] [[entries.providers]] - id = "" - version = "buildpack Version" + id = "" + version = "" extension = false [[entries.requires]] From e8dda220d4412b84feb45afabf861b1bab48eab1 Mon Sep 17 00:00:00 2001 From: Natalie Arellano Date: Tue, 12 Jul 2022 12:06:20 -0400 Subject: [PATCH 12/13] Clarify the meaning of `extension` in plan.toml Signed-off-by: Natalie Arellano --- platform.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/platform.md b/platform.md index 9ccf7723..051f9eca 100644 --- a/platform.md +++ b/platform.md @@ -1019,15 +1019,15 @@ version = "" Where: - Both `id` and `version` MUST be present for each buildpack object in a group. -- The value of `optional` MUST default to false if not specified. +- The value of `optional` MUST default to `false` if not specified. #### `plan.toml` (TOML) ```toml [[entries]] [[entries.providers]] - id = "" - version = "" + id = "" + version = "" extension = false [[entries.requires]] @@ -1038,7 +1038,8 @@ Where: Where: - `entries` MAY be empty - Each entry: - - MUST contain at least one buildpack in `providers` + - MUST contain at least one buildpack or image extension in `providers` + - If the provider is an image extension (**[experimental](#experimental-features)**), `extension` MUST be `true`; the value of `extension` MUST default to `false` if not specified - MUST contain at least one dependency requirement in `requires` - MUST exclusively contain dependency requirements with the same `` From a5acc10fba44ce3b1022438b6a591aaf0b6ccfff Mon Sep 17 00:00:00 2001 From: Natalie Arellano Date: Tue, 16 Aug 2022 09:51:49 -0400 Subject: [PATCH 13/13] Update platform.md Signed-off-by: Natalie Arellano --- platform.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/platform.md b/platform.md index 051f9eca..2bd0bfba 100644 --- a/platform.md +++ b/platform.md @@ -836,7 +836,7 @@ Where noted, certain features are considered experimental and susceptible to cha The platform SHOULD set `CNB_EXPERIMENTAL_MODE=` in the lifecycle's execution environment to control the behavior of experimental features. When an experimental feature is invoked, the lifecycle: -- SHALL warn and continue if `CNB_EXPERIMENTAL_MODE` is unset +- SHALL fail if `CNB_EXPERIMENTAL_MODE` is unset - SHALL warn and continue if `CNB_EXPERIMENTAL_MODE=warn` - SHALL fail if `CNB_EXPERIMENTAL_MODE=error` - SHALL continue without warning if `CNB_EXPERIMENTAL_MODE=silent`