Skip to content

feat: add Azure Policy builtins with YAML test suite#630

Merged
anakrish merged 6 commits intomicrosoft:mainfrom
anakrish:azure-policy-builtins
Mar 25, 2026
Merged

feat: add Azure Policy builtins with YAML test suite#630
anakrish merged 6 commits intomicrosoft:mainfrom
anakrish:azure-policy-builtins

Conversation

@anakrish
Copy link
Copy Markdown
Collaborator

Implement ARM template functions for Azure Policy evaluation:

Builtins:

  • String: indexOf, lastIndexOf, trim, format, split, startsWith, endsWith, padLeft, concat, replace, toLower, toUpper, substring, guid, uniqueString
  • DateTime: dateTimeAdd, dateTimeFromEpoch, dateTimeToEpoch, addDays
  • Collection: intersection, union, take, skip, first, last, min, max, range, items, tryGet, tryIndexFromEnd, empty, array, createObject
  • Encoding: base64, base64ToString, base64ToJson, uri, uriComponent, uriComponentToString, dataUri, dataUriToString
  • Numeric: int, float, intDiv, intMod
  • Misc: json, join, bool, string, coalesce, if, getParameter, resolveField
  • Logic: logicAll, logicAny

Key implementation details:

  • Unicode case-insensitive search via ICU4X case folding with single-pass fold_with_char_map() for indexOf/lastIndexOf
  • .NET composite formatting (System.String.Format) with alignment, standard and custom datetime format specifiers, numeric format specifiers
  • DateTime round-trip preserves input shape (Z vs +00:00, T vs space, fractional seconds) when no explicit output format is supplied
  • Zero-cost as_str() helper borrows directly from Value::String(Rc)
  • BTreeSet<&Value> in array union avoids redundant cloning

Test suite:

  • 53 YAML test files exercising all builtins via direct BUILTINS registry
  • Coverage for edge cases: empty inputs, Unicode, fractional seconds, invalid alignment, unknown format specifiers, RFC3339 offset shapes

Implement ARM template functions for Azure Policy evaluation:

Builtins:
- String: indexOf, lastIndexOf, trim, format, split, startsWith, endsWith,
  padLeft, concat, replace, toLower, toUpper, substring, guid, uniqueString
- DateTime: dateTimeAdd, dateTimeFromEpoch, dateTimeToEpoch, addDays
- Collection: intersection, union, take, skip, first, last, min, max,
  range, items, tryGet, tryIndexFromEnd, empty, array, createObject
- Encoding: base64, base64ToString, base64ToJson, uri, uriComponent,
  uriComponentToString, dataUri, dataUriToString
- Numeric: int, float, intDiv, intMod
- Misc: json, join, bool, string, coalesce, if, getParameter, resolveField
- Logic: logicAll, logicAny

Key implementation details:
- Unicode case-insensitive search via ICU4X case folding with single-pass
  fold_with_char_map() for indexOf/lastIndexOf
- .NET composite formatting (System.String.Format) with alignment, standard
  and custom datetime format specifiers, numeric format specifiers
- DateTime round-trip preserves input shape (Z vs +00:00, T vs space,
  fractional seconds) when no explicit output format is supplied
- Zero-cost as_str() helper borrows directly from Value::String(Rc<str>)
- BTreeSet<&Value> in array union avoids redundant cloning

Test suite:
- 53 YAML test files exercising all builtins via direct BUILTINS registry
- Coverage for edge cases: empty inputs, Unicode, fractional seconds,
  invalid alignment, unknown format specifiers, RFC3339 offset shapes

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
@anakrish anakrish requested a review from Copilot March 25, 2026 16:27
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds Azure Policy / ARM-template builtin function support behind the azure_policy feature and introduces a YAML-driven test harness to validate builtin behavior directly via the BUILTINS registry.

Changes:

  • Introduces azure_policy language module and string comparison utilities (ASCII key compare + ICU4X Unicode case folding).
  • Registers a broad set of Azure Policy / ARM template builtins (string, numeric, datetime, encoding, collection, misc, and logic helpers).
  • Adds a YAML-based test runner plus many YAML case files to exercise builtin semantics and edge cases.

Reviewed changes

Copilot reviewed 73 out of 74 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
tests/mod.rs Enables the Azure Policy YAML test module behind azure_policy feature.
tests/azure_policy_builtins/mod.rs Adds YAML test runner that invokes builtins from the registry.
tests/azure_policy_builtins/cases/add_days.yaml Adds YAML cases for add_days.
tests/azure_policy_builtins/cases/array.yaml Adds YAML cases for array.
tests/azure_policy_builtins/cases/base64.yaml Adds YAML cases for base64.
tests/azure_policy_builtins/cases/base64_to_json.yaml Adds YAML cases for base64_to_json.
tests/azure_policy_builtins/cases/base64_to_string.yaml Adds YAML cases for base64_to_string.
tests/azure_policy_builtins/cases/bool.yaml Adds YAML cases for bool.
tests/azure_policy_builtins/cases/coalesce.yaml Adds YAML cases for coalesce.
tests/azure_policy_builtins/cases/create_object.yaml Adds YAML cases for create_object.
tests/azure_policy_builtins/cases/data_uri.yaml Adds YAML cases for data_uri.
tests/azure_policy_builtins/cases/data_uri_to_string.yaml Adds YAML cases for data_uri_to_string.
tests/azure_policy_builtins/cases/date_time_add.yaml Adds YAML cases for date_time_add (formatting + round-trip shape).
tests/azure_policy_builtins/cases/date_time_from_epoch.yaml Adds YAML cases for date_time_from_epoch.
tests/azure_policy_builtins/cases/date_time_to_epoch.yaml Adds YAML cases for date_time_to_epoch.
tests/azure_policy_builtins/cases/empty.yaml Adds YAML cases for empty.
tests/azure_policy_builtins/cases/ends_with.yaml Adds YAML cases for ends_with.
tests/azure_policy_builtins/cases/first.yaml Adds YAML cases for first.
tests/azure_policy_builtins/cases/float.yaml Adds YAML cases for float.
tests/azure_policy_builtins/cases/format.yaml Adds YAML cases for format.
tests/azure_policy_builtins/cases/get_parameter.yaml Adds YAML cases for get_parameter.
tests/azure_policy_builtins/cases/guid.yaml Adds YAML cases for guid.
tests/azure_policy_builtins/cases/if.yaml Adds YAML cases for if.
tests/azure_policy_builtins/cases/index_from_end.yaml Adds YAML cases for index_from_end.
tests/azure_policy_builtins/cases/index_of.yaml Adds YAML cases for index_of.
tests/azure_policy_builtins/cases/int.yaml Adds YAML cases for int.
tests/azure_policy_builtins/cases/int_div.yaml Adds YAML cases for int_div.
tests/azure_policy_builtins/cases/int_mod.yaml Adds YAML cases for int_mod.
tests/azure_policy_builtins/cases/intersection.yaml Adds YAML cases for intersection.
tests/azure_policy_builtins/cases/ip_range_contains.yaml Adds YAML cases for ip_range_contains.
tests/azure_policy_builtins/cases/items.yaml Adds YAML cases for items.
tests/azure_policy_builtins/cases/join.yaml Adds YAML cases for join.
tests/azure_policy_builtins/cases/json.yaml Adds YAML cases for json.
tests/azure_policy_builtins/cases/last.yaml Adds YAML cases for last.
tests/azure_policy_builtins/cases/last_index_of.yaml Adds YAML cases for last_index_of.
tests/azure_policy_builtins/cases/logic_all.yaml Adds YAML cases for logic_all.
tests/azure_policy_builtins/cases/logic_any.yaml Adds YAML cases for logic_any.
tests/azure_policy_builtins/cases/max.yaml Adds YAML cases for max.
tests/azure_policy_builtins/cases/min.yaml Adds YAML cases for min.
tests/azure_policy_builtins/cases/pad_left.yaml Adds YAML cases for pad_left.
tests/azure_policy_builtins/cases/range.yaml Adds YAML cases for range.
tests/azure_policy_builtins/cases/resolve_field.yaml Adds YAML cases for resolve_field.
tests/azure_policy_builtins/cases/skip.yaml Adds YAML cases for skip.
tests/azure_policy_builtins/cases/split.yaml Adds YAML cases for split.
tests/azure_policy_builtins/cases/starts_with.yaml Adds YAML cases for starts_with.
tests/azure_policy_builtins/cases/string.yaml Adds YAML cases for string.
tests/azure_policy_builtins/cases/take.yaml Adds YAML cases for take.
tests/azure_policy_builtins/cases/trim.yaml Adds YAML cases for trim.
tests/azure_policy_builtins/cases/try_get.yaml Adds YAML cases for try_get.
tests/azure_policy_builtins/cases/try_index_from_end.yaml Adds YAML cases for try_index_from_end.
tests/azure_policy_builtins/cases/union.yaml Adds YAML cases for union.
tests/azure_policy_builtins/cases/unique_string.yaml Adds YAML cases for unique_string.
tests/azure_policy_builtins/cases/uri.yaml Adds YAML cases for uri.
tests/azure_policy_builtins/cases/uri_component.yaml Adds YAML cases for uri_component.
tests/azure_policy_builtins/cases/uri_component_to_string.yaml Adds YAML cases for uri_component_to_string.
src/lib.rs Exposes languages::azure_policy (feature gated) and re-exports builtins via unstable.
src/languages/mod.rs Adds the azure_policy language module (feature gated).
src/languages/azure_policy/mod.rs Introduces Azure Policy language root module.
src/languages/azure_policy/strings/mod.rs Adds Azure Policy string semantics module (keys vs full Unicode folding).
src/languages/azure_policy/strings/keys.rs Implements ASCII-only key equality/ordering for ARM property keys.
src/languages/azure_policy/strings/case_fold.rs Implements Unicode case folding with ICU4X for policy string comparisons.
src/builtins/mod.rs Registers Azure Policy builtins behind azure_policy feature.
src/builtins/azure_policy/mod.rs Adds Azure Policy builtin module and registration entry point.
src/builtins/azure_policy/helpers.rs Adds shared helpers for coercion and field/path resolution.
src/builtins/azure_policy/operators.rs Adds policy helpers: get_parameter, resolve_field, logic_all/any, if.
src/builtins/azure_policy/template_functions.rs Adds core ARM template-style functions (split/empty/first/last/etc.).
src/builtins/azure_policy/template_functions_collection.rs Adds collection builtins (intersection/union/take/skip/range/etc.).
src/builtins/azure_policy/template_functions_datetime.rs Adds datetime parsing/addition/formatting builtins.
src/builtins/azure_policy/template_functions_encoding.rs Adds encoding + URI/dataUri builtins with internal implementations.
src/builtins/azure_policy/template_functions_misc.rs Adds misc builtins (json/join/items/indexFromEnd/tryGet/guid/uniqueString).
src/builtins/azure_policy/template_functions_numeric.rs Adds numeric builtins (min/max/float/int_div/int_mod).
src/builtins/azure_policy/template_functions_string.rs Adds string builtins (indexOf/lastIndexOf/trim/format).
Cargo.toml Expands azure_policy feature dependencies (chrono/ipnet/icu_casemap).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

- Fix percent_encode to only uppercase hex digits, not entire string
- Remove guid/uniqueString (unsupported); delete custom SHA-1 impl
- Replace unwrap_or(0) with proper error in format placeholder parsing
- Hoist CaseMapper into static CaseMapperBorrowed for zero per-call overhead
- Pre-allocate Vec in range() with_capacity
- Update bindings/ffi and bindings/ruby Cargo.lock
- Fix uri_component test expectations for correct case preservation

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
@anakrish anakrish requested a review from Copilot March 25, 2026 18:26
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 71 out of 74 changed files in this pull request and generated 4 comments.

Comments suppressed due to low confidence (1)

tests/azure_policy_builtins/mod.rs:1

  • The comment says Undefined should be treated as acceptable when an error is expected, but the code immediately fails the test via bail!. Please either remove/adjust the comment to match the behavior, or change the behavior to align with the comment (e.g. allow Undefined when want_error is set, if that’s genuinely intended).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

- float(): return Undefined when as_f64() fails instead of leaking
  the original non-f64 representation
- createObject(): reject odd number of arguments with an error
  (ARM-template parity)
- format(): error on unknown numeric format specifiers instead of
  silently passing through (matches .NET FormatException behavior)
- format(): cap alignment width at 10,000 to prevent DoS from
  user-controlled format strings like {0,1000000000}
- Add YAML test cases for all new error behaviors

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 71 out of 74 changed files in this pull request and generated 4 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

- percent_decode: reject incomplete % escapes (e.g. "%", "%2") instead
  of treating them as literal characters
- parse_iso8601_duration: reject leftover digits without a unit designator
  at T boundary and end-of-input (e.g. "P1", "P1T2H")
- yaml_to_value: panic on unsupported YAML numeric representations instead
  of silently mapping to Null
- Revert unused src/languages/mod.rs changes (module is defined inline in
  lib.rs)

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
- fn_split: return input as single-element array for empty string
  delimiter instead of panicking (Rust's str::split("") panics)
- format: add test for F3 higher precision ({0:F3} + 1.23456 → 1.235)
- format: add test for N2 float with thousands separator
- format: add test for negative index error ({-1})
- split: add test for empty-string delimiter
- uri: add tests for query string and fragment in relative URI
- createObject: add test for non-string (numeric) keys

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 70 out of 73 changed files in this pull request and generated 7 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

- Add MAX_VARIADIC_ARGS (64) constant for variadic builtin arity
  instead of registering with 0 (logic_all, logic_any, min, max,
  format, intersection, union, coalesce, createObject); set
  dateTimeAdd to exact arity 3

- Switch indexOf/lastIndexOf to UTF-16 code-unit indices to match
  .NET String.IndexOf semantics (track ch.len_utf16() in
  fold_with_char_map, use encode_utf16().count() for empty-needle
  lastIndexOf)

- Use DateTime::<Utc>::from_timestamp for explicit timezone type

- Remove stale docs/azure-policy/casing.md link from module doc

- Fix misleading comment in want_error test branch (code bails on
  Undefined, not accepts it)

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 70 out of 73 changed files in this pull request and generated 1 comment.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@anakrish anakrish marked this pull request as ready for review March 25, 2026 21:46
@anakrish anakrish merged commit 5b60daa into microsoft:main Mar 25, 2026
58 checks passed
@anakrish anakrish deleted the azure-policy-builtins branch March 25, 2026 22:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants