Skip to content

chore(ci): use 15 percent impact threshold for SLOs#7930

Merged
BridgeAR merged 3 commits intomasterfrom
fayssal/slos-update-to-15
Apr 7, 2026
Merged

chore(ci): use 15 percent impact threshold for SLOs#7930
BridgeAR merged 3 commits intomasterfrom
fayssal/slos-update-to-15

Conversation

@faydef
Copy link
Copy Markdown
Contributor

@faydef faydef commented Apr 7, 2026

What does this PR do?

Loosens auto-generated SLO thresholds by increasing the significant impact threshold (T) from 5% to 15%.

  • For smaller-is-better metrics (e.g., latency): threshold = CI_upper / (1 - 0.15)
  • For larger-is-better metrics (e.g., throughput): threshold = CI_lower / (1 + 0.15)

Motivation

Benchmarks were producing false positive SLO breaches due to cross-run variability. The 5% margin above the CI bound was too tight, causing noise between runs to trigger failures. A 15% margin provides enough headroom to absorb this variability while still catching meaningful regressions.

Additional Notes

Only thresholds that were brought up in the OE report were changed.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 7, 2026

Overall package size

Self size: 5.47 MB
Deduped: 6.32 MB
No deduping: 6.32 MB

Dependency sizes | name | version | self size | total size | |------|---------|-----------|------------| | import-in-the-middle | 3.0.0 | 81.15 kB | 815.98 kB | | dc-polyfill | 0.1.10 | 26.73 kB | 26.73 kB |

🤖 This report was automatically generated by heaviest-objects-in-the-universe

@codecov
Copy link
Copy Markdown

codecov bot commented Apr 7, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 74.26%. Comparing base (a3baec5) to head (1edf3b7).
⚠️ Report is 2 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #7930   +/-   ##
=======================================
  Coverage   74.26%   74.26%           
=======================================
  Files         769      769           
  Lines       36063    36063           
=======================================
  Hits        26782    26782           
  Misses       9281     9281           
Flag Coverage Δ
aiguard-macos 39.44% <ø> (-0.10%) ⬇️
aiguard-ubuntu 39.56% <ø> (-0.10%) ⬇️
aiguard-windows 39.23% <ø> (-0.10%) ⬇️
apm-capabilities-tracing-macos 49.56% <ø> (ø)
apm-capabilities-tracing-ubuntu 49.47% <ø> (ø)
apm-capabilities-tracing-windows 49.34% <ø> (ø)
apm-integrations-child-process 38.74% <ø> (-0.13%) ⬇️
apm-integrations-couchbase-18 37.52% <ø> (-0.10%) ⬇️
apm-integrations-couchbase-eol 38.01% <ø> (-0.10%) ⬇️
apm-integrations-oracledb 37.87% <ø> (-0.10%) ⬇️
appsec-express 55.38% <ø> (-0.07%) ⬇️
appsec-fastify 51.71% <ø> (-0.07%) ⬇️
appsec-graphql 51.87% <ø> (-0.06%) ⬇️
appsec-kafka 44.49% <ø> (-0.09%) ⬇️
appsec-ldapjs 44.11% <ø> (-0.08%) ⬇️
appsec-lodash 43.71% <ø> (-0.08%) ⬇️
appsec-macos 58.11% <ø> (-0.07%) ⬇️
appsec-mongodb-core 48.89% <ø> (-0.08%) ⬇️
appsec-mongoose 49.55% <ø> (-0.08%) ⬇️
appsec-mysql 51.08% <ø> (-0.07%) ⬇️
appsec-node-serialize 43.29% <ø> (-0.08%) ⬇️
appsec-passport 47.76% <ø> (-0.09%) ⬇️
appsec-postgres 50.80% <ø> (+0.03%) ⬆️
appsec-sourcing 42.54% <ø> (-0.08%) ⬇️
appsec-stripe 44.73% <ø> (-0.09%) ⬇️
appsec-template 43.45% <ø> (-0.08%) ⬇️
appsec-ubuntu 58.19% <ø> (-0.07%) ⬇️
appsec-windows 57.91% <ø> (-0.09%) ⬇️
instrumentations-instrumentation-bluebird 32.33% <ø> (-0.10%) ⬇️
instrumentations-instrumentation-body-parser 40.63% <ø> (-0.09%) ⬇️
instrumentations-instrumentation-child_process 38.07% <ø> (-0.10%) ⬇️
instrumentations-instrumentation-cookie-parser 34.36% <ø> (-0.09%) ⬇️
instrumentations-instrumentation-express 34.67% <ø> (-0.09%) ⬇️
instrumentations-instrumentation-express-mongo-sanitize 34.49% <ø> (-0.09%) ⬇️
instrumentations-instrumentation-express-session 40.27% <ø> (-0.09%) ⬇️
instrumentations-instrumentation-fs 32.01% <ø> (-0.10%) ⬇️
instrumentations-instrumentation-generic-pool 29.44% <ø> (ø)
instrumentations-instrumentation-http 39.99% <ø> (-0.10%) ⬇️
instrumentations-instrumentation-knex 32.39% <ø> (-0.10%) ⬇️
instrumentations-instrumentation-mongoose 33.51% <ø> (-0.10%) ⬇️
instrumentations-instrumentation-multer 40.38% <ø> (-0.09%) ⬇️
instrumentations-instrumentation-mysql2 38.40% <ø> (-0.10%) ⬇️
instrumentations-instrumentation-passport 44.16% <ø> (-0.09%) ⬇️
instrumentations-instrumentation-passport-http 43.84% <ø> (-0.09%) ⬇️
instrumentations-instrumentation-passport-local 44.37% <ø> (-0.09%) ⬇️
instrumentations-instrumentation-pg 37.84% <ø> (-0.10%) ⬇️
instrumentations-instrumentation-promise 32.26% <ø> (-0.11%) ⬇️
instrumentations-instrumentation-promise-js 32.26% <ø> (-0.10%) ⬇️
instrumentations-instrumentation-q 32.31% <ø> (-0.10%) ⬇️
instrumentations-instrumentation-url 32.23% <ø> (-0.11%) ⬇️
instrumentations-instrumentation-when 32.28% <ø> (-0.11%) ⬇️
llmobs-ai 41.37% <ø> (-0.10%) ⬇️
llmobs-anthropic 40.84% <ø> (-0.09%) ⬇️
llmobs-bedrock 39.32% <ø> (-0.08%) ⬇️
llmobs-google-genai 39.87% <ø> (-0.09%) ⬇️
llmobs-langchain 39.45% <ø> (-0.08%) ⬇️
llmobs-openai 44.12% <ø> (-0.09%) ⬇️
llmobs-vertex-ai 40.13% <ø> (-0.09%) ⬇️
platform-core 31.47% <ø> (ø)
platform-esbuild 34.42% <ø> (ø)
platform-instrumentations-misc 34.11% <ø> (ø)
platform-shimmer 37.56% <ø> (ø)
platform-unit-guardrails 32.89% <ø> (ø)
platform-webpack 19.86% <ø> (ø)
plugins-azure-durable-functions 25.74% <ø> (ø)
plugins-azure-event-hubs 25.90% <ø> (ø)
plugins-azure-service-bus 25.26% <ø> (ø)
plugins-bullmq 43.60% <ø> (+0.02%) ⬆️
plugins-cassandra 38.02% <ø> (-0.10%) ⬇️
plugins-cookie 26.96% <ø> (ø)
plugins-cookie-parser 26.75% <ø> (ø)
plugins-crypto 26.73% <ø> (ø)
plugins-dd-trace-api 38.43% <ø> (-0.10%) ⬇️
plugins-express-mongo-sanitize 26.89% <ø> (ø)
plugins-express-session 26.70% <ø> (ø)
plugins-fastify 42.36% <ø> (-0.09%) ⬇️
plugins-fetch 38.51% <ø> (-0.10%) ⬇️
plugins-fs 38.75% <ø> (-0.10%) ⬇️
plugins-generic-pool 25.94% <ø> (ø)
plugins-google-cloud-pubsub 45.68% <ø> (-0.09%) ⬇️
plugins-grpc 41.01% <ø> (-0.09%) ⬇️
plugins-handlebars 26.94% <ø> (ø)
plugins-hapi 40.27% <ø> (-0.10%) ⬇️
plugins-hono 40.61% <ø> (-0.10%) ⬇️
plugins-ioredis 38.60% <ø> (-0.10%) ⬇️
plugins-knex 26.57% <ø> (ø)
plugins-langgraph 37.99% <ø> (-0.10%) ⬇️
plugins-ldapjs 24.43% <ø> (ø)
plugins-light-my-request 26.30% <ø> (ø)
plugins-limitd-client 32.61% <ø> (-0.10%) ⬇️
plugins-lodash 26.03% <ø> (ø)
plugins-mariadb 39.61% <ø> (-0.15%) ⬇️
plugins-memcached 38.34% <ø> (-0.10%) ⬇️
plugins-microgateway-core 39.34% <ø> (-0.17%) ⬇️
plugins-moleculer 40.63% <ø> (-0.09%) ⬇️
plugins-mongodb 39.27% <ø> (-0.10%) ⬇️
plugins-mongodb-core 39.14% <ø> (-0.10%) ⬇️
plugins-mongoose 38.92% <ø> (-0.10%) ⬇️
plugins-multer 26.70% <ø> (ø)
plugins-mysql 39.45% <ø> (-0.10%) ⬇️
plugins-mysql2 39.41% <ø> (-0.10%) ⬇️
plugins-node-serialize 27.00% <ø> (ø)
plugins-opensearch 37.74% <ø> (-0.10%) ⬇️
plugins-passport-http 26.76% <ø> (ø)
plugins-postgres 35.56% <ø> (-0.11%) ⬇️
plugins-process 26.73% <ø> (ø)
plugins-pug 26.96% <ø> (ø)
plugins-redis 39.04% <ø> (-0.10%) ⬇️
plugins-router 43.36% <ø> (-0.10%) ⬇️
plugins-sequelize 25.55% <ø> (ø)
plugins-test-and-upstream-amqp10 38.62% <ø> (-0.10%) ⬇️
plugins-test-and-upstream-amqplib 44.37% <ø> (-0.10%) ⬇️
plugins-test-and-upstream-apollo 39.24% <ø> (-0.09%) ⬇️
plugins-test-and-upstream-avsc 38.69% <ø> (-0.10%) ⬇️
plugins-test-and-upstream-bunyan 33.94% <ø> (-0.10%) ⬇️
plugins-test-and-upstream-connect 40.93% <ø> (-0.10%) ⬇️
plugins-test-and-upstream-graphql 40.28% <ø> (-0.10%) ⬇️
plugins-test-and-upstream-koa 40.52% <ø> (-0.10%) ⬇️
plugins-test-and-upstream-protobufjs 38.92% <ø> (-0.10%) ⬇️
plugins-test-and-upstream-rhea 44.37% <ø> (-0.10%) ⬇️
plugins-undici 39.36% <ø> (-0.09%) ⬇️
plugins-url 26.73% <ø> (ø)
plugins-valkey 38.31% <ø> (-0.09%) ⬇️
plugins-vm 26.73% <ø> (ø)
plugins-winston 34.12% <ø> (-0.24%) ⬇️
plugins-ws 42.12% <ø> (-0.10%) ⬇️
profiling-macos 40.65% <ø> (-0.10%) ⬇️
profiling-ubuntu 40.78% <ø> (-0.10%) ⬇️
profiling-windows 42.30% <ø> (+0.32%) ⬆️
serverless-azure-functions-client 25.62% <ø> (ø)
serverless-azure-functions-eventhubs 25.62% <ø> (ø)
serverless-azure-functions-servicebus 25.62% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@BridgeAR BridgeAR enabled auto-merge (squash) April 7, 2026 11:46
@datadog-prod-us1-3
Copy link
Copy Markdown

datadog-prod-us1-3 bot commented Apr 7, 2026

✅ Tests

🎉 All green!

❄️ No new flaky tests detected
🧪 All tests passed

🎯 Code Coverage (details)
Patch Coverage: 100.00%
Overall Coverage: 68.70% (+0.00%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 1edf3b7 | Docs | Datadog PR Page | Was this helpful? React with 👍/👎 or give us feedback!

@pr-commenter
Copy link
Copy Markdown

pr-commenter bot commented Apr 7, 2026

Benchmarks

Benchmark execution time: 2026-04-07 19:11:21

Comparing candidate commit 1edf3b7 in PR branch fayssal/slos-update-to-15 with baseline commit a3baec5 in branch master.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 229 metrics, 31 unstable metrics.

@faydef faydef changed the title use 15 percent chore(ci): use 15 percent impact threshold for SLOs Apr 7, 2026
@BridgeAR BridgeAR merged commit efc9681 into master Apr 7, 2026
992 of 998 checks passed
@BridgeAR BridgeAR deleted the fayssal/slos-update-to-15 branch April 7, 2026 19:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants