Skip to content

misc cleanup of dual circuit breaker (Dec 19)#947

Draft
adriangudas wants to merge 137 commits intomainfrom
adriangudas/adaptive-cleanup-misc-dec19
Draft

misc cleanup of dual circuit breaker (Dec 19)#947
adriangudas wants to merge 137 commits intomainfrom
adriangudas/adaptive-cleanup-misc-dec19

Conversation

@adriangudas
Copy link
Contributor

A few minor cleanup/fixes:

  • Fix DRY in the child circuit breaker class implementation by using a common module
  • Add an expectation for mark_success to improve classic test coverage
  • Use a setter method style for adaptive_circuit_breaker_selector
  • Use a class instead of a method for the ExperimentFlags in the demo

Note: This PR introduces a stylistic change to make adaptive_circuit_breaker_selector to be a setter method, so we'll have to change the way we call it from

Semian::DualCircuitBreaker.adaptive_circuit_breaker_selector(value)

to

Semian::DualCircuitBreaker.adaptive_circuit_breaker_selector = value

..and make sure that is effectively socialized with our team before merging.

AbdulRahmanAlHamali and others added 30 commits October 22, 2025 17:08
Update variable names

Fill sliding window with 1 hr worth data

Add comment

Update experiemnt resource to be deterministic

Change deterministic default value to false

Cleanup

Remove unused variable

Make initial seed error rate more customizable

Add seed_error_rate as a property
* Prefilling added

* Change initial duration to 900 s
* testing different circuit breaking scenarios

* adding concurrency

* adds more puts to get further information during phases

* Fixing concurrency, unprotected ping, extras

* update classic sustained test

* cleaning up outdated tests, and testing without ping rate

* modify ki instead of dividing by window size

---------

Co-authored-by: Abdulrahman Alhamali <abdulrahman.alhamali@shopify.com>
kris-gaudel and others added 22 commits November 20, 2025 16:24
)

* Replace p2 estimator with SES

* Fix PID tests to use new smoother

* Remove p2

* update images

* Correct comment

* update images

* Clean up comments, address Abed's concerns

* Update comment

* Update images

* run all experiments

* Unit tests for success criteria

* Add incident detection and adaptive convergence

* Update success criteria version 2

* Address Abed's comments

* Update experiments Gemfile to prevent error

* Update experiments

* Address Abed comments again

* Remove init

* Experiments

---------

Co-authored-by: Abdulrahman Alhamali <abdulrahman.alhamali@shopify.com>
* switch algorithm to a sliding window instead of discrete

* clean up AI slop PR

* provide observations per minute to the smoother from the pid controller

* fix tests

* remove unused smoother in setup

* fix alpha value

* fix tests and run experiments

* fix max size for sliding window and rerun experiments
Remove unnecessary comments and fix smoother initalization
* add elastic defensiveness

* remove kd

* update docs and run experiments

* fix tests

---------

Co-authored-by: Fernando Aguasvivas <fernando.aguasvivas@shopify.com>
…d classic modes (#808)

* use lambda to implement dual circuit breaker

* fix tests

* class method to allow setting a global selector for adaptive config

* remove unneeded methods that are no longer used
use CircuitBreakerBehaviour

* allow adaptive_circuit_breaker_selector to be unset

* make dynamic configuration call only on acquire

* require resource to be provided

* fix: avoid an extra lookup to use_adaptive in metrics

* various fixes

* nit: cleanup

* fix logging to both circuit breakers at the same time

* use logic from both acquire methods in dual_circuit_breaker

* fix last error tracking test by using properly scoped exceptions variable

* fix demos

* remove unused attr_reader for name
use symbols for active_breaker_type instead of strings
create separate acquire methods for readability
use active_breaker_type consistently to determine breaker type
remove useless AI-generated tests
fix useful tests

* nit: typo fix

* nit: remove TODO comment

* don't return if variables are nil (dangerous)
use_adaptive proc no longer exists at initialization (proc is set at runtime)

* address PR comments

* make active_circuit_breaker readable for tests
clean up unneeded tests and AI fluff

* improve track both breakers test with assert_equal

* replace "legacy" with "classic", notify on change in cb type

* replace legacy with classic

* notify on change in cb type, remove verbose comments, and refactor `handle_adaptive_acquire`

* metrics functions for dcb example

* have error handling parity between classic and adaptive

* fix exceptions parameter passing

* reference as instance variable

* use acquire directly from circuit breakers

* fix dcb test and remove verbose content

* fix double counting of errors

* undo the move of method to public

* make active_breaker_type public

* rerun experiments

---------

Co-authored-by: Kristofer Gaudel <kris.gaudel@shopify.com>
Co-authored-by: Abdulrahman Alhamali <abdulrahman.alhamali@shopify.com>
Add an expectation for mark_success to improve classic test coverage
Use a setter method style for adaptive_circuit_breaker_selector
Use a class instead of a method for the ExperimentFlags in the demo
end

def self.adaptive_circuit_breaker_selector(selector) # rubocop:disable Style/ClassMethodsDefinitions
def self.adaptive_circuit_breaker_selector=(selector) # rubocop:disable Style/ClassMethodsDefinitions
Copy link
Contributor Author

@adriangudas adriangudas Dec 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe this doesn't have to change. Or I can at least add the old method side-by-side so there's backwards compatibility.

@adriangudas
Copy link
Contributor Author

Actually, thinking of removing the code updates here (maybe the experiment / test updates are ok). I'm realizing now that we're better off considering our code frozen since it's in working state.

@adriangudas adriangudas marked this pull request as draft December 21, 2025 18:41
Base automatically changed from pid-take-2 to main March 19, 2026 15:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants