feat: support scaling direction-aware cooldown for task auto-scalers#19286
Draft
jtuglu1 wants to merge 1 commit intoapache:masterfrom
Draft
feat: support scaling direction-aware cooldown for task auto-scalers#19286jtuglu1 wants to merge 1 commit intoapache:masterfrom
jtuglu1 wants to merge 1 commit intoapache:masterfrom
Conversation
c72942f to
fc9c5cb
Compare
52e327f to
164f891
Compare
164f891 to
7fd9d61
Compare
...va/org/apache/druid/indexing/seekablestream/supervisor/SeekableStreamSupervisorSpecTest.java
Dismissed
Show dismissed
Hide dismissed
6fe7359 to
ad264b0
Compare
ad264b0 to
757cbde
Compare
757cbde to
96f9f68
Compare
96f9f68 to
8f4bfc5
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adds support for configuring different cooldowns for scaling direction. While both scaling actions do cause temporary disruption to ingestion, scaling down can cause more disruption than scaling up due to having less resources than when you started to recover from lag. Therefore, to allow for aggressive scale up while having a more conservative scale-down approach, this adds configuration for cool down period for both directions. Cool down for a specific scaling direction is evaluated as follows:
minTriggerScaleUpActionFrequencyMillis/minTriggerScaleDownActionFrequencyMillisminTriggerScaleActionFrequencyMillis600000.This also does the following:
ScaleActionSupplierwhich makes the contract clear for auto-scalers who want to implement it.Release note
Adds support for configuring different cooldowns for scaling direction for streaming task auto-scalers.
This PR has: