feat: support user identity forwarding to tasks via TaskAuthContext by jtuglu1 · Pull Request #19236 · apache/druid

jtuglu1 · 2026-03-30T23:30:20Z

Description

Create a mechanism to propagate user identity context from authenticator to tasks. This allows tasks to safely operate with a sort of per-task identity, allowing tasks to access information dynamically that would be hard to setup in the current state of things. This allows for auth mechanisms like catalog credential vending (through Iceberg Catalog, etc.) and allows Druid to operate more like a standalone engine that can integrate with other things within a larger data ecosystem.

Design

Constraints

This currently does NOT support task restarts.

Release note

Create a mechanism to propagate user identity context from authenticator to tasks to support things like credential vending, etc.

This PR has:

indexing-service/src/main/java/org/apache/druid/indexing/overlord/http/OverlordResource.java

...ib/druid-iceberg-extensions/src/main/java/org/apache/druid/iceberg/input/IcebergCatalog.java

...ruid-iceberg-extensions/src/main/java/org/apache/druid/iceberg/input/IcebergInputSource.java

...ing-service/src/main/java/org/apache/druid/indexing/common/task/TaskAuthContextProvider.java

+   * @return TaskAuthContext to inject into the task, or null to skip injection
+   */
+  @Nullable
+  TaskAuthContext createTaskAuthContext(AuthenticationResult authenticationResult, Task task);


...ing-service/src/main/java/org/apache/druid/indexing/common/task/TaskAuthContextProvider.java

+   * @return TaskAuthContext to inject into the task, or null to skip injection
+   */
+  @Nullable
+  TaskAuthContext createTaskAuthContext(AuthenticationResult authenticationResult, Task task);


...e/src/test/java/org/apache/druid/indexing/overlord/http/OverlordResourceAuthContextTest.java

cecemei · 2026-04-07T18:03:54Z

@jtuglu1 whats the status for this PR? should we move this to 38?

jtuglu1 · 2026-04-07T18:16:23Z

@jtuglu1 whats the status for this PR? should we move this to 38?

This one I wanted to get into v37 – I was hoping to get this merged today.

gianm

I reviewed the core changes only, focusing on trying to understand the security properties of the changes.

I wonder what your thoughts are on an alternate design that keeps the vended credentials inside the input source itself:

Add a new method to InputSource like scopeForUser(AuthenticationResult authResult). Default implementation is return this
Whenever a task is submitted, call scopeForUser on all of its input sources at whichever service initially accepts the task (either Broker [for SQL DML] or Overlord [for anything else]).
The IcebergInputSource would implement scopeForUser to fetch the vended credentials and transform itself into an input source that bakes in the vended credentials. It would use a PasswordProvider so it is redactable.

The idea would be to avoid the need for a new credential vending system in core, putting most of the changes inside the Iceberg extension instead.

gianm · 2026-04-08T07:31:05Z

processing/src/main/java/org/apache/druid/auth/TaskAuthContext.java

+
+  /**
+   * Returns sensitive credentials (e.g., OAuth tokens, API keys).
+   * This method MUST be redacted during serialization via {@link TaskAuthContextRedactionMixIn}.


This is over-eager, isn't it? The credentials must be serialized in some cases. When we send a task spec from Overlord to the server that will actually run the task, the credentials must be included. In some cases the nonredacted task file will need to end up on disk, so it can be run. Perhaps this should say "MUST be redacted when written to log files, the metadata store, or returned from user-facing APIs".

Yeah the comment here is a bit aggressive, but that's currently what is already done. The field is not redacted when being submitted from Overlord to MM/Peon/Indexer. It is only redacted currently on logging, persistence (to disk or db).

gianm · 2026-04-08T07:32:27Z

processing/src/main/java/org/apache/druid/auth/TaskAuthContext.java

+ *
+ * <p><b>Credential lifetime:</b> Vended credentials (OAuth tokens, STS session tokens) have
+ * limited lifetimes, typically 1-12 hours. There is currently no credential refresh mechanism,
+ * so long-running tasks may fail when credentials expire. Task restarts are also not supported


Any thoughts on how to handle long-running tasks? I expect it will be a problem. It's not uncommon for tasks to take hours.

I was going to add this in a separate change. Using a task auth context, inputSource could conceivably just use that with some implementable revend() method. The only issue I would want to avoid is having the subtasks refresh the credentials (and not the driver task), since this might cause thundering herd effect where many subtasks all attempt to refresh at once. Spark does a better job of this by letting executor handle credential refresh on the driver, then propagate that to the executors.

gianm · 2026-04-08T07:34:39Z

processing/src/main/java/org/apache/druid/auth/TaskAuthContext.java

+   *
+   * @return the identity string
+   */
+  String getIdentity();


The auth context is a @JsonProperty so I believe that means people can set it explicitly when they submit tasks. Does anything bad happen if someone sets a context and sets the identity to someone else? What do you think about clearing the user-provided auth context, if any, in OverlordResource?

Does anything bad happen if someone sets a context and sets the identity to someone else?

If we want to scope the auth context to Druid-settable only, then yes we should. IMO, I think that's a safer option but I could see people wanting to use this apart from Authorization result.

gianm · 2026-04-08T07:38:01Z

multi-stage-query/src/main/java/org/apache/druid/msq/sql/MSQTaskQueryMaker.java


-    FutureUtils.getUnchecked(overlordClient.runTask(taskId, controllerTask), true);
+    // Propagate auth context headers to Overlord for consumption
+    if (plannerContext.getAuthenticationResult() != null && plannerContext.getAuthenticationResult().getContext() != null) {


What is the purpose of this? Authentication context is meant to be an arbitrary extension-defined in-memory map. It may not in general take well to being stuffed into a header. It may also include sensitive information that shouldn't be sent in a header.

We need a way to propagate AuthenticationResult from the broker to the overlord. I could make an overridable serialization method/class for AuthenticationResult to ensure we only pass-through what's needed (and let the user specify that), but propagating the headers here was the least intrusive approach for that change. We still need to pass the authentication result somehow.

gianm · 2026-04-08T07:47:27Z

indexing-service/src/main/java/org/apache/druid/indexing/overlord/http/OverlordResource.java


+    // Inject auth context if provider is configured
+    if (taskAuthContextProvider != null) {
+      final AuthenticationResult authenticationResult = AuthorizationUtils.authenticationResultFromRequest(req);


How will this work in the SQL DML path, where the user submits a task to /druid/v2/sql/task/ and the Broker then submits the task using its own credentials? The current design is that the Broker authenticates the user, authorizes the DML operation, and then submits it to the Overlord using a service account (not the user's own credentials).

IMO it would make more sense in this case for the Broker to get the vended credentials and pass them along to the Overlord.

How will this work in the SQL DML path, where the user submits a task to /druid/v2/sql/task/ and the Broker then submits the task using its own credentials?

That's the thing. In our implementation, the broker passes through the user auth context and not its own credentials, so there's no use of the Broker credentials in the task payload (only used for validating the request came from the Brokers).

gianm · 2026-04-08T07:51:56Z

processing/src/main/java/org/apache/druid/auth/TaskAuthContext.java

+ */
+@ExtensionPoint
+@JsonTypeInfo(use = JsonTypeInfo.Id.NAME, property = "type")
+public interface TaskAuthContext


I didn't see an implementation of TaskAuthContext or TaskAuthContextProvider in this PR. Are they meant to be added later? What would they look like? I was wondering in particular what taskAuthContextProvider.createTaskAuthContext would do exactly with the authenticationResult that is passed in.

They are meant for users to implement. I have internal versions of these classes which take our internal identity tokens and propagate them through to the Iceberg input source, to be then used for vending S3 credentials to read data from Iceberg.

jtuglu1 · 2026-04-08T17:29:46Z

I reviewed the core changes only, focusing on trying to understand the security properties of the changes.

I wonder what your thoughts are on an alternate design that keeps the vended credentials inside the input source itself:

Add a new method to InputSource like scopeForUser(AuthenticationResult authResult). Default implementation is return this

Whenever a task is submitted, call scopeForUser on all of its input sources at whichever service initially accepts the task (either Broker [for SQL DML] or Overlord [for anything else]).

The IcebergInputSource would implement scopeForUser to fetch the vended credentials and transform itself into an input source that bakes in the vended credentials. It would use a PasswordProvider so it is redactable.

The idea would be to avoid the need for a new credential vending system in core, putting most of the changes inside the Iceberg extension instead.

Are you proposing pushing the vending of credentials using an identity to the broker/overlord prior to task submission? I'd ideally like to propagate the auth context to the task and have it vend the credentials at runtime, not at submit time.

Currently, the way this works is:

We extract user identity
We pass to an Iceberg catalog client which uses this identity to then vend S3 credentials
These S3 credentials are then attached to the subtasks that spawn from the driver task

jtuglu1 changed the title ~~Support user identity forwarding to tasks via TaskAuthContext~~ feat: support user identity forwarding to tasks via TaskAuthContext Mar 30, 2026

github-actions bot added Area - Batch Ingestion Area - Dependencies Area - Ingestion Area - MSQ For multi stage queries - https://github.com/apache/druid/issues/12262 labels Mar 30, 2026

jtuglu1 force-pushed the task-auth-context branch from f68a5ba to 3255f0b Compare March 30, 2026 23:31

jtuglu1 added the Design Review label Mar 30, 2026

github-advanced-security bot found potential problems Mar 31, 2026

View reviewed changes

jtuglu1 force-pushed the task-auth-context branch 3 times, most recently from f2012ec to 99c006c Compare March 31, 2026 02:13

github-advanced-security bot found potential problems Mar 31, 2026

View reviewed changes

jtuglu1 force-pushed the task-auth-context branch 4 times, most recently from 1f4e9ff to 09f57a7 Compare April 1, 2026 03:27

jtuglu1 added this to the 37.0.0 milestone Apr 1, 2026

jtuglu1 force-pushed the task-auth-context branch from 33be042 to 81fae04 Compare April 2, 2026 01:35

github-actions bot added the Area - Querying label Apr 2, 2026

jtuglu1 force-pushed the task-auth-context branch from 81fae04 to 6ac3139 Compare April 6, 2026 18:25

Support user identity forwarding to tasks via TaskAuthContext

74d8369

jtuglu1 force-pushed the task-auth-context branch from 6ac3139 to 74d8369 Compare April 7, 2026 18:35

gianm reviewed Apr 8, 2026

View reviewed changes

jtuglu1 requested review from abhishekrb19 and gianm April 8, 2026 18:27

Conversation

jtuglu1 commented Mar 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Design

Constraints

Release note

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Check notice

Check notice

Uh oh!

cecemei commented Apr 7, 2026

Uh oh!

jtuglu1 commented Apr 7, 2026

Uh oh!

gianm left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jtuglu1 Apr 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jtuglu1 Apr 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jtuglu1 commented Apr 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

jtuglu1 commented Mar 30, 2026 •

edited

Loading

jtuglu1 Apr 8, 2026 •

edited

Loading

jtuglu1 Apr 8, 2026 •

edited

Loading

jtuglu1 commented Apr 8, 2026 •

edited

Loading