Drill-XXXX: Refactor User Interface by cgivre · Pull Request #3037 · apache/drill

cgivre · 2026-02-06T15:45:09Z

DRILL-XXXX: Refactor User Interface

Description

This PR refactors Drill's UI and refactors the Query view, adds visualizations and dashboards and in general makes Drill much more user friendly.

Documentation

(Please describe user-visible changes similar to what should appear in the Drill documentation.)

Testing

(Please describe how this PR has been tested.)

exec/java-exec/src/main/java/org/apache/drill/exec/server/rest/SqlLabSpaServlet.java

exec/java-exec/src/main/java/org/apache/drill/exec/server/rest/DashboardResources.java

exec/java-exec/src/main/resources/webapp/src/pages/DashboardViewPage.tsx

exec/java-exec/src/main/java/org/apache/drill/exec/server/rest/DashboardResources.java

+    return Response.ok(imageFile, contentType)
+        .header("Cache-Control", "public, max-age=86400")
+        .header("Content-Disposition", "inline; filename=\"" + filename + "\"")
+        .header("Content-Security-Policy", "default-src 'none'; style-src 'unsafe-inline'")
+        .header("X-Content-Type-Options", "nosniff")
+        .build();


The problem was that adding a workspace would call emitChange, which updated the parent's config. The parent would re-render and pass the updated config back to FileSystemForm. The useEffect([config]) would then fire and re-initialize workspaces from config - but config only contains workspaces with valid names, so empty workspaces were filtered out. The fix is to only re-initialize workspaces when the connection string changes (indicating a different storage plugin), not on every config update. This allows local workspace changes to persist while still loading workspaces when the user switches plugins. Also cleaned up debug console.logs.

Prevent users from creating workspaces with duplicate names in the same plugin. When duplicate names are detected: - Display an error alert showing which names are duplicated - Disable the Save button until duplicates are resolved - Update validation in real-time as user types Added getDuplicateWorkspaceNames helper function to detect duplicates, and integrated it into the validation logic.

The workspace list wasn't being loaded on first component mount because lastConfigRef was initialized with config, so on the first useEffect run, lastConnection === currentConnection and initialization was skipped. Added initializedRef flag to ensure workspaces are loaded on first mount, then only re-initialize when the connection string changes. This ensures saved workspaces appear in the UI after page reload while still preserving local edits during the same session.

## Summary Add interactive column menus to the ResultsGrid component with 5 feature categories: 1. **Actions** - Hide/show columns, pin/unpin to left/right 2. **Statistics** - Calculate and display column stats (count, null count, distinct, min/max/avg) 3. **Filters** - Show top 10 values and data quality indicators 4. **Transform** - SQL transformations (UPPER, LOWER, TRIM, CAST, TRUNCATE, SUBSTRING) 5. **Export** - Copy to clipboard or export as CSV ## Architecture - **SQL Transformation Utility** (src/utils/sqlTransformations.ts): Regex-based transformation logic - Handles simple columns, table prefixes, backticks, and aliases - Safe column wrapping with SQL functions - Client-side statistics calculation - **ColumnMenu Component** (src/components/results/ColumnMenu.tsx): Custom menu with 5 tabs - Statistics calculation from displayed data - Column visibility and pinning controls - Top 10 values preview for quick data exploration - Text transformations and type casting options - **CustomHeader Component** (src/components/results/CustomHeader.tsx): AG Grid header integration - Menu button in column headers using Popover - Connects to ColumnMenu for user interactions - **ResultsGrid Updates** (src/components/results/ResultsGrid.tsx): - Column state persistence to localStorage - New toolbar buttons for column and sort management - Modal dialogs for managing column visibility and sort order - Event listeners for saving state on column changes - **SqlLabPage Handler** (src/pages/SqlLabPage.tsx): - handleTransformColumn callback that shows preview modal - Transformation confirmation before modifying SQL - Auto-executes query after transformation ## Features Implemented ✅ Column Actions (hide/show, pin/unpin) ✅ Column Statistics (count, nulls, distinct, min/max/avg, percentages) ✅ Data Quality Indicators (non-null %, uniqueness %) ✅ Quick Data Preview (top 10 values) ✅ SQL Transformations (UPPER, LOWER, TRIM, CAST, TRUNCATE, SUBSTRING) ✅ Export Functions (copy to clipboard, export as CSV) ✅ Column State Persistence (localStorage) ✅ Sort Management (view current sorts, clear all) ✅ Column Management (show/hide all columns) ## Technical Details - Regex-based SQL transformation instead of parser library (avoids 500KB+ dependency) - Statistics computed on displayed/paginated data (not full dataset) - AG Grid API integration for column visibility and pinning - localStorage for persisting column state across sessions - TypeScript interfaces for type safety - Ant Design components for consistent UI

Adds comprehensive query profile visualization with modern dashboard layout: **New Page (ProfileDetailPage.tsx):** - Header with back button, query ID, and action buttons (Run in SQL Lab, Download JSON) - Status bar with badge, user, foreman, and queue info - Stat cards: Status, Total Duration (with phase breakdown), Total Cost, Fragments - Two-column layout: - Left (2/3): Tabbed content with Overview, Plan, Fragments, Operators, Error Details - Right (1/3): AI Advisor panel with auto-loading analysis **Visualizations:** - Execution Waterfall: ECharts horizontal stacked bar (Planning|Queue|Execution) - Physical Plan Tree: Parses indented plan text into interactive tree chart - Fragment Gantt: Time-based chart showing fragment execution windows - Top Operators: Horizontal bar chart of slowest operators - Operator Metrics: Sortable/filterable table with timing, memory, record counts **AI Advisor Features:** - Auto-runs summary on page load (no user interaction needed) - Five prompt types: Summary, Bottlenecks, Optimize, Explain Plan, Explain Tab - Streaming responses with react-markdown rendering - Context-aware "Explain Tab" that changes based on active tab - Graceful UI when AI not configured **API Changes (queries.ts):** - Added DetailedQueryProfile interface with full execution details - Added DrillbitEndpoint, OperatorProfile, FragmentProfile hierarchies - Added getQueryProfileDetail(queryId) to fetch profile JSON **Router & Navigation:** - Added route: /profiles/:queryId → ProfileDetailPage - Updated ProfilesPage links to use React Router (no new tab) **Build:** TypeScript clean, production build successful (14.11s)

…ed metric cards - Add prominent SQL query display at top of page in collapsible card - Redesign stat cards with better spacing and responsive layout - Use full height cards with centered content for uniform appearance - Increase font sizes for better readability - Add responsive breakpoints (xs, sm, lg) for mobile compatibility - Improve visual hierarchy with better typography

- Move profile analysis context from ChatContext.resultSummary to user message body - ChatContext now only contains minimal metadata (currentSql, error) - Full profile context (plan, operators, timing) included in user message - Fixes backend deserialization error: ResultSummary cannot deserialize from String - Separates concerns: context metadata vs. analysis prompt content

Security enhancement: Automatically redact all credentials before sending profile data to the AI advisor. This prevents accidental exposure of: - API keys, tokens, and passwords in queries or filters - CredentialProvider objects and userCredentials from query plans - Connection strings, email addresses, IP addresses - Base64-encoded credentials and OAuth URLs Changes: - Add redactCredentials() helper function with comprehensive patterns - Apply redaction to query, plan, and error messages before AI submission - Add privacy notice in AI advisor panel explaining redaction - Specifically handle CredentialProvider objects from Drill execution plans The redaction is performed on client-side before any data is sent to external AI services.

Fixes reCAPTCHA verification errors on hosted versions: **Enhanced Token Detection:** - Check multiple meta tag locations (csrf-token, _csrf, csrf, X-CSRF-TOKEN) - Check multiple cookie names (drill.csrf.token, _csrf, XSRF-TOKEN, X-CSRF-TOKEN) - More robust token extraction to handle different deployment configurations **Better Error Messages:** - Specific handling for reCAPTCHA/CSRF errors (HTTP 403) - Clear guidance when token is missing or invalid - Distinguish between missing auth and reCAPTCHA failures - Add console warning when CSRF token cannot be found **Root Cause:** On hosted deployments (e.g., Digital Ocean), the CSRF token location or name may differ from default setup, causing requests to be rejected as suspicious. The enhanced detection handles these variations.

This reverts commit 45b2073.

cgivre added 3 commits February 5, 2026 22:17

Intial commit for phase 4: Dashboards

20517b4

More Phase 4 work

2825e3a

Dashboard Improvements

367fa23

cgivre self-assigned this Feb 6, 2026

cgivre changed the title ~~User Interface Improvements~~ Drill-XXXX: Refactor User Interface Feb 6, 2026

cgivre added enhancement PRs that add a new functionality to Drill doc-impacting PRs that affect the documentation ui PRs relating to the User Interface major-update labels Feb 6, 2026

Fix RAT

1321cc2

github-advanced-security bot found potential problems Feb 6, 2026

View reviewed changes

exec/java-exec/src/main/java/org/apache/drill/exec/server/rest/SqlLabSpaServlet.java Fixed Show fixed Hide fixed

exec/java-exec/src/main/java/org/apache/drill/exec/server/rest/SqlLabSpaServlet.java Fixed Show fixed Hide fixed

Add Image Upload to Dashboard

11c3f23

github-advanced-security bot found potential problems Feb 6, 2026

View reviewed changes

exec/java-exec/src/main/java/org/apache/drill/exec/server/rest/DashboardResources.java Fixed Show fixed Hide fixed

exec/java-exec/src/main/java/org/apache/drill/exec/server/rest/DashboardResources.java Fixed Show fixed Hide fixed

Image upload working

3a68908

github-advanced-security bot found potential problems Feb 6, 2026

View reviewed changes

exec/java-exec/src/main/resources/webapp/src/pages/DashboardViewPage.tsx Fixed Show fixed Hide fixed

cgivre added 3 commits February 6, 2026 16:48

Added Frontend Unit tests

5266921

Unit test fix

b47360a

Fix CodeQL issues

97bcaa4

github-advanced-security bot found potential problems Feb 8, 2026

View reviewed changes

exec/java-exec/src/main/resources/webapp/src/pages/DashboardViewPage.tsx Fixed Show fixed Hide fixed

Fix CodeQL issues

6b7d2b9

github-advanced-security bot found potential problems Feb 8, 2026

View reviewed changes

exec/java-exec/src/main/resources/webapp/src/pages/DashboardViewPage.tsx Fixed Show fixed Hide fixed

Try again...

5d24f9d

github-advanced-security bot found potential problems Feb 8, 2026

View reviewed changes

cgivre added 7 commits February 8, 2026 09:00

More CodeQL Fixes

ee49d3b

More code fixes

be9dee8

Fix build error

b8b68b2

Various fixes

f9f2a9d

XML Nav Tree Improvements

e196c07

Nav Tree working

9b175b3

Add big number

68c8660

cgivre and others added 30 commits February 11, 2026 14:12

More improvements

a208f70

Various improvements

b02d231

Dark mode:

b24e435

Minor bug fix

efc2d12

Fix state

9acd002

Fix Vault Unit Tests

9720fa8

Fix Docker based unit tests

788c9a6

Visualization improvements

91e57dd

Link fix

51d1db2

Various fixes

8c954d7

Fixing time grain issues

b74dbe7

WIP Time Grain

d236f8a

Time grain working

28fc2ef

Filter improvements and hardening.

5396bde

Refactor SQL generator to use Java port of SQL Glot

d00ad0a

Fix checkstyles

09c26ee

Add trendlines

8e53e30

Profile page improvements

16f7f4d

Revert "Improve CSRF token handling for hosted deployments"

c0278bd

This reverts commit 45b2073.

Added new visualizations

de91ff4

Fix streaming UI bug

db8d55e

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Drill-XXXX: Refactor User Interface#3037

Drill-XXXX: Refactor User Interface#3037
cgivre wants to merge 56 commits intoapache:masterfrom
cgivre:feature/sqllab-react-ui

cgivre commented Feb 6, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Check warning

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

cgivre commented Feb 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

DRILL-XXXX: Refactor User Interface

Description

Documentation

Testing

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Check warning

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

cgivre commented Feb 6, 2026 •

edited

Loading