Add getEvents full-history design doc#635
Add getEvents full-history design doc#635urvisavla wants to merge 9 commits intostellar:feature/full-historyfrom
Conversation
Add the RPC getEvents full-history design document with architecture diagrams. The doc covers the segment-based storage model, bitmap indexing, hot/cold segment lifecycle, and backfill process. Hot segment storage is presented as storage-backend agnostic (flat files vs embedded DB), while keeping in-memory bitmaps as a design decision.
There was a problem hiding this comment.
Pull request overview
Adds a design document for a proposed getEvents full-history storage/indexing backend, plus extracted diagram assets to support the doc.
Changes:
- Added
getEventsfull-history design doc describing segmentation, hot/cold storage, bitmap indexing, freeze/recovery, and capacity estimates. - Added query-routing flowchart diagram as a standalone PNG asset.
Reviewed changes
Copilot reviewed 1 out of 3 changed files in this pull request and generated 9 comments.
| File | Description |
|---|---|
docs/getevents-full-history-design.md |
New full-history getEvents backend design doc (architecture, indexing, storage, query paths, capacity placeholders). |
docs/query-routing-flowchart.png |
Diagram asset referenced by the design doc for query routing. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Adds a design doc describing the proposed storage + indexing architecture for serving full-history getEvents queries, along with an accompanying query-routing flowchart diagram.
Changes:
- Added a full-history
getEventsbackend design doc (segments, bitmap indexing, hot/cold storage, freeze/backfill, query path). - Added a query-routing flowchart diagram referenced by the design doc.
Reviewed changes
Copilot reviewed 1 out of 3 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
| docs/getevents-full-history-design.md | New full-history getEvents design document with architecture and implementation reference sections. |
| docs/query-routing-flowchart.png | New diagram referenced by the design doc’s query-routing section. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
There was a problem hiding this comment.
Pull request overview
Adds a design doc describing a proposed storage/indexing architecture to support full-history getEvents queries, including a query-routing flowchart diagram referenced by the doc.
Changes:
- Add full-history
getEventsbackend design doc (segments, bitmap index, freeze/backfill flows, capacity placeholders). - Add query routing flowchart image used by the design doc.
Reviewed changes
Copilot reviewed 1 out of 3 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
| design-docs/getevents-full-history-design.md | New design doc covering architecture + implementation reference for full-history getEvents. |
| design-docs/query-routing-flowchart.png | New diagram referenced by the doc for query routing flow. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Adds a design document describing a full-history storage + indexing backend for the getEvents RPC, including segment architecture (hot/cold), roaring-bitmap indexing, freeze/backfill workflows, and query routing, with supporting diagrams.
Changes:
- Add a full-history
getEventsbackend design doc covering data model, segment/index layout, query path, and operational workflows. - Add a query-routing flowchart diagram referenced by the doc.
Reviewed changes
Copilot reviewed 1 out of 3 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
| design-docs/getevents-full-history-design.md | New design doc defining storage/indexing architecture, query execution, freeze/backfill, and capacity placeholders. |
| design-docs/query-routing-flowchart.png | New diagram referenced by the doc for query routing flow. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
What
This PR contains the design doc for the full-history
getEventsstorage and indexing layer.The design for cold segment storage (which drives the backfill workflow) has been discussed and finalized. The hot/active segment storage sections are included but need further evaluation.
The document also includes placeholder sections (memory estimates, query performance, ingestion throughput, operational profile, observability, tiered storage, and scaling projections) that will be updated as benchmarking data becomes available.
Why
#637
Test plan