feat(x2a): add sorting by project status

[mareklibra]

The Status field on the projects' list page can be newly sorted.

First by name, then by modules summary.

Since this field is calculated, the sorting can not be done in the DB and so such sorting can become expensive.
If it proves to be a bottleneck in large setups, it will be a subject either to remove or find an optimization like materialized DB columns.

[rhdh-gh-app]

Important

This PR includes changes that affect public-facing API. Please ensure you are adding/updating documentation for new features or behavior.

Changed Packages

Package Name	Package Path	Changeset Bump	Current Version
@red-hat-developer-hub/backstage-plugin-x2a-backend	workspaces/x2a/plugins/x2a-backend	minor	v1.1.0

[rhdh-qodo-merge]

Review Summary by Qodo

Implement sorting by project status with in-memory pagination

✨ Enhancement

Walkthroughs

Description

• Implement in-memory sorting by project status field
• Sort projects by semantic status state order with secondary sort by module summary percentages
• Apply pagination after in-memory sort to handle computed field sorting
• Add comprehensive test coverage for status sorting with various scenarios

Diagram

flowchart LR
  A["Query with sort=status"] --> B["Skip DB pagination"]
  B --> C["Fetch all matching projects"]
  C --> D["Enrich projects with status"]
  D --> E["Sort in-memory by state order"]
  E --> F["Secondary sort by module summary"]
  F --> G["Apply pagination"]
  G --> H["Return paginated results"]

Loading

File Changes

1. workspaces/x2a/plugins/x2a-backend/src/services/X2ADatabaseService/index.ts ✨ Enhancement +92/-1

Add in-memory sorting logic for status field

• Add STATE_ORDER static mapping for semantic status state ordering
• Implement sortAndPaginateInMemory method for computed field sorting
• Detect non-DB sort fields and skip database pagination when needed
• Log warning when sorting large datasets by computed fields
• Apply in-memory sort and pagination after project enrichment

workspaces/x2a/plugins/x2a-backend/src/services/X2ADatabaseService/index.ts

---

2. workspaces/x2a/plugins/x2a-backend/src/services/X2ADatabaseService/projectOperations.ts ✨ Enhancement +38/-20

Support skipping database pagination for computed fields

• Add dbOptions parameter to listProjects method
• Conditionally skip database-level pagination when skipPagination is true
• Fetch all matching rows when sorting by computed fields
• Calculate totalCount from row length when pagination is skipped
• Maintain permission filtering regardless of pagination mode

workspaces/x2a/plugins/x2a-backend/src/services/X2ADatabaseService/projectOperations.ts

---

3. workspaces/x2a/plugins/x2a-backend/src/services/X2ADatabaseService/queryHelpers.ts ✨ Enhancement +9/-1

Add helpers to identify non-database sort fields

• Add NON_DB_SORT_FIELDS set to identify computed fields without DB columns
• Update mapSortToDatabaseColumn to return undefined for non-DB fields
• Add isNonDbSortField helper function to detect computed sort fields

workspaces/x2a/plugins/x2a-backend/src/services/X2ADatabaseService/queryHelpers.ts

---

View more (5)

4. workspaces/x2a/plugins/x2a-backend/src/services/constants.ts ⚙️ Configuration changes +6/-0

Add threshold constant for in-memory sort warnings

• Add IN_MEMORY_SORT_WARN_THRESHOLD constant set to 100
• Used to trigger warning logs when sorting large datasets in memory

workspaces/x2a/plugins/x2a-backend/src/services/constants.ts

---

5. workspaces/x2a/plugins/x2a-backend/src/router/projects.ts ✨ Enhancement +1/-0

Enable status field in sort parameter validation

• Add status to allowed sort enum values
• Add comment noting that status sorting is expensive for large datasets

workspaces/x2a/plugins/x2a-backend/src/router/projects.ts

---

6. workspaces/x2a/plugins/x2a-backend/src/services/X2ADatabaseService/projects.test.ts 🧪 Tests +473/-0

Add extensive test coverage for status sorting

• Add comprehensive test suite for status sorting with 10 test cases
• Test semantic state ordering in ascending and descending directions
• Test secondary sorting by module summary percentages (finished, error, running, etc.)
• Test pagination correctness after in-memory sort
• Test edge cases like empty lists, pages beyond data, and permission filtering
• Test all six status states in correct semantic order

workspaces/x2a/plugins/x2a-backend/src/services/X2ADatabaseService/projects.test.ts

---

7. workspaces/x2a/plugins/x2a-backend/src/router/projects.test.ts 🧪 Tests +107/-0

Add integration test for status sorting with pagination

• Add integration test for sorting projects by status with pagination
• Create three projects with different calculated statuses
• Verify ascending and descending sort order correctness
• Verify pagination works correctly with status sorting
• Test both page 0 and page 1 with pageSize 2

workspaces/x2a/plugins/x2a-backend/src/router/projects.test.ts

---

8. workspaces/x2a/.changeset/purple-otters-turn.md 📝 Documentation +5/-0

Add changeset for status sorting feature

• Add changeset entry documenting the new feature
• Mark as minor version bump for backend plugin

workspaces/x2a/.changeset/purple-otters-turn.md

---

[rhdh-qodo-merge]

Code Review by Qodo

🐞 Bugs (1) 📘 Rule violations (0) 📎 Requirement gaps (0) 📐 Spec deviations (0)

1. Unbounded status-sort enrichment 🐞 Bug ➹ Performance

Description

When sort=status is requested, the backend disables DB pagination and then concurrently enriches
every matching project via Promise.all, which triggers multiple DB queries per project/module.
This can overwhelm the DB connection pool and significantly degrade or crash the service under
larger datasets or repeated requests.

Code

workspaces/x2a/plugins/x2a-backend/src/services/X2ADatabaseService/index.ts[R133-156]

+    const sortByComputedField = isNonDbSortField(query.sort);
+
+    const result = await this.#projectOps.listProjects(query, options, {
+      skipPagination: sortByComputedField,
+    });
+
+    if (
+      sortByComputedField &&
+      result.totalCount > IN_MEMORY_SORT_WARN_THRESHOLD
+    ) {
+      // If this proves to be a performance bottleneck, let's consider either removing the sort by status or having materialized DB column (works with Postgres only).
+      this.#logger.warn(
+        `In-memory sort by "${query.sort}" is loading ${result.totalCount} projects.`,
+      );
+    }

    this.#logger.info(
      `this.#projectOps.listProjects finished, adding migration plans to projects`,
    );
    await Promise.all(result.projects.map(p => this.enrichProject(p)));
+
+    if (sortByComputedField) {
+      this.sortAndPaginateInMemory(result, query);
+    }

Evidence

listProjects sets skipPagination for computed sorts (status), causing
ProjectOperations.listProjects to return all matching rows (no limit/offset). The service then
runs Promise.all(...enrichProject) over the full set; each enrichProject calls listJobs +
listModules, and listModules itself performs multiple Promise.all job fetches per module,
amplifying DB load and concurrency.

workspaces/x2a/plugins/x2a-backend/src/services/X2ADatabaseService/index.ts[125-156]
workspaces/x2a/plugins/x2a-backend/src/services/X2ADatabaseService/projectOperations.ts[110-137]
workspaces/x2a/plugins/x2a-backend/src/services/X2ADatabaseService/index.ts[67-90]
workspaces/x2a/plugins/x2a-backend/src/services/X2ADatabaseService/index.ts[310-343]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`GET /projects?sort=status` disables DB pagination and then enriches *every* matching project concurrently. Enrichment performs additional DB queries (jobs + modules, plus multiple job lookups per module), so this becomes an unbounded fan-out that can exhaust the DB connection pool / CPU.

## Issue Context
- `skipPagination` is set when sorting by a computed field (currently `status`), which makes the DB query return all matching rows.
- The service then does `Promise.all(result.projects.map(p => this.enrichProject(p)))` without any concurrency limit.
- `enrichProject()` calls `listModules()` and `listJobs()`, and `listModules()` itself does multiple `Promise.all` calls over modules.

## Fix Focus Areas
- workspaces/x2a/plugins/x2a-backend/src/services/X2ADatabaseService/index.ts[125-156]
- workspaces/x2a/plugins/x2a-backend/src/services/X2ADatabaseService/index.ts[67-90]
- workspaces/x2a/plugins/x2a-backend/src/services/X2ADatabaseService/index.ts[310-343]
- workspaces/x2a/plugins/x2a-backend/src/services/X2ADatabaseService/projectOperations.ts[110-137]

## Suggested fix
1. Add a concurrency limiter for project enrichment when `skipPagination` is enabled (e.g., `p-limit` / custom queue) so you only enrich N projects at a time.
2. Consider enforcing a hard cap / config flag for computed-field sorting (e.g., reject or require additional filtering when `totalCount` exceeds a safe threshold), since the current threshold only logs a warning.
3. (Optional but impactful) Reduce query fan-out by batching: fetch init jobs for all projectIds in one query and/or compute module summaries with fewer queries instead of per-module `Promise.all` calls.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

ⓘ The new review experience is currently in Beta. Learn more

[rhdh-qodo-merge]

1. Unbounded status-sort enrichment 🐞 Bug ➹ Performance

When sort=status is requested, the backend disables DB pagination and then concurrently enriches
every matching project via Promise.all, which triggers multiple DB queries per project/module.
This can overwhelm the DB connection pool and significantly degrade or crash the service under
larger datasets or repeated requests.

Agent Prompt

## Issue description
`GET /projects?sort=status` disables DB pagination and then enriches *every* matching project concurrently. Enrichment performs additional DB queries (jobs + modules, plus multiple job lookups per module), so this becomes an unbounded fan-out that can exhaust the DB connection pool / CPU.

## Issue Context
- `skipPagination` is set when sorting by a computed field (currently `status`), which makes the DB query return all matching rows.
- The service then does `Promise.all(result.projects.map(p => this.enrichProject(p)))` without any concurrency limit.
- `enrichProject()` calls `listModules()` and `listJobs()`, and `listModules()` itself does multiple `Promise.all` calls over modules.

## Fix Focus Areas
- workspaces/x2a/plugins/x2a-backend/src/services/X2ADatabaseService/index.ts[125-156]
- workspaces/x2a/plugins/x2a-backend/src/services/X2ADatabaseService/index.ts[67-90]
- workspaces/x2a/plugins/x2a-backend/src/services/X2ADatabaseService/index.ts[310-343]
- workspaces/x2a/plugins/x2a-backend/src/services/X2ADatabaseService/projectOperations.ts[110-137]

## Suggested fix
1. Add a concurrency limiter for project enrichment when `skipPagination` is enabled (e.g., `p-limit` / custom queue) so you only enrich N projects at a time.
2. Consider enforcing a hard cap / config flag for computed-field sorting (e.g., reject or require additional filtering when `totalCount` exceeds a safe threshold), since the current threshold only logs a warning.
3. (Optional but impactful) Reduce query fan-out by batching: fetch init jobs for all projectIds in one query and/or compute module summaries with fewer queries instead of per-module `Promise.all` calls.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

[mareklibra]

This would be a complex change in the listProjects service, let's optimize when it proves to be needed.

[mareklibra]

Introducing maxConcurrency() function to address this issue at least from the stability perspective

[sonarqubecloud]

Quality Gate passed

Issues
2 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[eloycoto]

What is clear is that I made a big mistake not forcing DDD from the beginning, each change is getting painfully with hacks here and there(max_concurrency). We should start thinking on move to a DDD structure, mostly when we have events from agents and users, and it's going to get harder and harder.

We should start thinking on services(real ones), infraestructure(current services), doing aggregate roots, and keep things more individual/modularized.

We should create an epic and a RFC for getting into DDD