feat: adding project-discovery-worker (CM-949)#3832
Conversation
There was a problem hiding this comment.
Pull request overview
Introduces a new Temporal worker app (automatic_projects_discovery_worker) and adds initial database schema to support automatic project discovery/evaluation.
Changes:
- Added a new worker service with Temporal workflow/activity scaffolding plus a schedule that registers a cron-based Temporal Schedule.
- Added Docker and compose definitions to run the worker in the existing services stack.
- Added DB migrations for
projectCatalogandevaluatedProjectstables.
Reviewed changes
Copilot reviewed 13 out of 14 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| services/apps/automatic_projects_discovery_worker/tsconfig.json | Adds TS config for the new worker package. |
| services/apps/automatic_projects_discovery_worker/src/workflows/discoverProjects.ts | Introduces a placeholder Temporal workflow that calls a logging activity. |
| services/apps/automatic_projects_discovery_worker/src/workflows.ts | Exports the worker’s workflows for registration. |
| services/apps/automatic_projects_discovery_worker/src/schedules/scheduleProjectsDiscovery.ts | Registers a Temporal Schedule to trigger the workflow on a cron. |
| services/apps/automatic_projects_discovery_worker/src/main.ts | Worker bootstrap: init, schedule registration, start. |
| services/apps/automatic_projects_discovery_worker/src/activities/activities.ts | Placeholder activity implementation (logs a run). |
| services/apps/automatic_projects_discovery_worker/src/activities.ts | Barrel export for activities registration. |
| services/apps/automatic_projects_discovery_worker/package.json | Adds the new worker package and scripts. |
| scripts/services/docker/Dockerfile.automatic_projects_discovery_worker | Adds container build/run definition for the worker. |
| scripts/services/automatic-projects-discovery-worker.yaml | Adds docker-compose definitions for running the worker (prod + dev). |
| scripts/builders/automatic-projects-discovery-worker.env | Adds builder configuration for publishing/building the worker image. |
| pnpm-lock.yaml | Adds lockfile entries for the new workspace package (but currently includes an extra incorrect importer). |
| backend/src/database/migrations/V1770653666__add-automatic_projects_discovery-tables.sql | Adds tables/indexes for project discovery catalog + evaluations. |
| backend/src/database/migrations/U1770653666__add-automatic_projects_discovery-tables.sql | Adds down migration to drop the tables/indexes. |
Files not reviewed (1)
- pnpm-lock.yaml: Language not supported
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| version: 3.3.3 | ||
|
|
||
| services/apps/automatic_project_discovery_worker: | ||
| dependencies: | ||
| '@crowd/archetype-standard': | ||
| specifier: workspace:* | ||
| version: link:../../archetypes/standard | ||
| '@crowd/archetype-worker': | ||
| specifier: workspace:* | ||
| version: link:../../archetypes/worker | ||
| '@crowd/common': | ||
| specifier: workspace:* | ||
| version: link:../../libs/common | ||
| '@crowd/common_services': | ||
| specifier: workspace:* | ||
| version: link:../../libs/common_services | ||
| '@crowd/data-access-layer': | ||
| specifier: workspace:* | ||
| version: link:../../libs/data-access-layer | ||
| '@crowd/logging': | ||
| specifier: workspace:* | ||
| version: link:../../libs/logging | ||
| '@crowd/redis': | ||
| specifier: workspace:* | ||
| version: link:../../libs/redis | ||
| '@crowd/temporal': | ||
| specifier: workspace:* | ||
| version: link:../../libs/temporal | ||
| '@crowd/types': | ||
| specifier: workspace:* | ||
| version: link:../../libs/types | ||
| '@temporalio/activity': | ||
| specifier: ~1.11.8 | ||
| version: 1.11.8 | ||
| '@temporalio/client': | ||
| specifier: ~1.11.8 | ||
| version: 1.11.8 | ||
| '@temporalio/workflow': | ||
| specifier: ~1.11.8 | ||
| version: 1.11.8 | ||
| tsx: | ||
| specifier: ^4.7.1 | ||
| version: 4.7.3 | ||
| typescript: | ||
| specifier: ^5.6.3 | ||
| version: 5.6.3 | ||
| devDependencies: | ||
| '@types/node': | ||
| specifier: ^20.8.2 | ||
| version: 20.12.7 | ||
| nodemon: | ||
| specifier: ^3.0.1 | ||
| version: 3.1.0 | ||
|
|
||
| services/apps/automatic_projects_discovery_worker: | ||
| dependencies: | ||
| '@crowd/archetype-standard': | ||
| specifier: workspace:* | ||
| version: link:../../archetypes/standard | ||
| '@crowd/archetype-worker': | ||
| specifier: workspace:* | ||
| version: link:../../archetypes/worker | ||
| '@crowd/common': | ||
| specifier: workspace:* | ||
| version: link:../../libs/common | ||
| '@crowd/common_services': |
There was a problem hiding this comment.
pnpm-lock.yaml contains two importers for this new app: services/apps/automatic_projects_discovery_worker (correct) and services/apps/automatic_project_discovery_worker (missing the s in projects). The latter path doesn’t exist in the repo and will leave the lockfile in an inconsistent state (pnpm will warn/error depending on settings). Please regenerate or edit the lockfile so only the correct importer remains.
| nodemon: | ||
| specifier: ^3.0.1 | ||
| version: 3.1.0 | ||
|
|
There was a problem hiding this comment.
Stale workspace importer in lockfile
Medium Severity
pnpm-lock.yaml adds an importer for services/apps/automatic_project_discovery_worker, but that workspace path is not present while the real app is services/apps/automatic_projects_discovery_worker. This leaves a stale duplicate importer in the lockfile, creating an inconsistent workspace definition that can destabilize pnpm install --frozen-lockfile behavior and dependency maintenance.
| svc.log.info('Schedule already registered in Temporal.') | ||
| svc.log.info('Configuration may have changed since. Please make sure they are in sync.') | ||
| } else { | ||
| throw new Error(err) |
There was a problem hiding this comment.
throw new Error(err) loses original error context
Low Severity
throw new Error(err) wraps the caught err (likely already an Error) inside a new Error, which discards the original stack trace, error type, and any custom properties. The resulting error message will be the stringified version of the original error (e.g., "Error: ...") rather than the original message. Rethrowing err directly would preserve all original context, making debugging much easier.
e669595 to
b03cb7a
Compare
|
|
b03cb7a to
425e4cb
Compare
Signed-off-by: Umberto Sgueglia <usgueglia@contractor.linuxfoundation.org>
Signed-off-by: Umberto Sgueglia <usgueglia@contractor.linuxfoundation.org>
Signed-off-by: Umberto Sgueglia <usgueglia@contractor.linuxfoundation.org>
Signed-off-by: Umberto Sgueglia <usgueglia@contractor.linuxfoundation.org>
Signed-off-by: Umberto Sgueglia <usgueglia@contractor.linuxfoundation.org>
Signed-off-by: Umberto Sgueglia <usgueglia@contractor.linuxfoundation.org>
Signed-off-by: Umberto Sgueglia <usgueglia@contractor.linuxfoundation.org>
b5c3b03 to
7e6dc18
Compare


Summary
This PR introduces the basic structure for the new worker.
The implementation includes the foundational setup required to run the worker, without full business logic.
Included
Note
Medium Risk
Adds new database tables/indexes and introduces a new Temporal-scheduled worker service; migration/infra changes can impact deploys and scheduling behavior if misconfigured.
Overview
Adds the initial automatic projects discovery foundation: a new Temporal worker app that registers a cron schedule (env-configurable, default daily) to start a
discoverProjectsworkflow and run a placeholder activity.Introduces new DB schema for discovery/evaluation via migrations creating
projectCatalogandevaluatedProjectstables (with uniqueness and query indexes) plus a matching down migration, and adds Docker/compose build+run wiring for the new worker along withpnpm-lock.yamlworkspace entries.Written by Cursor Bugbot for commit 7e6dc18. This will update automatically on new commits. Configure here.