Local Runway by johncmerfeld · Pull Request #9 · edanalytics/runway

johncmerfeld · 2026-01-15T00:03:33Z

@edandylytics did most of the brainwork for this feature; I think it's finally ready to go! I had to transfer this over from the old Runway repo, so please be on the lookout for any rollbacks of code changes that have come in since the switch.

How to try this

Follow the instructions in app/README.md. It should be pretty quick to get up and running jobs in either local mode.

What this does

When running locally:

Instead of kicking of an ECS task, the app launches the executor either in a local container or as a Python child process
Instead of the user uploading files to S3, "select file" sends to a local folder in the Runway repo
Instead of the executor uploading its artifacts to S3, it writes them to the same storage location, from which the app reads them
The user should not notice any difference in behavior. Output data are still uploaded to whatever ODS is configured
When running locally, don't cache the registry. This makes it easier to test in-progress bundles

What's changed

Minimal changes to the executor. The app indicates it is running locally by way of a DEPLOYMENT_MODE env var and by sending file:// sources instead of s3:// ones. We could couple those more tightly but they are nominally independent
Heaviest app change is new mechanisms for local file upload/download. These are pretty straightforward but it's kind of a lot of code
Otherwise the app has a reworked model of the executor, where the AWS version is just one of three operating modes. There are some other small changes due to the app sometimes needing to be available over host.docker.internal
Documentation and setup
Delete some files from the old days when we had a different theory of local execution

For reviewers

I think we achieved pretty good demarcation between the local stuff and production code, but this is a ratchet upward in complexity so we'll want to make sure none of the deployed functionality is impacted. There are a couple small TODOs that are really just opinion questions. New Typescript code should be scrutinized :)

Instead of making the app aware of local vs deployed file storage (filesystem fallback, local upload/download endpoints, file:// URLs), use Adobe S3Mock as a drop-in S3 replacement. Same SDK, different endpoint — one code path for both environments. - FileService: S3Client accepts LOCAL_S3_ENDPOINT_URL + forcePathStyle - Remove localFilePath, localStorageRoot, localExecutorStorageRoot - Remove local upload/download controller endpoints and service methods - Remove file:// URL generation in earthbeam-api.service - Always use s3 protocol and presigned URLs for file operations - Add S3Mock service to docker-compose - Executor (Python): boto3 accepts S3_ENDPOINT_URL - Local executors pass S3_ENDPOINT_URL + dummy AWS creds to subprocess - Simplify main.ts listen (0.0.0.0 is the default anyway) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Mermaid sequence diagram for the job execution flow, AWS dependencies table, key file index for AWS touchpoints and app-executor communication, executor lifecycle steps, and S3 path structure. Gives AI agents (and developers) a quick map of how the pieces connect without reading every source file. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Replace the ExecutorService abstract class with a TypeScript interface and a string-based DI token (EXECUTOR_SERVICE). Extract the shared envVars helper as a standalone function (executorEnvVars) that each implementation calls directly. Lighter weight than inheritance — each implementation owns its constructor and dependencies explicitly rather than inheriting them. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Collapse the abstract EventEmitterService + three subclasses (AwsEventBridge, Logger, Noop) into a single service that conditionally creates the EventBridge client. No factory needed in the module — the service checks isLocalExecutor() at construction time and skips EventBridge in local mode (log-only). Remove LOCAL_EVENTS env var since the behavior is now derived from LOCAL_EXECUTOR. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Remove executorEnvVars helper — each implementation now calls apiAuth and appConfig directly. Three lines of logic don't warrant a shared function with three parameters. Rename executor.abstract.service.ts → executor.service.ts since it's an interface, not an abstract class. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Remove LOCAL_EXECUTOR_CALLBACK_BASE_URL — it was an unused escape hatch. The docker executor already passes --add-host=host.docker.internal:host-gateway, so the hardcoded host.docker.internal path works reliably. Also clean up getExternalApiConfig to use its local variables. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

… mode isLocalExecutor now explicitly checks for 'python' | 'docker' instead of truthy coercion. Bundle refetch on every request is now controlled by a dedicated BUNDLE_CACHE_DISABLED flag rather than piggybacking on executor mode — local dev doesn't inherently need cache-busting. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

…HANDLER Replace the single conditional EventEmitterService with three implementations (eventbridge, log, noop) behind an interface + DI token, selected by the LOCAL_EVENT_HANDLER env var. EventBridge is the default when unset (prod). This follows the pattern of specific feature flags rather than a bundled "local mode" — you can mix and match, e.g. run executor locally but still emit to EventBridge if you have AWS creds. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Remove TODO comments for executor interface approach (keeping inlined env vars) and docker networking (keeping host.docker.internal). Remove empty LOCAL_EXECUTOR= from .env.test and startJob TODO comment. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

- Route all config through AppConfigService instead of direct process.env reads (FileService, ExecutorAwsService, EventEmitterEventBridgeService, local executor services) - Guard FileService S3Mock override with isDevEnvironment() check so LOCAL_S3_ENDPOINT_URL cannot accidentally activate in prod - Add startup warnings when LOCAL_EXECUTOR or LOCAL_EVENT_EMITTER are set outside of development (NODE_ENV !== 'development') - Rename BUNDLE_CACHE_DISABLED → LOCAL_BUNDLE_CACHE_DISABLED for consistent LOCAL_ prefix convention - Rename LOCAL_EVENT_HANDLER → LOCAL_EVENT_EMITTER to match service name Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

NX_LOAD_DOT_ENV_FILES=false on test commands prevents NX from auto-loading .env into process.env. ConfigModule.forRoot({ ignoreEnvFile: true }) in the test harness prevents NestJS from loading .env when creating the test app. Tests now rely solely on .env.test, loaded by loadEnvVars() in global-setup. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

NestJS v10+ defaults to listening on localhost (127.0.0.1), which makes the API unreachable from the executor Docker container via host.docker.internal. Made-with: Cursor

- check all dependencies upfront (node, npm, docker, docker compose, git) and report all missing at once - non-destructive env file setup (skip if exists, backfill LOCAL_EVENT_EMITTER if missing from older .env) - replace fragile sleep 3 with docker compose --wait backed by a postgres healthcheck in docker-compose.yml - executor mode prompt with docker/python choice, detects current setting as default on re-run, updates api/.env in-place via sed - python mode: skip venv creation if exists, git pull bundles if already cloned - docker mode: always rebuild image to pick up code changes - remove npm install -g prisma; fix prisma scripts in package.json to use npx (local devDependency) - fix executor setup.py: explicit find_packages() so setuptools auto-discovery doesn't choke on the output/ directory - add executor/.dockerignore to exclude output/ and local-run/ from docker build context Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Extract the manual seed SQL from README into api/seed.sql with ON CONFLICT DO NOTHING on every insert (IdP config, partner, school year, bundles, partner-bundle associations). init.sh runs it automatically after migrations via docker compose exec. Simplify README setup instructions — the three manual steps (docker compose up, wait, run seed SQL) are now a single ./init.sh invocation. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Replace the hardcoded bundle list in seed.sql with a new sync-bundles.sh script that fetches the earthmover_edfi_bundles registry.json from GitHub and upserts all bundle paths into the DB. This keeps local dev in sync with the published registry without manual seed updates. - add app/api/sync-bundles.sh: standalone, idempotent bundle sync - remove bundle inserts from seed.sql, add \echo labels for clarity - init.sh: add curl/jq dep checks, call sync-bundles.sh after seed, cd to own directory for portability, fix summary ANSI rendering - update README bundle instructions to reference sync-bundles.sh Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

- add AWS_REGION to .env.test and .env.copyme so fresh clones don't fail on missing region when AWS SDK clients are constructed - add 23-24 and 24-25 school years to seed data - remove stale TODOs from integration tests - update init.sh summary: remove IdP config step, add dev/dev login - update README: S3Mock replaces storage directory for local files Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

edandylytics · 2026-02-27T18:07:34Z

Pushed some commits with the following goals:

Make it so the app can be fully set up, from a fresh clone, with a single command (./app/init.sh). This command should also be non-destructive so you can use it to refresh seed data and switch between local modes.
Make the local modes something the app is only aware of while bootstrapping. Deployed and local modes follow the same code paths -- everything is the same shape. Code related to local modes are isolated in separate services.
Allow each local mode dependency to be turned on/off independently. For example, I might need to develop locally against S3 or EventBridge or ECS while keeping the other dependencies in their local mode.

The big change was using S3Mock rather than the file system for managing files. This worked fine out of the box. There are alternatives (e.g. LocalStack) that also support AWS SDKs and we can switch if we discover we don't like S3Mock for some reason and that should be a minor change. I originally was using minio until I saw that it stopped being maintained just a couple weeks ago, and swithing to S3Mock was basically swapping out a few lines in docker compose.

There was also various cleanup stuff -- e.g. env vars from .env were leaking through when running tests, so I tightened up how the test harness loads env vars to just source from .env.test.

Because tests already mock the components that are injected differently in local vs. deployed mode, there didn't seem to be a meaningful set of tests for this, though I'm open to doing more here.

@bhaugeea I'd like your take. I think the main point of review should be that we're not introducing regressions or undue risk for deployed envs.

johncmerfeld · 2026-03-02T17:07:09Z

Confirmed that once I fixed my .env file this is working great for me locally

bhaugeea

I did some git stuff and copying and pasting to check the various code chunks that were more moved than edited, and they all check out. The modularity and DI work well and give a nice DX too. Both local options run fine on the example file that Andy gave me. I ran into two hiccups when running the init script, neither a bug per se but only a local case that wasn't explicitly handled (one was that the script asks which local mode you want to run, but doesn't edit the .env file based on that, and the other is a python thing commented below). Beyond that, a couple typos which can be ignored per your judgement, a typescript quirk which is easy to fix, and an executor logging question.

Only not ticking the Approve box because I don't know what the plan for approval is given the joint nature of the PR.

bhaugeea · 2026-02-27T22:33:08Z

app/api/src/earthbeam/executor/executor.service.ts

+import { Run } from '@prisma/client';
+
+export interface ExecutorService {
+  start(run: Run): Promise<void>;


I don't know why this is, but you seem to have to use the arrow function syntax if you want good enforcement of this type signature.

start: (run: Run) => Promise<void>;

bhaugeea · 2026-03-02T14:50:18Z

app/README.md


+#### 4. Configure an ODS
+
+You'll need a valid (even if nono-prod) place to send data in order to run local jobs.


nono typo

bhaugeea · 2026-03-02T16:24:53Z

app/api/src/earthbeam/executor/executor.local-docker.service.ts

+    });
+    proc.stderr?.on('data', (data) => {
+      this.logger.error(`Executor stderr: ${data}`);
+    });


It seems like all the executor logs - other than the ones from git that circumvent the main logger service - are going here instead of the stdout handler. Maybe there's a good reason for that but I thought I'd mention it.

Yeah, I think we might want to adjust when the executor sends logs to stdoout vs. stderr. I'd like to make these adjustments over time. We've talked about doing some re-work on the executor logs to make them easier to navigate and that'd be a good time to take a look at this.

bhaugeea · 2026-03-02T16:33:02Z

app/init.sh

+    fail "Python 3.10+ required, found ${py_version}"
+    exit 1
+  fi
+  ok "python3 ${py_version}"


I also had to install python3.12-venv, and had to manually delete the venv that was originally created because it lacked pip.

Added a check for this and I think also addressed the issue you and @johncmerfeld ran into with not having LOCAL_EXECUTOR in a existing .env

bhaugeea · 2026-03-02T17:06:47Z

app/README.md

+
+### Running the executor locally
+
+In deployed environments, the executor runs as a Task in Elastic Container Service (ECS). Locally, the app initiate the executor based on the `LOCAL_EXECUTOR` environment variable:


initiates typo

- check for ensurepip before creating python venv, with a clear message pointing to the python3.x-venv apt package - detect and recreate broken venvs that are missing pip - wait for postgres to accept host connections after docker compose up, avoiding ECONNREFUSED on migrations - backfill LOCAL_EXECUTOR in api/.env so the sed in step 7 always works - fix typos in README, fix executor service interface type Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

bhaugeea

Looks great to me!

johncmerfeld added 17 commits January 14, 2026 10:51

resolve branched changes

9aeac22

exec in broken state

c3e9bef

Executor: Fix check for unmatched student IDs (#6)

834f56d

resolve branched changes

856618f

exec in broken state

572d993

Executor: Fix check for unmatched student IDs (#6)

135e189

resolve conflict

afea8be

other exec commits

74116e1

other exec commits

0f2a092

cleanup

58954ea

more tweaks

e6d7f02

working version

88e1dc5

working local version

896e7cc

documentation

0d5ac64

env example

cf43c55

clean up local executor

b55ed8f

small exec cleanup

91cf3a9

johncmerfeld marked this pull request as ready for review January 15, 2026 00:03

ODS note

d253894

johncmerfeld requested review from edandylytics and jalvord1 January 15, 2026 00:05

johncmerfeld self-assigned this Jan 15, 2026

johncmerfeld and others added 8 commits January 15, 2026 08:20

fix job factory for tests

1e9a892

always refresh registry when running locally

3c6add2

document in-flight bundle development

dffedf8

more error handling around executor start, and safeguard pythono version

1cbe181

remove outdated and irrelevant local command

e96aa0e

Merge branch 'development' into local-runway

b4897c0

edandylytics and others added 2 commits February 26, 2026 10:38

edandylytics force-pushed the local-runway branch from 585d720 to 8190932 Compare February 26, 2026 17:02

edandylytics and others added 12 commits February 26, 2026 11:43

clean up resolved TODOs

6c09061

Remove TODO comments for executor interface approach (keeping inlined env vars) and docker networking (keeping host.docker.internal). Remove empty LOCAL_EXECUTOR= from .env.test and startJob TODO comment. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

listen on 0.0.0.0 when using local docker executor

df43ec7

NestJS v10+ defaults to listening on localhost (127.0.0.1), which makes the API unreachable from the executor Docker container via host.docker.internal. Made-with: Cursor

edandylytics requested a review from bhaugeea February 27, 2026 18:07

remove fe changes

9f1dbe6

bhaugeea reviewed Mar 2, 2026

View reviewed changes

edandylytics requested a review from bhaugeea March 2, 2026 18:11

bhaugeea approved these changes Mar 2, 2026

View reviewed changes

edandylytics merged commit 845b6e8 into development Mar 2, 2026
7 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Local Runway#9

Local Runway#9
edandylytics merged 42 commits intodevelopmentfrom
local-runway

johncmerfeld commented Jan 15, 2026 •

edited

Loading

Uh oh!

edandylytics commented Feb 27, 2026

Uh oh!

johncmerfeld commented Mar 2, 2026

Uh oh!

bhaugeea left a comment

Uh oh!

bhaugeea Feb 27, 2026

Uh oh!

bhaugeea Mar 2, 2026

Uh oh!

bhaugeea Mar 2, 2026

Uh oh!

edandylytics Mar 2, 2026

Uh oh!

bhaugeea Mar 2, 2026

Uh oh!

edandylytics Mar 2, 2026

Uh oh!

bhaugeea Mar 2, 2026

Uh oh!

bhaugeea left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants


		#### 4. Configure an ODS

		You'll need a valid (even if nono-prod) place to send data in order to run local jobs.


		### Running the executor locally

		In deployed environments, the executor runs as a Task in Elastic Container Service (ECS). Locally, the app initiate the executor based on the `LOCAL_EXECUTOR` environment variable:

Conversation

johncmerfeld commented Jan 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

How to try this

What this does

What's changed

For reviewers

Uh oh!

edandylytics commented Feb 27, 2026

Uh oh!

johncmerfeld commented Mar 2, 2026

Uh oh!

bhaugeea left a comment

Choose a reason for hiding this comment

Uh oh!

bhaugeea Feb 27, 2026

Choose a reason for hiding this comment

Uh oh!

bhaugeea Mar 2, 2026

Choose a reason for hiding this comment

Uh oh!

bhaugeea Mar 2, 2026

Choose a reason for hiding this comment

Uh oh!

edandylytics Mar 2, 2026

Choose a reason for hiding this comment

Uh oh!

bhaugeea Mar 2, 2026

Choose a reason for hiding this comment

Uh oh!

edandylytics Mar 2, 2026

Choose a reason for hiding this comment

Uh oh!

bhaugeea Mar 2, 2026

Choose a reason for hiding this comment

Uh oh!

bhaugeea left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

johncmerfeld commented Jan 15, 2026 •

edited

Loading