Monorepo restructure: add DS app, migrate web to workspaces, and integrate signed DS storage for backups/shares#212
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
审阅者指南将项目重构为一个由 Turbo 驱动的 monorepo,拆分为独立的 web 和 DS 应用;引入由专用的基于 Next.js + PostgreSQL 的 DS 服务器支撑的「已签名去中心化存储(DS)协议」;重构 ATProto 流程以优先使用 DS 进行备份和分享;并将共用的 UI/i18n/config 抽取到可复用的包中。 通过 web /api/blob 写入已签名 DS blob 的时序图sequenceDiagram
actor User
participant Browser
participant WebApi as Web_app_api_blob
participant AtprotoLib as Atproto_session_and_record
participant DsSigned as ds_signed_request
participant DsApi as DS_api_blob
participant Signature as DS_signature_verifier
participant DB as DS_PostgreSQL
User->>Browser: Save encrypted backup
Browser->>WebApi: POST /api/blob
WebApi->>AtprotoLib: getAtprotoSession()
alt ATProto session exists
WebApi->>AtprotoLib: getRecord(app.onecalendar.ds,self)
AtprotoLib-->>WebApi: ds endpoint URL
alt ds endpoint is configured
WebApi->>DsSigned: signedDsFetch(session, ds, "/api/blob", POST, {encrypted_data, iv, timestamp})
DsSigned->>DsSigned: createPayload(method,path,timestamp,body)
DsSigned->>DsSigned: signPayload(payload, dpopPrivateKeyPem)
DsSigned->>DsApi: POST /api/blob
Note over DsSigned,DsApi: Sends x-app-token, x-did, x-timestamp,<br/>x-signature, x-dpop-jwk
DsApi->>Signature: requireSignedRequest(request, rawBody)
Signature->>Signature: validate x-app-token
Signature->>Signature: rebuild payload and verify signature
Signature-->>DsApi: did (on success)
DsApi->>DB: upsert calendar_backups by user_id=did
DB-->>DsApi: ok
DsApi-->>DsSigned: 200 {success:true}
DsSigned-->>WebApi: 200 OK
WebApi-->>Browser: 200 {success:true, backend: "ds", ds}
else ds endpoint missing
WebApi-->>Browser: 400 {error:"ATProto DS is not configured"}
end
else no ATProto session
WebApi->>DB: use legacy local backup flow
DB-->>WebApi: ok
WebApi-->>Browser: 200 {success:true, backend:"db"}
end
通过 /api/ds/migrate 和 DS migrate 接口进行 DS 迁移的时序图sequenceDiagram
actor User
participant Browser
participant WebMigrate as Web_api_ds_migrate
participant Atproto as Atproto_session_and_record
participant DsSigned as ds_signed_request
participant DsFrom as Source_DS_server
participant DsTo as Target_DS_server
User->>Browser: Click "migrate" with target DS URL
Browser->>WebMigrate: POST /api/ds/migrate {toDs}
WebMigrate->>Atproto: getAtprotoSession()
alt no ATProto session
WebMigrate-->>Browser: 401 {error:"ATProto login required"}
else ATProto session exists
WebMigrate->>Atproto: getRecord(app.onecalendar.ds,self)
Atproto-->>WebMigrate: current ds (fromDs)
alt fromDs missing
WebMigrate-->>Browser: 400 {error:"No source DS record found"}
else fromDs present and != toDs
Note over WebMigrate,DsFrom: Export from source DS
WebMigrate->>DsSigned: signedDsFetch(session, fromDs, "/api/migrate/export", POST, {did})
DsSigned->>DsFrom: POST /api/migrate/export (signed)
DsFrom-->>DsSigned: 200 {backups, shares}
DsSigned-->>WebMigrate: payload
Note over WebMigrate,DsTo: Import into target DS
WebMigrate->>DsSigned: signedDsFetch(session, toDs, "/api/migrate/import", POST, payload)
DsSigned->>DsTo: POST /api/migrate/import (signed)
DsTo-->>DsSigned: 200 {success:true}
DsSigned-->>WebMigrate: ok
Note over WebMigrate,DsFrom: Cleanup on source DS
WebMigrate->>DsSigned: signedDsFetch(session, fromDs, "/api/migrate/cleanup", POST, {did})
DsSigned->>DsFrom: POST /api/migrate/cleanup (signed)
DsFrom-->>DsSigned: 200 {success:true}
DsSigned-->>WebMigrate: ok
WebMigrate->>Atproto: putRecord(app.onecalendar.ds,self,{ds:toDs,updatedAt})
Atproto-->>WebMigrate: ok
WebMigrate-->>Browser: 200 {success:true, fromDs, toDs, switched:true}
end
end
DS PostgreSQL 表的实体关系图erDiagram
calendar_backups {
SERIAL id
TEXT user_id
TEXT encrypted_data
TEXT iv
BIGINT timestamp
TIMESTAMPTZ updated_at
}
shares {
SERIAL id
TEXT user_id
TEXT share_id
TEXT data
BIGINT timestamp
TIMESTAMPTZ created_at
}
users {
TEXT did
}
users ||--o{ calendar_backups : has_backup
users ||--o{ shares : has_share
shares {
%% logical indexes
}
calendar_backups {
%% logical index on user_id (unique per user)
}
文件级变更
提示与命令与 Sourcery 交互
自定义使用体验访问你的 控制面板 来:
获取帮助Original review guide in EnglishReviewer's GuideRestructures the project into a Turbo-powered monorepo with separate web and DS apps, introduces a signed decentralized storage (DS) protocol backed by a dedicated Next.js DS server with PostgreSQL, refactors ATProto flows to prefer DS for backups and shares, and extracts shared UI/i18n/config into reusable packages. Sequence diagram for signed DS blob write via web /api/blobsequenceDiagram
actor User
participant Browser
participant WebApi as Web_app_api_blob
participant AtprotoLib as Atproto_session_and_record
participant DsSigned as ds_signed_request
participant DsApi as DS_api_blob
participant Signature as DS_signature_verifier
participant DB as DS_PostgreSQL
User->>Browser: Save encrypted backup
Browser->>WebApi: POST /api/blob
WebApi->>AtprotoLib: getAtprotoSession()
alt ATProto session exists
WebApi->>AtprotoLib: getRecord(app.onecalendar.ds,self)
AtprotoLib-->>WebApi: ds endpoint URL
alt ds endpoint is configured
WebApi->>DsSigned: signedDsFetch(session, ds, "/api/blob", POST, {encrypted_data, iv, timestamp})
DsSigned->>DsSigned: createPayload(method,path,timestamp,body)
DsSigned->>DsSigned: signPayload(payload, dpopPrivateKeyPem)
DsSigned->>DsApi: POST /api/blob
Note over DsSigned,DsApi: Sends x-app-token, x-did, x-timestamp,<br/>x-signature, x-dpop-jwk
DsApi->>Signature: requireSignedRequest(request, rawBody)
Signature->>Signature: validate x-app-token
Signature->>Signature: rebuild payload and verify signature
Signature-->>DsApi: did (on success)
DsApi->>DB: upsert calendar_backups by user_id=did
DB-->>DsApi: ok
DsApi-->>DsSigned: 200 {success:true}
DsSigned-->>WebApi: 200 OK
WebApi-->>Browser: 200 {success:true, backend: "ds", ds}
else ds endpoint missing
WebApi-->>Browser: 400 {error:"ATProto DS is not configured"}
end
else no ATProto session
WebApi->>DB: use legacy local backup flow
DB-->>WebApi: ok
WebApi-->>Browser: 200 {success:true, backend:"db"}
end
Sequence diagram for DS migration via /api/ds/migrate and DS migrate endpointssequenceDiagram
actor User
participant Browser
participant WebMigrate as Web_api_ds_migrate
participant Atproto as Atproto_session_and_record
participant DsSigned as ds_signed_request
participant DsFrom as Source_DS_server
participant DsTo as Target_DS_server
User->>Browser: Click "migrate" with target DS URL
Browser->>WebMigrate: POST /api/ds/migrate {toDs}
WebMigrate->>Atproto: getAtprotoSession()
alt no ATProto session
WebMigrate-->>Browser: 401 {error:"ATProto login required"}
else ATProto session exists
WebMigrate->>Atproto: getRecord(app.onecalendar.ds,self)
Atproto-->>WebMigrate: current ds (fromDs)
alt fromDs missing
WebMigrate-->>Browser: 400 {error:"No source DS record found"}
else fromDs present and != toDs
Note over WebMigrate,DsFrom: Export from source DS
WebMigrate->>DsSigned: signedDsFetch(session, fromDs, "/api/migrate/export", POST, {did})
DsSigned->>DsFrom: POST /api/migrate/export (signed)
DsFrom-->>DsSigned: 200 {backups, shares}
DsSigned-->>WebMigrate: payload
Note over WebMigrate,DsTo: Import into target DS
WebMigrate->>DsSigned: signedDsFetch(session, toDs, "/api/migrate/import", POST, payload)
DsSigned->>DsTo: POST /api/migrate/import (signed)
DsTo-->>DsSigned: 200 {success:true}
DsSigned-->>WebMigrate: ok
Note over WebMigrate,DsFrom: Cleanup on source DS
WebMigrate->>DsSigned: signedDsFetch(session, fromDs, "/api/migrate/cleanup", POST, {did})
DsSigned->>DsFrom: POST /api/migrate/cleanup (signed)
DsFrom-->>DsSigned: 200 {success:true}
DsSigned-->>WebMigrate: ok
WebMigrate->>Atproto: putRecord(app.onecalendar.ds,self,{ds:toDs,updatedAt})
Atproto-->>WebMigrate: ok
WebMigrate-->>Browser: 200 {success:true, fromDs, toDs, switched:true}
end
end
Entity relationship diagram for DS PostgreSQL tableserDiagram
calendar_backups {
SERIAL id
TEXT user_id
TEXT encrypted_data
TEXT iv
BIGINT timestamp
TIMESTAMPTZ updated_at
}
shares {
SERIAL id
TEXT user_id
TEXT share_id
TEXT data
BIGINT timestamp
TIMESTAMPTZ created_at
}
users {
TEXT did
}
users ||--o{ calendar_backups : has_backup
users ||--o{ shares : has_share
shares {
%% logical indexes
}
calendar_backups {
%% logical index on user_id (unique per user)
}
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
There was a problem hiding this comment.
Hey - 我发现了 1 个问题,并给出了一些整体层面的反馈:
- 新增的 ATProto DS 设置和 DS 引导对话框(例如在
Settings和首页对话框中)引入了若干硬编码的英文字符串;由于应用的其他部分都通过 i18n 层来处理文案,建议也把这些字符串接入translations,以便 DS 相关的 UX 同样可以本地化。 - 在
apps/ds/lib/signature.ts中,每个已签名请求都会访问https://plc.directory来解析 DID 密钥,而且错误分类是通过对错误消息进行字符串匹配来完成的;如果增加一个短时内存缓存来存储 DID 公钥,并使用显式的错误类型,将可以减少延迟,并让状态处理的方式更加稳健。 - DS 应用同时在
devDependencies中声明了@types/pg,又定义了一个自定义的全局模块apps/ds/types/pg.d.ts;删除自定义声明,改为依赖官方的@types/pg类型会更简洁,也更不容易出错。
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- 新增的 ATProto DS 设置和 DS 引导对话框(例如在 `Settings` 和首页对话框中)引入了若干硬编码的英文字符串;由于应用的其他部分都通过 i18n 层来处理文案,建议也把这些字符串接入 `translations`,以便 DS 相关的 UX 同样可以本地化。
- 在 `apps/ds/lib/signature.ts` 中,每个已签名请求都会访问 `https://plc.directory` 来解析 DID 密钥,而且错误分类是通过对错误消息进行字符串匹配来完成的;如果增加一个短时内存缓存来存储 DID 公钥,并使用显式的错误类型,将可以减少延迟,并让状态处理的方式更加稳健。
- DS 应用同时在 `devDependencies` 中声明了 `@types/pg`,又定义了一个自定义的全局模块 `apps/ds/types/pg.d.ts`;删除自定义声明,改为依赖官方的 `@types/pg` 类型会更简洁,也更不容易出错。
## Individual Comments
### Comment 1
<location path="apps/web/app/(app)/app/page.tsx" line_range="88-86" />
<code_context>
- return false
- }, [minimumWaitDone, hasSessionCookie, isLoaded, isSignedIn, dbReady])
-
- if (shouldShowAuthWait) {
- return <AuthWaitingLoading />
- }
</code_context>
<issue_to_address>
**issue (bug_risk):** 没有 DS endpoint 的 ATProto 用户会卡在加载界面,永远看不到 DS 配置对话框。
由于 `shouldShowAuthWait` 只有在 `dbReady` 为 `true` 时才会停止返回 `true`,因此一个已经登录 ATProto 但尚未配置 DS 的用户会一直停留在加载状态:`checkDbDataReady` 会打开 DS 对话框(`setDsDialogOpen(true)`),但从不会把 `dbReady` 设为 `true`,所以 `<AuthWaitingLoading />` 会一直被渲染,对话框也就始终不会显示出来。
要修复这个问题,可以选择以下任一方式:
- 调整 `shouldShowAuthWait`,使得 `atprotoSignedIn && !atprotoDs` 不会阻塞主界面(及对话框)的渲染,或者
- 一旦检测到 `sessionData.signedIn && !dsData.ds`,就将 `dbReady` 设为 `true`,以便主树(包括对话框)可以渲染。
例如:
```ts
const shouldShowAuthWait = useMemo(() => {
if (!minimumWaitDone) return true;
if (hasSessionCookie && !isLoaded) return true;
if (isSignedIn && !dbReady && !(atprotoSignedIn && !atprotoDs)) return true;
return false;
}, [minimumWaitDone, hasSessionCookie, isLoaded, isSignedIn, dbReady, atprotoSignedIn, atprotoDs]);
```
</issue_to_address>帮我变得更有用!请在每条评论上点 👍 或 👎,我会根据你的反馈改进后续的评审。
Original comment in English
Hey - I've found 1 issue, and left some high level feedback:
- The new ATProto DS settings and DS-onboarding dialog (e.g. in
Settingsand the home page dialog) introduce several hard-coded English strings; since the rest of the app uses the i18n layer, consider wiring these throughtranslationsso DS UX is localized as well. - In
apps/ds/lib/signature.ts, every signed request hitshttps://plc.directoryto resolve the DID key and error classification is done by string-matching error messages; adding a short-lived in-memory cache for DID public keys and using explicit error types would reduce latency and make status handling less brittle. - The DS app declares both
@types/pgindevDependenciesand a custom ambientapps/ds/types/pg.d.tsmodule; it would be cleaner and less error-prone to remove the custom declaration and rely on the official@types/pgtypes.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- The new ATProto DS settings and DS-onboarding dialog (e.g. in `Settings` and the home page dialog) introduce several hard-coded English strings; since the rest of the app uses the i18n layer, consider wiring these through `translations` so DS UX is localized as well.
- In `apps/ds/lib/signature.ts`, every signed request hits `https://plc.directory` to resolve the DID key and error classification is done by string-matching error messages; adding a short-lived in-memory cache for DID public keys and using explicit error types would reduce latency and make status handling less brittle.
- The DS app declares both `@types/pg` in `devDependencies` and a custom ambient `apps/ds/types/pg.d.ts` module; it would be cleaner and less error-prone to remove the custom declaration and rely on the official `@types/pg` types.
## Individual Comments
### Comment 1
<location path="apps/web/app/(app)/app/page.tsx" line_range="88-86" />
<code_context>
- return false
- }, [minimumWaitDone, hasSessionCookie, isLoaded, isSignedIn, dbReady])
-
- if (shouldShowAuthWait) {
- return <AuthWaitingLoading />
- }
</code_context>
<issue_to_address>
**issue (bug_risk):** ATProto users without a DS endpoint get stuck on the loading screen and never see the DS configuration dialog.
Because `shouldShowAuthWait` only stops returning `true` once `dbReady` is `true`, an ATProto-signed-in user with no DS configured never leaves the loading state: `checkDbDataReady` opens the DS dialog (`setDsDialogOpen(true)`) but never sets `dbReady` to `true`, so `<AuthWaitingLoading />` is always rendered and the dialog is never shown.
To fix this, either:
- Adjust `shouldShowAuthWait` so `atprotoSignedIn && !atprotoDs` does not block rendering the main UI (and dialog), or
- Set `dbReady` to `true` once you detect `sessionData.signedIn && !dsData.ds`, so the main tree (including the dialog) can render.
For example:
```ts
const shouldShowAuthWait = useMemo(() => {
if (!minimumWaitDone) return true;
if (hasSessionCookie && !isLoaded) return true;
if (isSignedIn && !dbReady && !(atprotoSignedIn && !atprotoDs)) return true;
return false;
}, [minimumWaitDone, hasSessionCookie, isLoaded, isSignedIn, dbReady, atprotoSignedIn, atprotoDs]);
```
</issue_to_address>Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
| new CustomEvent("atproto-ds-updated", { | ||
| detail: { ds: dsData.ds || null }, | ||
| }), | ||
| ) |
There was a problem hiding this comment.
issue (bug_risk): 没有 DS endpoint 的 ATProto 用户会卡在加载界面,永远看不到 DS 配置对话框。
由于 shouldShowAuthWait 只有在 dbReady 为 true 时才会停止返回 true,因此一个已经登录 ATProto 但尚未配置 DS 的用户会一直停留在加载状态:checkDbDataReady 会打开 DS 对话框(setDsDialogOpen(true)),但从不会把 dbReady 设为 true,所以 <AuthWaitingLoading /> 会一直被渲染,对话框也就始终不会显示出来。
要修复这个问题,可以选择以下任一方式:
- 调整
shouldShowAuthWait,使得atprotoSignedIn && !atprotoDs不会阻塞主界面(及对话框)的渲染,或者 - 一旦检测到
sessionData.signedIn && !dsData.ds,就将dbReady设为true,以便主树(包括对话框)可以渲染。
例如:
const shouldShowAuthWait = useMemo(() => {
if (!minimumWaitDone) return true;
if (hasSessionCookie && !isLoaded) return true;
if (isSignedIn && !dbReady && !(atprotoSignedIn && !atprotoDs)) return true;
return false;
}, [minimumWaitDone, hasSessionCookie, isLoaded, isSignedIn, dbReady, atprotoSignedIn, atprotoDs]);Original comment in English
issue (bug_risk): ATProto users without a DS endpoint get stuck on the loading screen and never see the DS configuration dialog.
Because shouldShowAuthWait only stops returning true once dbReady is true, an ATProto-signed-in user with no DS configured never leaves the loading state: checkDbDataReady opens the DS dialog (setDsDialogOpen(true)) but never sets dbReady to true, so <AuthWaitingLoading /> is always rendered and the dialog is never shown.
To fix this, either:
- Adjust
shouldShowAuthWaitsoatprotoSignedIn && !atprotoDsdoes not block rendering the main UI (and dialog), or - Set
dbReadytotrueonce you detectsessionData.signedIn && !dsData.ds, so the main tree (including the dialog) can render.
For example:
const shouldShowAuthWait = useMemo(() => {
if (!minimumWaitDone) return true;
if (hasSessionCookie && !isLoaded) return true;
if (isSignedIn && !dbReady && !(atprotoSignedIn && !atprotoDs)) return true;
return false;
}, [minimumWaitDone, hasSessionCookie, isLoaded, isSignedIn, dbReady, atprotoSignedIn, atprotoDs]);There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 7c95f68c5b
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| `${ds.replace(/\/$/, "")}/api/share/${encodeURIComponent(id)}`, | ||
| { | ||
| cache: "no-store", | ||
| headers: { | ||
| "x-app-token": appToken, |
There was a problem hiding this comment.
Validate DS URL before sending x-app-token
This public endpoint fetches ds from a handle-controlled ATProto record and forwards x-app-token to that URL without any allowlist or origin validation. An attacker can set their DS record to an attacker host and call /api/share/public to exfiltrate DS_APP_TOKEN, breaking the DS trust boundary (and enabling server-side requests to arbitrary/internal hosts).
Useful? React with 👍 / 👎.
| "SELECT user_id, share_id, data, timestamp FROM shares WHERE share_id = $1 LIMIT 1", | ||
| [shareId], |
There was a problem hiding this comment.
Scope DS share lookup to owner DID
This query looks up shares only by share_id, but writes are keyed by (user_id, share_id), so identical IDs from different users on the same DS are valid. In that case, public reads can return the wrong row, allowing one user to shadow/spoof another user's shared link data.
Useful? React with 👍 / 👎.
| const cleanupRes = await signedDsFetch({ | ||
| session: atproto, | ||
| ds: fromDs, | ||
| path: "/api/migrate/cleanup", | ||
| method: "POST", |
There was a problem hiding this comment.
Update DS record before source cleanup
The migration flow deletes source DS data before updating the ATProto DS pointer. If the subsequent putRecord call fails (e.g., transient PDS/network failure), the user record still points to fromDs even though data there was already removed, causing apparent data loss/unavailability until manual repair.
Useful? React with 👍 / 👎.
Motivation
apps/webandapps/dsworkspaces to enable a dedicated decentralized storage (DS) server and shared packages.Description
apps/dsNext.js app implementing DS REST endpoints and DB access includingapi/blob,api/share,api/migrate/{export,import,cleanup}, table creation (ensureTables) and a signature verification layer inlib/signature.tsthat validates signed requests from ATProto sessions.apps/webworkspace and refactor the original web app intoapps/web/*, adding DS integration:api/ds/config,api/ds/migrate,api/ds/proxy,lib/ds-signed-request.ts,lib/ds-client.ts, and changes to blob/share APIs to prefer DS when an ATProto DS is configured.packages/uiand i18n intopackages/i18n, addpackages/configfor shared config, update import paths and component references, and addpackages/ui/src/utils.tshelper.turbo.json, changepackage.jsonto workspaces and monorepo scripts, addapps/*andpackages/*manifests, updateREADME.mdfor monorepo usage, and adjust.gitignoreand other build configs.Testing
README.md) to validate project layout and generators completed without file-path errors.DS_APP_TOKEN,POSTGRES_URL, and ATProto session secrets.Codex Task
Summary by Sourcery
将项目转换为 Turborepo 单体仓库(monorepo)结构,拆分为独立的 web 应用和去中心化存储(DS)应用,并将 web 应用接入使用签名 DS 后端的 ATProto 用户备份和分享功能。
New Features:
Enhancements:
@repo/ui包,并将 i18n 的生成与运行时逻辑集中到@repo/i18n包中。localStorage读取位置设置,而不再依赖应用特定的 hooks。@repo/config包统一 TS/Next/tailwind/postcss 配置,并相应地对各应用的 TS 配置进行对齐。Build:
apps/*和packages/*工作区结构,并在根目录的package.json中使用 Turbo 驱动开发和构建脚本。Documentation:
Original summary in English
Summary by Sourcery
Convert the project to a Turborepo monorepo with separate web and decentralized storage (DS) apps and wire the web app to use a signed DS backend for ATProto user backups and shares.
New Features:
Enhancements:
Build:
Documentation: