feat(memory): managed auto-memory and auto-dream system#3087
feat(memory): managed auto-memory and auto-dream system#3087LaZzyMan wants to merge 41 commits intoQwenLM:mainfrom
Conversation
Feature 3 - Memory Saved Notification: - Add HistoryItemMemorySaved type to types.ts - Create MemorySavedMessage component for rendering '● Saved/Updated N memories' - In useGeminiStream: detect in-turn memory writes via mapToDisplay's memoryWriteCount field and emit 'memory_saved' history item after turn - In client.ts: capture background dream/extract promises and expose via consumePendingMemoryTaskPromises(); useGeminiStream listens post-turn and emits 'Updated N memories' notification for background tasks Feature 4 - Memory Count Badge: - Add isMemoryOp field to IndividualToolCallDisplay - Add memoryWriteCount/memoryReadCount to HistoryItemToolGroup - Add detectMemoryOp() in useReactToolScheduler using isAutoMemPath - ToolGroupMessage renders '● Recalled N memories, Wrote N memories' badge at the top of tool groups that touch memory files Fix: process.env bracket-access in paths.ts (noPropertyAccessFromIndexSignature) Fix: MemoryDialog.test.tsx mock useSettings to satisfy SettingsProvider requirement
…ps, fix MEMORY.md path
Problem 1 - Auto-approve memory file operations:
- write-file.ts: getDefaultPermission() checks isAutoMemPath; returns 'allow'
for managed auto-memory files, 'ask' for all other files
- edit.ts: same pattern
Problem 2 - Feature 4 UX: collapse memory-only tool groups:
- ToolGroupMessage: detect when all tool calls have isMemoryOp set (pure memory
group) and all are complete; render compact '● Recalled/Wrote N memories
(ctrl+o to expand)' instead of individual tool call rows
- ctrl+o toggles expand/collapse when isFocused and group is memory-only
- Mixed groups (memory + other tools) keep badge-at-top behaviour
- Expanded state shows individual tool calls with '● Memory operations
(ctrl+o to collapse)' header
Problem 3 - MEMORY.md path mismatch:
- prompt.ts: Step 2 now references full absolute path ${memoryDir}/MEMORY.md
so the model writes to the correct location inside the memory directory,
not to the parent project directory
Fix tests:
- write-file.test.ts: add getProjectRoot to mockConfigInternal
- prompt.test.ts: update assertion to match full-path section header
…ool detection - Remove duplicate 'Saved N memories' notification: the tool group badge already shows 'Wrote N memories'; the separate HistoryItemMemorySaved addItem after onComplete was double-counting. Keep only the background-task path (consumePendingMemoryTaskPromises). - Remove ctrl+o expand: Ink's Static area freezes items on first render and cannot respond to user input. useInput/useState(isExpanded) in a Static item is a no-op. Removed the dead code; memory-only groups now always render as the compact summary (no fake interactive hint). - Fix Edit tool detection: detectMemoryOp was checking for 'edit_file' but the real tool name constant is 'edit'. Also removed non-existent 'create_file' (write_file covers all writes). Now editing MEMORY.md is correctly identified as a memory write op, collapses to 'Wrote N memories', and is auto-approved.
…background agent The previous implementation ran an AgentHeadless background agent that could take 5+ minutes with zero UI feedback — user saw a blank screen for the entire duration and then at most one line of text. Fix: /dream now returns submit_prompt with the consolidation task prompt so it runs as a regular AI conversation turn. Tool calls (read_file, write_file, edit, grep_search, list_directory, glob) are immediately visible as collapsed tool groups as the model works through the memory files — identical UX to Claude Code. Also export buildConsolidationTaskPrompt from dreamAgentPlanner so dreamCommand can reuse the same detailed consolidation prompt that was already written.
Add getMemoryBaseDir() to getDefaultPermission() allow list in ls.ts, glob.ts, and grep.ts — mirrors the existing pattern in read-file.ts. Without this, ListFiles/Glob/Grep on ~/.qwen/* would trigger an approval dialog, blocking /dream at its very first step.
Match Claude Code's headless-agent intent: background memory agents must never block on interactive permission prompts. Wrap background runtime config so getApprovalMode() returns YOLO, ensuring any ask decision is auto-approved instead of hanging forever. Add regression test covering the wrapped approval mode.
Make managed auto-memory extraction follow the Claude Code architecture: background extraction now uses a forked agent to read/write memory files directly, instead of planning patches and applying them with a separate filesystem pipeline. Keep the old patch/model path only as fallback if the forked agent fails. Add regression tests covering the new execution path and tool whitelist.
Delete the old patch/model/heuristic extraction path entirely. Managed auto-memory extract now runs only through the forked-agent execution flow, with no planner/apply fallback stages remaining. Also remove obsolete exports/tests and update scheduler/integration coverage to use the forked-agent-only architecture.
meta.json, extract-cursor.json, and consolidation.lock are internal bookkeeping files, not user-visible memories. Move them one level up to the project state dir (parent of memory/) so that the memory/ directory contains only MEMORY.md and topic files, matching the clean layout of the upstream reference implementation. Add getAutoMemoryProjectStateDir() helper in paths.ts and update the three path accessors + store.test.ts path assertions accordingly.
The /dream command submits a prompt to the main agent (submit_prompt), which writes memory files directly. Because it bypasses dreamScheduler, meta.json was never updated and /memory always showed 'never'. Fix by: - Exporting writeDreamManualRunToMetadata() from dream.ts - Adding optional onComplete callback to SubmitPromptActionReturn and SubmitPromptResult (types.ts / commands/types.ts) - Propagating onComplete through slashCommandProcessor.ts - Firing onComplete after turn completion in useGeminiStream.ts - Providing the callback in dreamCommand.ts to write lastDreamAt
…y mode --global/--project are legacy save_memory tool concepts. In managed auto-memory mode the forked agent decides the appropriate type (user/feedback/project/reference) based on the content of the fact. Also improve the prompt wording to explicitly ask the agent to choose the correct type, reducing the tendency to default to 'project'.
Subscribe to getManagedAutoMemoryDreamTaskRegistry() in Footer via a useDreamRunning() hook. While any dream task for the current project is pending or running, display '✦ dreaming' in the right section of the footer bar, between Debug Mode and context usage.
… patterns
Five improvements based on Claude Code parity audit:
1. Memoize getAutoMemoryRoot (paths.ts)
- Add _autoMemoryRootCache Map, keyed by projectRoot
- findCanonicalGitRoot() walks the filesystem per call; memoize avoids
repeated git-tree traversal on hot-path schedulers/scanners
- Expose clearAutoMemoryRootCache() for test teardown
2. Lock file stores PID + isProcessRunning reclaim (dreamScheduler.ts)
- acquireDreamLock() writes process.pid to the lock file body
- lockExists() reads PID and calls process.kill(pid, 0); dead/missing
PID reclaims the lock immediately instead of waiting 2h
- Stale threshold reduced to 1h (PID-reuse guard, same as CC)
3. Session scan throttle (dreamScheduler.ts)
- Add SESSION_SCAN_INTERVAL_MS = 10min (same as CC)
- Add lastSessionScanAt Map<projectRoot, number> to ManagedAutoMemoryDreamRuntime
- When time-gate passes but session-gate doesn't, throttle prevents
re-scanning the filesystem on every user turn
4. mtime-based session counting (dreamScheduler.ts)
- Replace fragile recentSessionIdsSinceDream Set in meta.json with
filesystem mtime scan (listSessionsTouchedSince)
- Mirrors Claude Code's listSessionsTouchedSince: reads session JSONL
files from Storage.getProjectDir()/chats/, filters by mtime > lastDreamAt
- Immune to meta.json corruption/loss; no per-turn metadata write
- ManagedAutoMemoryDreamRuntime accepts injectable SessionScannerFn
for clean unit testing without real session files
5. Extraction mutual exclusion extended to write_file/edit (extractScheduler.ts)
- historySliceUsesMemoryTool() now checks write_file/edit/replace/create_file
tool calls whose file_path is within isAutoMemPath()
- Previously only detected save_memory; missed direct file writes by
the main agent, causing redundant background extraction
…lify /forget - Add docs/users/features/memory.md: comprehensive user-facing guide covering QWEN.md instructions, auto-memory behaviour, all memory commands, and troubleshooting; replaces the placeholder auto-memory.md - Update docs/users/features/_meta.ts: rename entry auto-memory → memory - Update docs/users/features/commands.md: add /init, /remember, /forget, /dream rows; fix /memory description; remove /init duplicate - Update docs/users/configuration/settings.md: add memory.* settings section (enableManagedAutoMemory, enableManagedAutoDream) between tools and permissions - Remove /forget --apply flag: preview-then-apply flow replaced with direct deletion; update forgetCommand.ts, en.js, zh.js accordingly - Add all auto-memory i18n keys to de, ja, pt, ru locales (18 keys each): Open auto-memory folder, Auto-memory/Auto-dream status lines, never/on/off, ✦ dreaming, /forget and /remember usage strings, all managed-memory messages - Remove dead save_memory branch from extractScheduler.partWritesToMemory() - Add ✦ dreaming indicator to Footer.tsx with i18n; fix Footer.test.tsx mocks - Refactor MemoryDialog.tsx auto-dream status line to use i18n - Remove save_memory tool (memoryTool.ts/test); clean up webui references - Add extractionPlanner.ts, const.ts and associated tests - Delete stale docs/users/configuration/memory.md and docs/developers/tools/memory.md (content superseded)
Code Review: Auto-Memory System (PR #3087)📋 Review SummaryThis PR introduces a comprehensive auto-memory system that automatically extracts durable facts from conversations and consolidates them through periodic "dream" passes. The implementation spans ~40 new files (~5,000+ lines) in 🔍 General FeedbackPositive aspects:
Architectural observations:
Concerns:
🎯 Specific Feedback🔴 CriticalSecurity & Permissions:
Data Loss Risk:
🟡 HighReliability & Error Handling:
Testing Gaps:
Performance:
🟢 MediumCode Quality:
Type Safety:
🔵 LowDocumentation:
Naming & Style:
Developer Experience:
✅ Highlights
SummaryThis is an ambitious and well-architected PR that brings Qwen Code to parity with Claude Code's auto-memory capabilities. The core design is sound, but the critical security issues around YOLO mode enforcement and potential data loss in dream deduplication must be addressed before merge. The high-priority reliability improvements (atomic writes, lock recovery) should be implemented to prevent user data loss in edge cases. Recommended actions:
|
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| const indentedMatch = rawLine.match( | ||
| /^\s{2,}(?:[-*]\s+)?(Why|How to apply|How_to_apply):\s*(.+)$/i, | ||
| ); |
There was a problem hiding this comment.
Fixed in 601d1a8. Changed \s{2,}(?:[-*]\s+)? to [\t ]{2,}(?:[-*][\t ]+)? — using explicit character classes instead of \s eliminates the overlapping quantifier that caused the polynomial backtracking on tab-heavy strings.
| const topLevelMatch = trimmed.match( | ||
| /^(?:\*\*)?(Why|How to apply|How_to_apply)(?:\*\*)?:\s*(.+)$/i, | ||
| ); |
There was a problem hiding this comment.
Fixed in 601d1a8. Changed :\s*(.+)$ to :[ \t]*(\S.*)$ — requiring the value to start with a non-whitespace character removes the ambiguity between \s* and .+ that allowed exponential backtracking on strings like why:\t\t\t....
- fix(read-file): narrow auto-allow from getMemoryBaseDir() (~/.qwen) to
isAutoMemPath(projectRoot) to prevent exposing settings.json / OAuth
credentials without user approval (wenshao review)
- fix(forget): per-entry deletion instead of whole-file unlink
- assign stable per-entry IDs (relativePath:index for multi-entry files)
so the model can target individual entries without removing siblings
- rewrite file keeping unmatched entries; only unlink when file becomes
empty (wenshao review)
- fix(entries): round-trip correctness for multi-entry new-format bodies
- parseAutoMemoryEntries: plain-text line closes current entry and opens
a new one (was silently ignored when current was already set)
- renderAutoMemoryBody: emit blank line between adjacent entries so the
parser can detect entry boundaries on re-read (wenshao review)
- fix(entries): resolve two CodeQL polynomial-regex alerts
- indentedMatch: \s{2,}(?:[-*]\s+)? → [\t ]{2,}(?:[-*][\t ]+)?
- topLevelMatch: :\s*(.+)$ → :[ \t]*(\S.*)$
(github-advanced-security review)
- fix(scan.test): use forward-slash literal for relativePath expectation
since listMarkdownFiles() normalises all separators to '/' on all
platforms including Windows
|
All critical findings from @wenshao and @github-advanced-security have been addressed in commit 601d1a8:
Ready for re-review. |
Using path.relative() instead of string startsWith() is more robust across platforms — it correctly handles Windows path-separator differences and avoids potential edge cases where a path prefix match could succeed on non-separator boundaries. Addresses github-actions review item 3 (PR #3087).
|
Fixed in 5cd4908. Replaced const rel = path.relative(memRoot, normalizedPath);
return rel === '' || (!rel.startsWith('..') && !path.isAbsolute(rel));This handles Windows path-separator differences and removes the edge case where a prefix match could succeed on non-separator boundaries. |
3 participants
TLDR
This PR introduces a fully managed auto-memory system — the model now learns from conversations automatically and maintains a persistent, queryable memory store across sessions, without any manual user action.
Two new autonomous background processes drive this:
~/.qwen/projects/<canonical-git-root>/memory/Both processes run entirely out of band; the main conversation is never blocked. All worktrees of the same repository share one memory directory.
What changed
Core — new
packages/core/src/memory/module (~40 files, ~5 000 lines)paths.ts,const.ts,types.tsstore.ts,entries.ts,indexer.tsscan.ts,recall.ts,relevanceSelector.tsextract.ts,extractScheduler.ts,extractionAgentPlanner.tsdream.ts,dreamScheduler.ts,dreamAgentPlanner.tsforget.ts,governance.ts,status.ts/forgetsemantic matching, governance queries, status reportingprompt.tsstate.ts,memoryAge.tsCore — background task runtime (
packages/core/src/background/)New generic runtime for running tasks out of band without blocking the main loop:
taskRegistry.ts— register/lookup named taskstaskScheduler.ts— time-based trigger (checks every minute)taskDrainer.ts— graceful shutdown: drains running tasks before exitCLI — new and changed commands
/memory/remember/forget <query>/dreamUI — Footer indicator
✦ dreamingappears in the footer while the background dream pass runs, translated across all 6 locales (en, zh, de, ja, pt, ru).Removed
save_memorytool (memoryTool.ts) and all its references — replaced by the extraction agentSaveMemoryToolCallwebui componentDocs and i18n
docs/users/features/memory.md— new user-facing guide: QWEN.md vs auto-memory, all commands, settings, troubleshootingdocs/users/configuration/settings.md—memory.enableManagedAutoMemoryandmemory.enableManagedAutoDreamsettings documentedScreenshots / Video Demo
/memoryshow the manage dialog:/rememberwill add memory manually:after a turn auto-memory will extract memories from the conversation:
auto-memory will be saved in other sessions:
dream can be actived by
/dreamor auto-dream after a turnReviewer Test Plan
/memory) and verify topic files were created/dream— confirm✦ dreamingappears in the footer during the run and the dialog reports updated last-dream time/remember prefer snake_case for Python— confirm the entry appears in the memory folder/forget snake_case— confirm the entry is deleted immediately"memory": { "enableManagedAutoMemory": false }in~/.qwen/settings.json— open/memorydialog and confirm the toggle reflects the change without restartde,ja,pt, orruand open/memory— verify all strings are translatedTesting Matrix
Linked issues / bugs
#2991
Closes the managed auto-memory tracking issue.