Seed canonical morph types and regenerate search index#2219
Seed canonical morph types and regenerate search index#2219myieye merged 28 commits intofeat/sync-morph-typesfrom
Conversation
|
Important Review skippedAuto incremental reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
📝 WalkthroughWalkthroughThis PR adds a canonical registry of morph types ( Changes
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Possibly related issues
Possibly related PRs
Suggested labels
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
github-actions
bot
added
the
💻 FW Lite
myieye
force-pushed
the
claude/add-lexeme-headwords-TowRX
branch
from
dc45a12 to
823531a
Compare
myieye
force-pushed
the
claude/fix-morph-type-syncing-bQo8X
branch
from
46c8f87 to
f8f4cfe
Compare
myieye
changed the base branch from
claude/add-lexeme-headwords-TowRX
to
feat/sync-morph-types
|
The latest updates on your projects. Learn more about Argos notifications ↗︎
|
|
Now working on getting tests to pass. Writing down my findings as I go, so this is going to be a little bit stream-of-consciousness. Sorting order tests are acting non-deterministic, with different sort orders returning on different runs of the same test. I.e., one run of the test might produce the sort order [suffix, root, prefix] and the next run of the same test might produce [suffix, prefix, root]. I'm starting to suspect homograph numbers may be to blame. The MorphTokens_DoNotAffectSortOrder test creates three entries, "aaaa" (a root), "-aaaa" (a suffix), and "aaaa-" (a prefix). Those should not be treated as homographs... but if they are being given homograph numbers, then the order of the homograph numbers may entirely depend on the order in which the homographs are found. ... And I just noticed that the sorting tests are ending up sorting those three words in an order that matches the alphabetical ordering of their GUID. Which would be consistent with the theory that the homograph number is what's being sorted on here. ... But a good argument against that is that the tests that are failing are the LcmCrdt sorting tests, where the "sort by HomographNumber" lines are still commented out! Might have to search for a new theory. |
|
It looks like the predefined morph types aren't being seeded in the sorting tests, and that may be the cause of the failures. |
|
The issue for the sort tests was that the morph types, and other prepopulated data, are populated in CrdtProjectsService.CreateProject, which calls SeedSystemData. But there's a comment in MiniLcmApiFixture explaining that it doesn't call CrdtProjectsService.CreateProject because that ends up opening and closing the DB context (due to the I added a step to MiniLcmApiFixture to seed morph types (but not other initial data like semantic domains or parts of speech), and now the tests appear to be passing because the sort order logic can actually look up the morph types and get their SecondaryOrder. The earlier test failures were because the morph types weren't present, so the secondary order was always 0 (the order of Stem), and so the DB found records that were exactly equal in all clauses of its ORDER BY statement, and was therefore allowed to return them in arbitrary order. Which also explains why the sorting tests were failing non-deterministically, because the order of the rows returned by the DB was non-deterministic — probably the order that the rows were stored in the SQLite database, which would depend on a host of factors. |
|
Sena-3 sync test errors are because the predefined morph types that Claude generated include correct names and abbreviations, but do not include descriptions. So when the sena-3 project is synced from FW, it creates 19 changes (when 0 are expected) that just consist of bringing in the descriptions. I'll push a commit that includes descriptions in the predefined data, because if we're matching FW then let's match all of FW. |
|
Note that the descriptions in Sena-3 do not completely match the descriptions found in new FW projects. Specifically, the description for Stem in Sena-3 is a single run, but in new FW projects, the description for Stem includes some runs with the "Emphasized Text" style. The text is the same as far as I can see, it's just that some of the words were emphasized at some point in the past. Which means that when we sync projects that are not sena-3, we should expect at least one change in morph types, as the .fwdata file will contain a different Stem description (with different numbers of runs) than what I'm currently putting into the CanonicalMorphTypes data. Ways we could handle that:
|
|
I believe the reordering of the entries in the project snapshots may be because But the change in commit 1e94608 is good enough for now. Tests are passing; remaining failing check is Argos complaining about the visual change of including morph type prefixes in the UI, and I don't seem to have the access permissions to tell Argos "Yes, that's the new standard of what it should look like". Everything except that is green, so tests are all passing now and PR can be marked ready. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
🧹 Nitpick comments (1)
backend/FwLite/MiniLcm/Models/CanonicalMorphTypes.cs (1)
11-11: Avoid exposing mutable canonical instances globally.
Allis frozen, but eachMorphTypevalue is still mutable and shared process-wide. Please expose immutable definitions (or clone on access) so callers cannot mutate canonical state accidentally.As per coding guidelines "Prefer Records for DTOs and immutable data structures in C# code".
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@backend/FwLite/MiniLcm/Models/CanonicalMorphTypes.cs` at line 11, The static field CanonicalMorphTypes.All currently exposes shared mutable MorphType instances; update the API so callers cannot mutate canonical state by either converting MorphType into an immutable type (preferably a record or readonly struct) and rebuilding CreateAll() to produce those immutable instances, or keep MorphType as-is but change CanonicalMorphTypes.All to store and return defensive copies (e.g., expose a method/property that returns clones from CreateAll()/All instead of the original instances). Locate CreateAll(), the MorphType type definition, and the All field to implement one of these approaches and ensure any consumers use the new immutable type or the clone-returning accessor.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Nitpick comments:
In `@backend/FwLite/MiniLcm/Models/CanonicalMorphTypes.cs`:
- Line 11: The static field CanonicalMorphTypes.All currently exposes shared
mutable MorphType instances; update the API so callers cannot mutate canonical
state by either converting MorphType into an immutable type (preferably a record
or readonly struct) and rebuilding CreateAll() to produce those immutable
instances, or keep MorphType as-is but change CanonicalMorphTypes.All to store
and return defensive copies (e.g., expose a method/property that returns clones
from CreateAll()/All instead of the original instances). Locate CreateAll(), the
MorphType type definition, and the All field to implement one of these
approaches and ensure any consumers use the new immutable type or the
clone-returning accessor.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: df3c003c-8913-498d-bba3-1e2df8c429be
📒 Files selected for processing (28)
backend/FwLite/FwDataMiniLcmBridge/Api/Sorting.csbackend/FwLite/FwLiteProjectSync.Tests/FluentAssertGlobalConfig.csbackend/FwLite/FwLiteProjectSync.Tests/Import/ResumableTests.csbackend/FwLite/FwLiteProjectSync.Tests/ProjectSnapshotSerializationTests.csbackend/FwLite/FwLiteProjectSync.Tests/Sena3SyncTests.csbackend/FwLite/FwLiteProjectSync.Tests/SyncTests.csbackend/FwLite/FwLiteProjectSync/CrdtFwdataProjectSyncService.csbackend/FwLite/LcmCrdt.Tests/Changes/ChangeSerializationTests.csbackend/FwLite/LcmCrdt.Tests/Data/FilteringTests.csbackend/FwLite/LcmCrdt.Tests/Data/MigrationTests.csbackend/FwLite/LcmCrdt.Tests/Data/RegressionTestHelper.csbackend/FwLite/LcmCrdt.Tests/Data/SnapshotDeserializationTests.csbackend/FwLite/LcmCrdt.Tests/DataModelSnapshotTests.csbackend/FwLite/LcmCrdt.Tests/FullTextSearch/EntrySearchServiceTests.csbackend/FwLite/LcmCrdt.Tests/MiniLcmApiFixture.csbackend/FwLite/LcmCrdt.Tests/MiniLcmTests/SortingTests.csbackend/FwLite/LcmCrdt.Tests/MorphTypeSeedingTests.csbackend/FwLite/LcmCrdt.Tests/OpenProjectTests.csbackend/FwLite/LcmCrdt/CrdtProjectsService.csbackend/FwLite/LcmCrdt/CurrentProjectService.csbackend/FwLite/LcmCrdt/Data/MiniLcmRepository.csbackend/FwLite/LcmCrdt/Migrations/20260318120000_RegenerateSearchTableForMorphTypes.Designer.csbackend/FwLite/LcmCrdt/Migrations/20260318120000_RegenerateSearchTableForMorphTypes.csbackend/FwLite/LcmCrdt/Objects/PreDefinedData.csbackend/FwLite/MiniLcm.Tests/QueryEntryTestsBase.csbackend/FwLite/MiniLcm.Tests/SortingTestsBase.csbackend/FwLite/MiniLcm/Models/CanonicalMorphTypes.csbackend/FwLite/Taskfile.yml
💤 Files with no reviewable changes (3)
- backend/FwLite/MiniLcm.Tests/QueryEntryTestsBase.cs
- backend/FwLite/LcmCrdt.Tests/MiniLcmTests/SortingTests.cs
- backend/FwLite/MiniLcm.Tests/SortingTestsBase.cs
|
We should have consistent ordering of entries in our snapshots. It's just part of the Get entries API: The result is sorted. I had test failures in the branch I already merged due to sort order and fixed them by commenting out the HomographNumber from fwdata sorting. Because we fall back to sorting by ID, we get consistent sorting in both CRDT and FWDATA even without homographs numbers. That's likely not all relevant. But, I don't think it's correct to not enforce strict ordering of entries. I didn't read everything you've written. |
rmunn
left a comment
rmunn
left a comment
There was a problem hiding this comment.
A few changes needed before we can merge.
rmunn
left a comment
rmunn
left a comment
There was a problem hiding this comment.
Looks good now, all test changes I felt were necessary have now been made.
|
@CodeRabbit review |
✅ Actions performedReview triggered.
|
|
@CodeRabbit fullreview |
|
🧠 Learnings used✅ Actions performedFull review triggered. |
There was a problem hiding this comment.
Actionable comments posted: 2
♻️ Duplicate comments (1)
backend/FwLite/LcmCrdt.Tests/MorphTypeSeedingTests.cs (1)
35-42:⚠️ Potential issue | 🟡 MinorMissing seeded
Descriptionparity assertionThis test validates most canonical fields but skips
Description, so a description-mapping regression can pass unnoticed.Suggested assertion addition
foreach (var canonical in CanonicalMorphTypes.All.Values) { var mt = morphTypes.Should().ContainSingle(m => m.Kind == canonical.Kind).Subject; mt.Id.Should().Be(canonical.Id); mt.Name["en"].Should().Be(canonical.Name["en"]); mt.Abbreviation["en"].Should().Be(canonical.Abbreviation["en"]); + mt.Description["en"].Should().BeEquivalentTo(canonical.Description["en"]); mt.Prefix.Should().Be(canonical.Prefix); mt.Postfix.Should().Be(canonical.Postfix); mt.SecondaryOrder.Should().Be(canonical.SecondaryOrder); }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@backend/FwLite/LcmCrdt.Tests/MorphTypeSeedingTests.cs` around lines 35 - 42, The test in MorphTypeSeedingTests checks many canonical fields but omits Description, so add an assertion to compare the seeded morph type's Description for the same locale; after locating the existing assertions that inspect mt (the Subject from morphTypes.Should().ContainSingle(m => m.Kind == canonical.Kind)), add a check that mt.Description["en"].Should().Be(canonical.Description["en"]) (or the appropriate locale key used elsewhere) to ensure description parity between the seeded mt and canonical.
🧹 Nitpick comments (1)
backend/FwLite/FwLiteProjectSync.Tests/Sena3SyncTests.cs (1)
114-117: Strengthen canonical parity check withNameandAbbreviationassertions.The test currently validates IDs/affixes/order, but misses user-visible label drift.
Suggested test hardening
var canonical = CanonicalMorphTypes.All[fwMorphType.Kind]; canonical.Id.Should().Be(fwMorphType.Id, $"GUID for {fwMorphType.Kind} should match FwData"); + canonical.Name["en"].Should().Be(fwMorphType.Name["en"], $"Name for {fwMorphType.Kind} should match FwData"); + canonical.Abbreviation["en"].Should().Be(fwMorphType.Abbreviation["en"], $"Abbreviation for {fwMorphType.Kind} should match FwData"); canonical.Prefix.Should().Be(fwMorphType.Prefix, $"Prefix for {fwMorphType.Kind} should match FwData"); canonical.Postfix.Should().Be(fwMorphType.Postfix, $"Postfix for {fwMorphType.Kind} should match FwData"); canonical.SecondaryOrder.Should().Be(fwMorphType.SecondaryOrder, $"SecondaryOrder for {fwMorphType.Kind} should match FwData");🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@backend/FwLite/FwLiteProjectSync.Tests/Sena3SyncTests.cs` around lines 114 - 117, Add assertions to the existing parity checks to validate user-facing labels by asserting canonical.Name.Should().Be(fwMorphType.Name, $"Name for {fwMorphType.Kind} should match FwData") and canonical.Abbreviation.Should().Be(fwMorphType.Abbreviation, $"Abbreviation for {fwMorphType.Kind} should match FwData"); place these alongside the existing Id/Prefix/Postfix/SecondaryOrder assertions in the Sena3SyncTests (where canonical and fwMorphType are compared) so name and abbreviation drift will fail the test.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@backend/FwLite/LcmCrdt.Tests/Data/RegressionTestHelper.cs`:
- Around line 73-78: The RegressionVersion enum now includes v3 but tests and
test data are not updated; either add a v3.sql test fixture and wiring or remove
the enum. To fix: create the missing backend/FwLite/LcmCrdt.Tests/Data/v3.sql
with the expected scripted DB state, then update MigrationTests.cs to add
[InlineData(RegressionVersion.v3)] to both VerifyAfterMigrationFromScriptedDb
and VerifyRegeneratedSnapshotsAfterMigrationFromScriptedDb, ensuring the test
helpers (e.g., methods that map enum to file names) recognize "v3";
alternatively, if v3 is premature, remove the v3 member from the
RegressionVersion enum to restore consistency.
In `@backend/FwLite/LcmCrdt.Tests/MorphTypeSeedingTests.cs`:
- Around line 99-103: The test SeedingIsIdempotent in class
MorphTypeSeedingTests doesn't actually re-trigger seeding because MigrationTasks
is static, so either make the test explicitly force a second seed run by
resetting the static state (clear MigrationTasks) before calling MigrateDb a
second time, or rename the test to reflect that it only verifies duplicate-safe
counts after reopen; update the test to call MigrationTasks.Clear() (or the
actual static reset method/property) and then call MigrateDb again to assert
idempotence, or change the test name to e.g.,
SeedingDoesNotCreateDuplicatesOnReopen to match current behavior.
---
Duplicate comments:
In `@backend/FwLite/LcmCrdt.Tests/MorphTypeSeedingTests.cs`:
- Around line 35-42: The test in MorphTypeSeedingTests checks many canonical
fields but omits Description, so add an assertion to compare the seeded morph
type's Description for the same locale; after locating the existing assertions
that inspect mt (the Subject from morphTypes.Should().ContainSingle(m => m.Kind
== canonical.Kind)), add a check that
mt.Description["en"].Should().Be(canonical.Description["en"]) (or the
appropriate locale key used elsewhere) to ensure description parity between the
seeded mt and canonical.
---
Nitpick comments:
In `@backend/FwLite/FwLiteProjectSync.Tests/Sena3SyncTests.cs`:
- Around line 114-117: Add assertions to the existing parity checks to validate
user-facing labels by asserting canonical.Name.Should().Be(fwMorphType.Name,
$"Name for {fwMorphType.Kind} should match FwData") and
canonical.Abbreviation.Should().Be(fwMorphType.Abbreviation, $"Abbreviation for
{fwMorphType.Kind} should match FwData"); place these alongside the existing
Id/Prefix/Postfix/SecondaryOrder assertions in the Sena3SyncTests (where
canonical and fwMorphType are compared) so name and abbreviation drift will fail
the test.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: ba07c2f8-bf11-4a7f-b91f-abb9d34c74b9
📒 Files selected for processing (27)
backend/FwLite/FwDataMiniLcmBridge.Tests/MiniLcmTests/SortingTests.csbackend/FwLite/FwLiteProjectSync.Tests/FluentAssertGlobalConfig.csbackend/FwLite/FwLiteProjectSync.Tests/Import/ResumableTests.csbackend/FwLite/FwLiteProjectSync.Tests/ProjectSnapshotSerializationTests.csbackend/FwLite/FwLiteProjectSync.Tests/Sena3SyncTests.csbackend/FwLite/FwLiteProjectSync.Tests/SyncTests.csbackend/FwLite/FwLiteProjectSync/CrdtFwdataProjectSyncService.csbackend/FwLite/LcmCrdt.Tests/Changes/ChangeSerializationTests.csbackend/FwLite/LcmCrdt.Tests/Data/FilteringTests.csbackend/FwLite/LcmCrdt.Tests/Data/MigrationTests.csbackend/FwLite/LcmCrdt.Tests/Data/RegressionTestHelper.csbackend/FwLite/LcmCrdt.Tests/Data/SnapshotDeserializationTests.csbackend/FwLite/LcmCrdt.Tests/DataModelSnapshotTests.csbackend/FwLite/LcmCrdt.Tests/FullTextSearch/EntrySearchServiceTests.csbackend/FwLite/LcmCrdt.Tests/MiniLcmApiFixture.csbackend/FwLite/LcmCrdt.Tests/MiniLcmTests/SortingTests.csbackend/FwLite/LcmCrdt.Tests/MorphTypeSeedingTests.csbackend/FwLite/LcmCrdt.Tests/OpenProjectTests.csbackend/FwLite/LcmCrdt/CrdtProjectsService.csbackend/FwLite/LcmCrdt/CurrentProjectService.csbackend/FwLite/LcmCrdt/Migrations/20260318120000_RegenerateSearchTableForMorphTypes.Designer.csbackend/FwLite/LcmCrdt/Migrations/20260318120000_RegenerateSearchTableForMorphTypes.csbackend/FwLite/LcmCrdt/Objects/PreDefinedData.csbackend/FwLite/MiniLcm.Tests/QueryEntryTestsBase.csbackend/FwLite/MiniLcm.Tests/SortingTestsBase.csbackend/FwLite/MiniLcm/Models/CanonicalMorphTypes.csbackend/FwLite/Taskfile.yml
💤 Files with no reviewable changes (1)
- backend/FwLite/MiniLcm.Tests/QueryEntryTestsBase.cs
myieye
force-pushed
the
feat/sync-morph-types
branch
from
e9f7255 to
6af9827
Compare
Project snapshots for CRDT and FW are returning entries in different order, for some not-yet-identified reason. As long as the entries with the same ID compare the same, we should not fail the Sena-3 sync tests just because the snapshots have the entries in a different order. Once the tests pass, I may revert this commit and investigate why the entries in the snapshot are coming out in a different order.
No need to set "WithoutStrictOrderingFor(x -> x.Entries)" if the FW and CRDT APIs are sorting entries the same way. The difference in sorting was because the FW API had been set to sort by HomographNumber, but CRDT has not yet implemented HomographNumber so the sorting was different. Removing the sort by HomographNumber from the FW API results in the tests passing because snapshots are sorted in the same order again.
Test comments should reflect actual SecondaryOrder values from canonical morph types, so that we don't get confused later.
Check that descriptions got created since they're essential to having the correct number of changes in Sena-3 sync tests, and do a bit more to check that the SeedingIsIdempotent_OpeningProjectTwiceDoesNotDuplicate test is actually checking what we think it's checking.
Seeding the morph types does not actually create a change in the serialization format of project snapshots, so we don't actually need a RegressionVersion.v3 and corresponding tests for it.
myieye
force-pushed
the
claude/fix-morph-type-syncing-bQo8X
branch
from
26cf94c to
bcbe1bc
Compare
|
@coderabbitai full review |
✅ Actions performedFull review triggered. |
* Seed canonical morph-types into CRDT projects - Add CanonicalMorphTypes with all 19 morph-type definitions (GUIDs from LibLCM) - Seed morph-types for new projects via PreDefinedData.PredefinedMorphTypes - Seed morph-types for existing projects in MigrateDb (before FTS refresh) - Add EF migration to clear FTS table so headwords are rebuilt with morph tokens - Patch legacy snapshots (empty MorphTypes) in sync layer to prevent duplicates * Stop creating morph-types in tests. They're now prepopulated * Stop printing verify diff content. It's too much. * Seed morph types before API testing * Add descriptions to canonical morph types * Sync morph-types when importing, because they already exist in CRDT * Verify our canonical morph-types match new fwdata projects * Fix non-FTS relevance order with morph-tokens in query
* Seed canonical morph-types into CRDT projects - Add CanonicalMorphTypes with all 19 morph-type definitions (GUIDs from LibLCM) - Seed morph-types for new projects via PreDefinedData.PredefinedMorphTypes - Seed morph-types for existing projects in MigrateDb (before FTS refresh) - Add EF migration to clear FTS table so headwords are rebuilt with morph tokens - Patch legacy snapshots (empty MorphTypes) in sync layer to prevent duplicates * Stop creating morph-types in tests. They're now prepopulated * Stop printing verify diff content. It's too much. * Seed morph types before API testing * Add descriptions to canonical morph types * Sync morph-types when importing, because they already exist in CRDT * Verify our canonical morph-types match new fwdata projects * Fix non-FTS relevance order with morph-tokens in query
* Seed canonical morph-types into CRDT projects - Add CanonicalMorphTypes with all 19 morph-type definitions (GUIDs from LibLCM) - Seed morph-types for new projects via PreDefinedData.PredefinedMorphTypes - Seed morph-types for existing projects in MigrateDb (before FTS refresh) - Add EF migration to clear FTS table so headwords are rebuilt with morph tokens - Patch legacy snapshots (empty MorphTypes) in sync layer to prevent duplicates * Stop creating morph-types in tests. They're now prepopulated * Stop printing verify diff content. It's too much. * Seed morph types before API testing * Add descriptions to canonical morph types * Sync morph-types when importing, because they already exist in CRDT * Verify our canonical morph-types match new fwdata projects * Fix non-FTS relevance order with morph-tokens in query
* Seed canonical morph-types into CRDT projects - Add CanonicalMorphTypes with all 19 morph-type definitions (GUIDs from LibLCM) - Seed morph-types for new projects via PreDefinedData.PredefinedMorphTypes - Seed morph-types for existing projects in MigrateDb (before FTS refresh) - Add EF migration to clear FTS table so headwords are rebuilt with morph tokens - Patch legacy snapshots (empty MorphTypes) in sync layer to prevent duplicates * Stop creating morph-types in tests. They're now prepopulated * Stop printing verify diff content. It's too much. * Seed morph types before API testing * Add descriptions to canonical morph types * Sync morph-types when importing, because they already exist in CRDT * Verify our canonical morph-types match new fwdata projects * Fix non-FTS relevance order with morph-tokens in query
* Seed canonical morph-types into CRDT projects - Add CanonicalMorphTypes with all 19 morph-type definitions (GUIDs from LibLCM) - Seed morph-types for new projects via PreDefinedData.PredefinedMorphTypes - Seed morph-types for existing projects in MigrateDb (before FTS refresh) - Add EF migration to clear FTS table so headwords are rebuilt with morph tokens - Patch legacy snapshots (empty MorphTypes) in sync layer to prevent duplicates * Stop creating morph-types in tests. They're now prepopulated * Stop printing verify diff content. It's too much. * Seed morph types before API testing * Add descriptions to canonical morph types * Sync morph-types when importing, because they already exist in CRDT * Verify our canonical morph-types match new fwdata projects * Fix non-FTS relevance order with morph-tokens in query
3 participants
Resolves #2212
Summary
This PR ensures all canonical morph type definitions are seeded into new and existing CRDT projects, and regenerates the full-text search index to include morph-type prefix/postfix tokens in searchable headwords.
Key Changes
Added canonical morph type definitions (
CanonicalMorphTypes.cs): Defines 20 standard morph types matching FieldWorks/LibLCM specifications with verified GUIDs, names, abbreviations, and affixes. Uses frozen dictionary for immutability and performance.Automatic seeding on project creation and migration: Modified
CrdtProjectsServiceandCurrentProjectServiceto seed canonical morph types when:SeedNewProjectData: trueMigrateDb)Search index regeneration: Added migration
20260318120000_RegenerateSearchTableForMorphTypesthat clears theEntrySearchRecordtable to force FTS rebuild with updated morph-type affixes.Legacy snapshot patching: Updated
CrdtFwdataProjectSyncServiceto patch legacy snapshots missing morph types during sync operations, preventing duplicates when syncing with FwData.Comprehensive test coverage: Added
MorphTypeSeedingTestsverifying:MorphTypeKindenum values are coveredAdded integration test in
Sena3SyncTestsvalidating canonical definitions match FwData morph types.Regression test versioning: Updated
RegressionTestHelperto version 3 for snapshot compatibility.Implementation Details
PreDefinedData.PredefinedMorphTypes()to integrate with existing predefined data infrastructure.https://claude.ai/code/session_01WDKE2vXP4gjMWjfn4cmL4p