fix: blank Scrapers page on databases with high scene counts#2135
Open
mvdeckard wants to merge 2 commits intoxbapps:masterfrom
Open
fix: blank Scrapers page on databases with high scene counts#2135mvdeckard wants to merge 2 commits intoxbapps:masterfrom
mvdeckard wants to merge 2 commits intoxbapps:masterfrom
Conversation
- Fix Scrapers page not showing scrapers on slow devices due to race condition with async store loading. Changed showAllScrapers to local component state with default true instead of persisted store state. - Fix Auto-limit scraping being enabled even when scrape failed (e.g., "Studio code not found" errors). Now only enables after successful scrape. - Fix SLR URL generation producing invalid double-dash URLs. Now uses the `label` field from SLR API directly instead of regenerating URL slugs from titles, preserving legitimate double-dash URLs while avoiding invalid ones. - Disable limit scraping when user selects "Force update scenes" to allow full re-scrape, matching the behavior of "Delete scraped scenes".
mvdeckard
changed the title
- Replace N individual COUNT queries with single GROUP BY query - Use map-based lookup for HasScraper check instead of nested loop - Add index on scenes.scraper_id via migration for existing users - Add gorm index annotation for new installs Fixes blank Scrapers page issue affecting users with large scene databases (20k+ scenes) on resource-constrained systems (NAS/Docker).
mvdeckard
force-pushed
the
fix/scraper-fixes
branch
from
d0735a2 to
b9061ce
Compare
Contributor
|
FYI: It probably doesn't matter for adding an index, but migrations that change the schema should go further up the migration list. Search for the line (currently about 850) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary