Skip to content

WIP Switch scraper over to selenium#343

Draft
A1Liu wants to merge 6 commits intomainfrom
selenium-scraper
Draft

WIP Switch scraper over to selenium#343
A1Liu wants to merge 6 commits intomainfrom
selenium-scraper

Conversation

@A1Liu
Copy link
Copy Markdown
Collaborator

@A1Liu A1Liu commented Aug 26, 2023

Did not want to do this, but I guess we gotta. Selenium seems to be the best/only practical choice for human-in-the-loop scraping, which is what we need to do in order to pass the captcha and also avoid any captcha arms race nonsense.

Notes

  • Need to remove the scanner auto-closing stuff because when it reads from stdin, the scanner should not be closed by us

@A1Liu A1Liu marked this pull request as draft August 26, 2023 18:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant