pokemon tcg market analysis fueled by microservices

the repo for this project can be found here

a millennium problem: i am broke

i wanna play the BDIF for PTCG Standard at time of writing, but i have no money... it's time to collect market pricing data!!! tcgplayer however does not provide access...how do i get the data i need?.. by some combination of web scraping or finding undocumented apis probably

approach 1 (FAILED)

use undetected-chromedriver, seleniumrequests, etc

pain points:

• unable to use seleniumgrid + docker + kubernetes for at scale scraping b/c seleniumgrid doesnt support custom drivers
• computationally expensive, realized there was no need to even use chrome drivers (see nodriver as an alternative)
• unnecessary to use, tcgplayer does not seem to care about the api im hitting
• too many dependencies for what should be a simple solution

approach 2 (FAILED)

use undetected-chromedriver's cooler, younger cousin (nodriver)

pain points:

• still not really necessary
• because it doesn't use selenium, i cant use seleniumrequests to hit their api and pretend im a real human

approach 3 (current)

fall back on just using httpx and asyncio

pain points:

• easily detected if they wanted to detect me

good points:

• easier to create a flask app around and deploy
• faster & simpler