Website to Dataset in an instant

1000 items in one API request... creating a dataset from a simple API call. I enjoyed this one, there will be a part 2 where I clean the data with Pandas.

This is a scrapy project using the sitemap spider, saving the data to an sqlite database using a pipeline.

Join the Discord to discuss all things Python and Web with our growing community!   / discord  

If you are new, welcome! I am John, a self taught Python developer working in the web and data space. I specialize in data extraction and JSON web API's both server and client. If you like programming and web content as much as I do, you can subscribe for weekly content.

:: Links ::
My Patrons Really keep the channel alive, and get early content   / johnwatsonrooney   (NEW free tier)

Recommender Scraper API https://www.scrapingbee.com?fpr=jhnwr

I Host almost all my stuff on Digital Ocean https://m.do.co/c/c7c90f161ff6

I rundown of the gear I use to create videos https://www.amazon.co.uk/shop/johnwat...

Proxies I recommend https://nodemaven.com/?a_aid=JohnWats...

:: Disclaimer ::
Some/all of the links above are affiliate links. By clicking on these links I receive a small commission should you chose to purchase any services or items.