Photo by Bruno Nascimento on Unsplash

First Steps

Create a Scrapy Project

pip install Scrapy 
conda install -c conda-forge scrapy
scrapy startproject productscraper
cd productscraper


pipeline | ReadFromMongoDB(uri='mongodb://localhost:27017',
db='testdb',
coll='input')
pipeline | WriteToMongoDB(uri='mongodb://localhost:27017',
db='testdb',
coll='output',
batch_size=10)



Selectors


Photo by Nathan Dumlao on Unsplash


Photo by Andrew Neel on Unsplash


Photo by Caspar Camille Rubin on Unsplash
function getMostFrequent(arr) {
const hashmap = arr.reduce( (acc, val) =>…



Photo by Alina Grubnyak on Unsplash


Photo by Nate Grant on Unsplash
function hang(secs){
const doneAt = Date.now() + (secs * 1000)
while( doneAt > Date.now ) {// just do nothing }}
hang(50)

Bilge Demirkaya

Software Engineer at 42 Silicon Valley | Technical Writer at ContentLab.io

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store