Scrapy – Dockerized Scrapy Development Environment

I wrote a Dockerfile which will follow the Scrapy installation instruction for Ubuntu. I had a hard time using pip to make it work, some errors like missing openssl/xxx.h.. Anyway, now you have a recipe to build the Image which contains BeautifulSoup4, Scrapy and iPython.

Checkout my github repository for more information.

You can modify the Dockerfile to only include the functionalities you need.

# start the container daemon in background
sudo docker run -v <hostdir>:<containerdir> -d <image>
# attach to the running container
sudo docker attach –sig-proxy=true <container>
# detach and leaving the container running without exiting.
CTRL-P + CTRL-Q

docker_scrapy_dockerfiledocker_scrapy_ipython

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s