Scrapy-Splash: Failed to run docker container with scrapinghub/splash:latest as base image
up vote
0
down vote
favorite
Am building a python Scrapy application which uses some Azure Services and Scrapy-Splash. I tried creating a docker image for my application with scrapinghub/splash:latest as base image in my local windows machine.
Below is the Dockerfile am using,
FROM scrapinghub/splash:latest
WORKDIR /usr/src/snapshot
ADD requirements.txt ./
RUN pip install -r requirements.txt
ADD . ./
EXPOSE 8888 80
ENTRYPOINT ["/usr/src/snapshot/init_container.sh"]
The init_container.sh file has the statement to run the application, python /usr/src/snapshot/SiteCrawler.py.
Now when I run the docker image with the command docker run testsnapshot:0.1, the application starts and stops due to import error, ImportError: No module named azure.servicebus
I tried creating a docker image of the application with python:3.6.6 as base image, it works fine.
The application build the docker image and installs the packages from requirements.txt correctly.
Attaching below my requirements.txt content
asn1crypto==0.24.0
attrs==18.2.0
Automat==0.7.0
azure-common==1.1.16
azure-nspkg==3.0.2
azure-servicebus==0.21.1
azure-storage==0.36.0
certifi==2018.10.15
cffi==1.11.5
chardet==3.0.4
constantly==15.1.0
cryptography==2.3.1
cssselect==1.0.3
hyperlink==18.0.0
idna==2.7
incremental==17.5.0
lxml==4.2.5
parsel==1.5.0
pip==18.0
pyasn1==0.4.4
pyasn1-modules==0.2.2
pycparser==2.19
PyDispatcher==2.0.5
PyHamcrest==1.9.0
pyOpenSSL==18.0.0
python-dateutil==2.7.3
queuelib==1.5.0
requests==2.20.0
Scrapy==1.5.1
scrapy-splash==0.7.2
service-identity==17.0.0
setuptools==39.0.1
six==1.11.0
Twisted==16.1.1
urllib3==1.24
w3lib==1.19.0
zope.interface==4.5.0
python docker scrapy azureservicebus scrapy-splash
asked Nov 8 at 8:33
sadiqmc
5611
add a comment |
up vote
0
down vote
favorite
Am building a python Scrapy application which uses some Azure Services and Scrapy-Splash. I tried creating a docker image for my application with scrapinghub/splash:latest as base image in my local windows machine.
Below is the Dockerfile am using,
FROM scrapinghub/splash:latest
WORKDIR /usr/src/snapshot
ADD requirements.txt ./
RUN pip install -r requirements.txt
ADD . ./
EXPOSE 8888 80
ENTRYPOINT ["/usr/src/snapshot/init_container.sh"]
The init_container.sh file has the statement to run the application, python /usr/src/snapshot/SiteCrawler.py.
Now when I run the docker image with the command docker run testsnapshot:0.1, the application starts and stops due to import error, ImportError: No module named azure.servicebus
I tried creating a docker image of the application with python:3.6.6 as base image, it works fine.
The application build the docker image and installs the packages from requirements.txt correctly.
Attaching below my requirements.txt content
asn1crypto==0.24.0
attrs==18.2.0
Automat==0.7.0
azure-common==1.1.16
azure-nspkg==3.0.2
azure-servicebus==0.21.1
azure-storage==0.36.0
certifi==2018.10.15
cffi==1.11.5
chardet==3.0.4
constantly==15.1.0
cryptography==2.3.1
cssselect==1.0.3
hyperlink==18.0.0
idna==2.7
incremental==17.5.0
lxml==4.2.5
parsel==1.5.0
pip==18.0
pyasn1==0.4.4
pyasn1-modules==0.2.2
pycparser==2.19
PyDispatcher==2.0.5
PyHamcrest==1.9.0
pyOpenSSL==18.0.0
python-dateutil==2.7.3
queuelib==1.5.0
requests==2.20.0
Scrapy==1.5.1
scrapy-splash==0.7.2
service-identity==17.0.0
setuptools==39.0.1
six==1.11.0
Twisted==16.1.1
urllib3==1.24
w3lib==1.19.0
zope.interface==4.5.0
python docker scrapy azureservicebus scrapy-splash
asked Nov 8 at 8:33
sadiqmc
5611
Got it worked. Just addedVOLUME ["/usr/src/snapshot"]to Docker file. But now I face a different issue where the splash url is not accessible from my scrapy. I tried withhttp://127.0.0.1:8050andhttp://0.0.0.0:8050
– sadiqmc
2 days ago
add a comment |
up vote
0
down vote
favorite
up vote
0
down vote
favorite
Am building a python Scrapy application which uses some Azure Services and Scrapy-Splash. I tried creating a docker image for my application with scrapinghub/splash:latest as base image in my local windows machine.
Below is the Dockerfile am using,
FROM scrapinghub/splash:latest
WORKDIR /usr/src/snapshot
ADD requirements.txt ./
RUN pip install -r requirements.txt
ADD . ./
EXPOSE 8888 80
ENTRYPOINT ["/usr/src/snapshot/init_container.sh"]
The init_container.sh file has the statement to run the application, python /usr/src/snapshot/SiteCrawler.py.
Now when I run the docker image with the command docker run testsnapshot:0.1, the application starts and stops due to import error, ImportError: No module named azure.servicebus
I tried creating a docker image of the application with python:3.6.6 as base image, it works fine.
The application build the docker image and installs the packages from requirements.txt correctly.
Attaching below my requirements.txt content
asn1crypto==0.24.0
attrs==18.2.0
Automat==0.7.0
azure-common==1.1.16
azure-nspkg==3.0.2
azure-servicebus==0.21.1
azure-storage==0.36.0
certifi==2018.10.15
cffi==1.11.5
chardet==3.0.4
constantly==15.1.0
cryptography==2.3.1
cssselect==1.0.3
hyperlink==18.0.0
idna==2.7
incremental==17.5.0
lxml==4.2.5
parsel==1.5.0
pip==18.0
pyasn1==0.4.4
pyasn1-modules==0.2.2
pycparser==2.19
PyDispatcher==2.0.5
PyHamcrest==1.9.0
pyOpenSSL==18.0.0
python-dateutil==2.7.3
queuelib==1.5.0
requests==2.20.0
Scrapy==1.5.1
scrapy-splash==0.7.2
service-identity==17.0.0
setuptools==39.0.1
six==1.11.0
Twisted==16.1.1
urllib3==1.24
w3lib==1.19.0
zope.interface==4.5.0
python docker scrapy azureservicebus scrapy-splash
asked Nov 8 at 8:33
sadiqmc
5611
Am building a python Scrapy application which uses some Azure Services and Scrapy-Splash. I tried creating a docker image for my application with scrapinghub/splash:latest as base image in my local windows machine.
Below is the Dockerfile am using,
FROM scrapinghub/splash:latest
WORKDIR /usr/src/snapshot
ADD requirements.txt ./
RUN pip install -r requirements.txt
ADD . ./
EXPOSE 8888 80
ENTRYPOINT ["/usr/src/snapshot/init_container.sh"]
The init_container.sh file has the statement to run the application, python /usr/src/snapshot/SiteCrawler.py.
Now when I run the docker image with the command docker run testsnapshot:0.1, the application starts and stops due to import error, ImportError: No module named azure.servicebus
I tried creating a docker image of the application with python:3.6.6 as base image, it works fine.
The application build the docker image and installs the packages from requirements.txt correctly.
Attaching below my requirements.txt content
asn1crypto==0.24.0
attrs==18.2.0
Automat==0.7.0
azure-common==1.1.16
azure-nspkg==3.0.2
azure-servicebus==0.21.1
azure-storage==0.36.0
certifi==2018.10.15
cffi==1.11.5
chardet==3.0.4
constantly==15.1.0
cryptography==2.3.1
cssselect==1.0.3
hyperlink==18.0.0
idna==2.7
incremental==17.5.0
lxml==4.2.5
parsel==1.5.0
pip==18.0
pyasn1==0.4.4
pyasn1-modules==0.2.2
pycparser==2.19
PyDispatcher==2.0.5
PyHamcrest==1.9.0
pyOpenSSL==18.0.0
python-dateutil==2.7.3
queuelib==1.5.0
requests==2.20.0
Scrapy==1.5.1
scrapy-splash==0.7.2
service-identity==17.0.0
setuptools==39.0.1
six==1.11.0
Twisted==16.1.1
urllib3==1.24
w3lib==1.19.0
zope.interface==4.5.0
python docker scrapy azureservicebus scrapy-splash
python docker scrapy azureservicebus scrapy-splash
asked Nov 8 at 8:33
sadiqmc
5611
asked Nov 8 at 8:33
sadiqmc
5611
asked Nov 8 at 8:33
sadiqmc
5611
asked Nov 8 at 8:33
sadiqmc
5611
asked Nov 8 at 8:33
sadiqmc
5611
5611
Got it worked. Just addedVOLUME ["/usr/src/snapshot"]to Docker file. But now I face a different issue where the splash url is not accessible from my scrapy. I tried withhttp://127.0.0.1:8050andhttp://0.0.0.0:8050
– sadiqmc
2 days ago
add a comment |
Got it worked. Just addedVOLUME ["/usr/src/snapshot"]to Docker file. But now I face a different issue where the splash url is not accessible from my scrapy. I tried withhttp://127.0.0.1:8050andhttp://0.0.0.0:8050
– sadiqmc
2 days ago
Got it worked. Just added
VOLUME ["/usr/src/snapshot"] to Docker file. But now I face a different issue where the splash url is not accessible from my scrapy. I tried with http://127.0.0.1:8050 and http://0.0.0.0:8050– sadiqmc
2 days ago
Got it worked. Just added
VOLUME ["/usr/src/snapshot"] to Docker file. But now I face a different issue where the splash url is not accessible from my scrapy. I tried with http://127.0.0.1:8050 and http://0.0.0.0:8050– sadiqmc
2 days ago
add a comment |
active
oldest
votes
active
oldest
votes
active
oldest
votes
active
oldest
votes
active
oldest
votes
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53203995%2fscrapy-splash-failed-to-run-docker-container-with-scrapinghub-splashlatest-as%23new-answer', 'question_page');
}
);
Post as a guest
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Got it worked. Just added
VOLUME ["/usr/src/snapshot"]to Docker file. But now I face a different issue where the splash url is not accessible from my scrapy. I tried withhttp://127.0.0.1:8050andhttp://0.0.0.0:8050– sadiqmc
2 days ago