Python dependencies and Docker and Crontab
up vote
0
down vote
favorite
I'm having a strange issue when trying to run a Python script via Crontab within a Docker container. Here's the setup.
I have a Dockerfile like this:
FROM python:3-onbuild
# Install cron and dependencies
RUN apt-get update && apt-get -y install cron
# Add crontab file in the cron directory
ADD crontab /etc/cron.d/simple-cron
WORKDIR /src
COPY . /src
# Give execution rights on the cron job
RUN chmod 0644 /etc/cron.d/simple-cron
RUN chmod 777 /usr/local/lib/python3.6/site-packages
# Create the log file to be able to run tail
RUN touch /var/log/cron.log
# Run the command on container startup
CMD ["cron", "-f"]
And a crontab file like this:
* * * * * root python /src/test.py > /proc/1/fd/1 2>/proc/1/fd/2
This executes correctly with no problems when I start the container. The first line of my Python file is import redis
, which causes the script to fail, saying that the module cannot be found.
The interesting thing is that when I manually execute docker exec -it [container_name] python test.py
, everything works exactly as expected.
What could be the issue here? I'm thinking that the requirements file isn't being installed to the right location?
python docker redis cron
add a comment |
up vote
0
down vote
favorite
I'm having a strange issue when trying to run a Python script via Crontab within a Docker container. Here's the setup.
I have a Dockerfile like this:
FROM python:3-onbuild
# Install cron and dependencies
RUN apt-get update && apt-get -y install cron
# Add crontab file in the cron directory
ADD crontab /etc/cron.d/simple-cron
WORKDIR /src
COPY . /src
# Give execution rights on the cron job
RUN chmod 0644 /etc/cron.d/simple-cron
RUN chmod 777 /usr/local/lib/python3.6/site-packages
# Create the log file to be able to run tail
RUN touch /var/log/cron.log
# Run the command on container startup
CMD ["cron", "-f"]
And a crontab file like this:
* * * * * root python /src/test.py > /proc/1/fd/1 2>/proc/1/fd/2
This executes correctly with no problems when I start the container. The first line of my Python file is import redis
, which causes the script to fail, saying that the module cannot be found.
The interesting thing is that when I manually execute docker exec -it [container_name] python test.py
, everything works exactly as expected.
What could be the issue here? I'm thinking that the requirements file isn't being installed to the right location?
python docker redis cron
Where do you install your requirements?
– Klaus D.
Nov 10 at 0:26
add a comment |
up vote
0
down vote
favorite
up vote
0
down vote
favorite
I'm having a strange issue when trying to run a Python script via Crontab within a Docker container. Here's the setup.
I have a Dockerfile like this:
FROM python:3-onbuild
# Install cron and dependencies
RUN apt-get update && apt-get -y install cron
# Add crontab file in the cron directory
ADD crontab /etc/cron.d/simple-cron
WORKDIR /src
COPY . /src
# Give execution rights on the cron job
RUN chmod 0644 /etc/cron.d/simple-cron
RUN chmod 777 /usr/local/lib/python3.6/site-packages
# Create the log file to be able to run tail
RUN touch /var/log/cron.log
# Run the command on container startup
CMD ["cron", "-f"]
And a crontab file like this:
* * * * * root python /src/test.py > /proc/1/fd/1 2>/proc/1/fd/2
This executes correctly with no problems when I start the container. The first line of my Python file is import redis
, which causes the script to fail, saying that the module cannot be found.
The interesting thing is that when I manually execute docker exec -it [container_name] python test.py
, everything works exactly as expected.
What could be the issue here? I'm thinking that the requirements file isn't being installed to the right location?
python docker redis cron
I'm having a strange issue when trying to run a Python script via Crontab within a Docker container. Here's the setup.
I have a Dockerfile like this:
FROM python:3-onbuild
# Install cron and dependencies
RUN apt-get update && apt-get -y install cron
# Add crontab file in the cron directory
ADD crontab /etc/cron.d/simple-cron
WORKDIR /src
COPY . /src
# Give execution rights on the cron job
RUN chmod 0644 /etc/cron.d/simple-cron
RUN chmod 777 /usr/local/lib/python3.6/site-packages
# Create the log file to be able to run tail
RUN touch /var/log/cron.log
# Run the command on container startup
CMD ["cron", "-f"]
And a crontab file like this:
* * * * * root python /src/test.py > /proc/1/fd/1 2>/proc/1/fd/2
This executes correctly with no problems when I start the container. The first line of my Python file is import redis
, which causes the script to fail, saying that the module cannot be found.
The interesting thing is that when I manually execute docker exec -it [container_name] python test.py
, everything works exactly as expected.
What could be the issue here? I'm thinking that the requirements file isn't being installed to the right location?
python docker redis cron
python docker redis cron
asked Nov 9 at 23:53
Austin Osborne
84
84
Where do you install your requirements?
– Klaus D.
Nov 10 at 0:26
add a comment |
Where do you install your requirements?
– Klaus D.
Nov 10 at 0:26
Where do you install your requirements?
– Klaus D.
Nov 10 at 0:26
Where do you install your requirements?
– Klaus D.
Nov 10 at 0:26
add a comment |
1 Answer
1
active
oldest
votes
up vote
0
down vote
With python in docker I definitely find it helpful to use a vendoring tool. Instead of copying the local site-packages, run a script or series of RUN
commands which:
- Establish the virtualenv
- Activate it
- call
pip install -r requirements.txt
Here's an example from a script I use in dev:
pip3 install virtualenv;
virtualenv --no-site-packages --python=python3.6 app;
virtualenv --relocatable app;
source ./app/bin/activate;
pip3 install -r requirements.txt;
2
Since a docker container is already an isolated environment it does not give any extra advantage to use a vurtual env there. Also your answer does not seem to handle the actual problem and is just a general advice.
– Klaus D.
Nov 10 at 0:25
Since the error is that the module can't be found, I actually expect that this approach may resolve the issue. It ensures that deps are in place and it ensures that they are available to the interpreter. @KlausD.
– Charles Landau
Nov 10 at 0:28
add a comment |
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
0
down vote
With python in docker I definitely find it helpful to use a vendoring tool. Instead of copying the local site-packages, run a script or series of RUN
commands which:
- Establish the virtualenv
- Activate it
- call
pip install -r requirements.txt
Here's an example from a script I use in dev:
pip3 install virtualenv;
virtualenv --no-site-packages --python=python3.6 app;
virtualenv --relocatable app;
source ./app/bin/activate;
pip3 install -r requirements.txt;
2
Since a docker container is already an isolated environment it does not give any extra advantage to use a vurtual env there. Also your answer does not seem to handle the actual problem and is just a general advice.
– Klaus D.
Nov 10 at 0:25
Since the error is that the module can't be found, I actually expect that this approach may resolve the issue. It ensures that deps are in place and it ensures that they are available to the interpreter. @KlausD.
– Charles Landau
Nov 10 at 0:28
add a comment |
up vote
0
down vote
With python in docker I definitely find it helpful to use a vendoring tool. Instead of copying the local site-packages, run a script or series of RUN
commands which:
- Establish the virtualenv
- Activate it
- call
pip install -r requirements.txt
Here's an example from a script I use in dev:
pip3 install virtualenv;
virtualenv --no-site-packages --python=python3.6 app;
virtualenv --relocatable app;
source ./app/bin/activate;
pip3 install -r requirements.txt;
2
Since a docker container is already an isolated environment it does not give any extra advantage to use a vurtual env there. Also your answer does not seem to handle the actual problem and is just a general advice.
– Klaus D.
Nov 10 at 0:25
Since the error is that the module can't be found, I actually expect that this approach may resolve the issue. It ensures that deps are in place and it ensures that they are available to the interpreter. @KlausD.
– Charles Landau
Nov 10 at 0:28
add a comment |
up vote
0
down vote
up vote
0
down vote
With python in docker I definitely find it helpful to use a vendoring tool. Instead of copying the local site-packages, run a script or series of RUN
commands which:
- Establish the virtualenv
- Activate it
- call
pip install -r requirements.txt
Here's an example from a script I use in dev:
pip3 install virtualenv;
virtualenv --no-site-packages --python=python3.6 app;
virtualenv --relocatable app;
source ./app/bin/activate;
pip3 install -r requirements.txt;
With python in docker I definitely find it helpful to use a vendoring tool. Instead of copying the local site-packages, run a script or series of RUN
commands which:
- Establish the virtualenv
- Activate it
- call
pip install -r requirements.txt
Here's an example from a script I use in dev:
pip3 install virtualenv;
virtualenv --no-site-packages --python=python3.6 app;
virtualenv --relocatable app;
source ./app/bin/activate;
pip3 install -r requirements.txt;
answered Nov 10 at 0:13
Charles Landau
1,3951214
1,3951214
2
Since a docker container is already an isolated environment it does not give any extra advantage to use a vurtual env there. Also your answer does not seem to handle the actual problem and is just a general advice.
– Klaus D.
Nov 10 at 0:25
Since the error is that the module can't be found, I actually expect that this approach may resolve the issue. It ensures that deps are in place and it ensures that they are available to the interpreter. @KlausD.
– Charles Landau
Nov 10 at 0:28
add a comment |
2
Since a docker container is already an isolated environment it does not give any extra advantage to use a vurtual env there. Also your answer does not seem to handle the actual problem and is just a general advice.
– Klaus D.
Nov 10 at 0:25
Since the error is that the module can't be found, I actually expect that this approach may resolve the issue. It ensures that deps are in place and it ensures that they are available to the interpreter. @KlausD.
– Charles Landau
Nov 10 at 0:28
2
2
Since a docker container is already an isolated environment it does not give any extra advantage to use a vurtual env there. Also your answer does not seem to handle the actual problem and is just a general advice.
– Klaus D.
Nov 10 at 0:25
Since a docker container is already an isolated environment it does not give any extra advantage to use a vurtual env there. Also your answer does not seem to handle the actual problem and is just a general advice.
– Klaus D.
Nov 10 at 0:25
Since the error is that the module can't be found, I actually expect that this approach may resolve the issue. It ensures that deps are in place and it ensures that they are available to the interpreter. @KlausD.
– Charles Landau
Nov 10 at 0:28
Since the error is that the module can't be found, I actually expect that this approach may resolve the issue. It ensures that deps are in place and it ensures that they are available to the interpreter. @KlausD.
– Charles Landau
Nov 10 at 0:28
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53234740%2fpython-dependencies-and-docker-and-crontab%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Where do you install your requirements?
– Klaus D.
Nov 10 at 0:26