Python dependencies and Docker and Crontab











up vote
0
down vote

favorite












I'm having a strange issue when trying to run a Python script via Crontab within a Docker container. Here's the setup.



I have a Dockerfile like this:



FROM python:3-onbuild

# Install cron and dependencies
RUN apt-get update && apt-get -y install cron

# Add crontab file in the cron directory
ADD crontab /etc/cron.d/simple-cron
WORKDIR /src
COPY . /src

# Give execution rights on the cron job
RUN chmod 0644 /etc/cron.d/simple-cron
RUN chmod 777 /usr/local/lib/python3.6/site-packages

# Create the log file to be able to run tail
RUN touch /var/log/cron.log

# Run the command on container startup
CMD ["cron", "-f"]


And a crontab file like this:



* * * * * root python /src/test.py > /proc/1/fd/1 2>/proc/1/fd/2


This executes correctly with no problems when I start the container. The first line of my Python file is import redis, which causes the script to fail, saying that the module cannot be found.



The interesting thing is that when I manually execute docker exec -it [container_name] python test.py, everything works exactly as expected.



What could be the issue here? I'm thinking that the requirements file isn't being installed to the right location?










share|improve this question






















  • Where do you install your requirements?
    – Klaus D.
    Nov 10 at 0:26















up vote
0
down vote

favorite












I'm having a strange issue when trying to run a Python script via Crontab within a Docker container. Here's the setup.



I have a Dockerfile like this:



FROM python:3-onbuild

# Install cron and dependencies
RUN apt-get update && apt-get -y install cron

# Add crontab file in the cron directory
ADD crontab /etc/cron.d/simple-cron
WORKDIR /src
COPY . /src

# Give execution rights on the cron job
RUN chmod 0644 /etc/cron.d/simple-cron
RUN chmod 777 /usr/local/lib/python3.6/site-packages

# Create the log file to be able to run tail
RUN touch /var/log/cron.log

# Run the command on container startup
CMD ["cron", "-f"]


And a crontab file like this:



* * * * * root python /src/test.py > /proc/1/fd/1 2>/proc/1/fd/2


This executes correctly with no problems when I start the container. The first line of my Python file is import redis, which causes the script to fail, saying that the module cannot be found.



The interesting thing is that when I manually execute docker exec -it [container_name] python test.py, everything works exactly as expected.



What could be the issue here? I'm thinking that the requirements file isn't being installed to the right location?










share|improve this question






















  • Where do you install your requirements?
    – Klaus D.
    Nov 10 at 0:26













up vote
0
down vote

favorite









up vote
0
down vote

favorite











I'm having a strange issue when trying to run a Python script via Crontab within a Docker container. Here's the setup.



I have a Dockerfile like this:



FROM python:3-onbuild

# Install cron and dependencies
RUN apt-get update && apt-get -y install cron

# Add crontab file in the cron directory
ADD crontab /etc/cron.d/simple-cron
WORKDIR /src
COPY . /src

# Give execution rights on the cron job
RUN chmod 0644 /etc/cron.d/simple-cron
RUN chmod 777 /usr/local/lib/python3.6/site-packages

# Create the log file to be able to run tail
RUN touch /var/log/cron.log

# Run the command on container startup
CMD ["cron", "-f"]


And a crontab file like this:



* * * * * root python /src/test.py > /proc/1/fd/1 2>/proc/1/fd/2


This executes correctly with no problems when I start the container. The first line of my Python file is import redis, which causes the script to fail, saying that the module cannot be found.



The interesting thing is that when I manually execute docker exec -it [container_name] python test.py, everything works exactly as expected.



What could be the issue here? I'm thinking that the requirements file isn't being installed to the right location?










share|improve this question













I'm having a strange issue when trying to run a Python script via Crontab within a Docker container. Here's the setup.



I have a Dockerfile like this:



FROM python:3-onbuild

# Install cron and dependencies
RUN apt-get update && apt-get -y install cron

# Add crontab file in the cron directory
ADD crontab /etc/cron.d/simple-cron
WORKDIR /src
COPY . /src

# Give execution rights on the cron job
RUN chmod 0644 /etc/cron.d/simple-cron
RUN chmod 777 /usr/local/lib/python3.6/site-packages

# Create the log file to be able to run tail
RUN touch /var/log/cron.log

# Run the command on container startup
CMD ["cron", "-f"]


And a crontab file like this:



* * * * * root python /src/test.py > /proc/1/fd/1 2>/proc/1/fd/2


This executes correctly with no problems when I start the container. The first line of my Python file is import redis, which causes the script to fail, saying that the module cannot be found.



The interesting thing is that when I manually execute docker exec -it [container_name] python test.py, everything works exactly as expected.



What could be the issue here? I'm thinking that the requirements file isn't being installed to the right location?







python docker redis cron






share|improve this question













share|improve this question











share|improve this question




share|improve this question










asked Nov 9 at 23:53









Austin Osborne

84




84












  • Where do you install your requirements?
    – Klaus D.
    Nov 10 at 0:26


















  • Where do you install your requirements?
    – Klaus D.
    Nov 10 at 0:26
















Where do you install your requirements?
– Klaus D.
Nov 10 at 0:26




Where do you install your requirements?
– Klaus D.
Nov 10 at 0:26












1 Answer
1






active

oldest

votes

















up vote
0
down vote













With python in docker I definitely find it helpful to use a vendoring tool. Instead of copying the local site-packages, run a script or series of RUN commands which:




  1. Establish the virtualenv

  2. Activate it

  3. call pip install -r requirements.txt


Here's an example from a script I use in dev:



pip3 install virtualenv;
virtualenv --no-site-packages --python=python3.6 app;
virtualenv --relocatable app;
source ./app/bin/activate;
pip3 install -r requirements.txt;





share|improve this answer

















  • 2




    Since a docker container is already an isolated environment it does not give any extra advantage to use a vurtual env there. Also your answer does not seem to handle the actual problem and is just a general advice.
    – Klaus D.
    Nov 10 at 0:25










  • Since the error is that the module can't be found, I actually expect that this approach may resolve the issue. It ensures that deps are in place and it ensures that they are available to the interpreter. @KlausD.
    – Charles Landau
    Nov 10 at 0:28











Your Answer






StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});


}
});














draft saved

draft discarded


















StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53234740%2fpython-dependencies-and-docker-and-crontab%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown

























1 Answer
1






active

oldest

votes








1 Answer
1






active

oldest

votes









active

oldest

votes






active

oldest

votes








up vote
0
down vote













With python in docker I definitely find it helpful to use a vendoring tool. Instead of copying the local site-packages, run a script or series of RUN commands which:




  1. Establish the virtualenv

  2. Activate it

  3. call pip install -r requirements.txt


Here's an example from a script I use in dev:



pip3 install virtualenv;
virtualenv --no-site-packages --python=python3.6 app;
virtualenv --relocatable app;
source ./app/bin/activate;
pip3 install -r requirements.txt;





share|improve this answer

















  • 2




    Since a docker container is already an isolated environment it does not give any extra advantage to use a vurtual env there. Also your answer does not seem to handle the actual problem and is just a general advice.
    – Klaus D.
    Nov 10 at 0:25










  • Since the error is that the module can't be found, I actually expect that this approach may resolve the issue. It ensures that deps are in place and it ensures that they are available to the interpreter. @KlausD.
    – Charles Landau
    Nov 10 at 0:28















up vote
0
down vote













With python in docker I definitely find it helpful to use a vendoring tool. Instead of copying the local site-packages, run a script or series of RUN commands which:




  1. Establish the virtualenv

  2. Activate it

  3. call pip install -r requirements.txt


Here's an example from a script I use in dev:



pip3 install virtualenv;
virtualenv --no-site-packages --python=python3.6 app;
virtualenv --relocatable app;
source ./app/bin/activate;
pip3 install -r requirements.txt;





share|improve this answer

















  • 2




    Since a docker container is already an isolated environment it does not give any extra advantage to use a vurtual env there. Also your answer does not seem to handle the actual problem and is just a general advice.
    – Klaus D.
    Nov 10 at 0:25










  • Since the error is that the module can't be found, I actually expect that this approach may resolve the issue. It ensures that deps are in place and it ensures that they are available to the interpreter. @KlausD.
    – Charles Landau
    Nov 10 at 0:28













up vote
0
down vote










up vote
0
down vote









With python in docker I definitely find it helpful to use a vendoring tool. Instead of copying the local site-packages, run a script or series of RUN commands which:




  1. Establish the virtualenv

  2. Activate it

  3. call pip install -r requirements.txt


Here's an example from a script I use in dev:



pip3 install virtualenv;
virtualenv --no-site-packages --python=python3.6 app;
virtualenv --relocatable app;
source ./app/bin/activate;
pip3 install -r requirements.txt;





share|improve this answer












With python in docker I definitely find it helpful to use a vendoring tool. Instead of copying the local site-packages, run a script or series of RUN commands which:




  1. Establish the virtualenv

  2. Activate it

  3. call pip install -r requirements.txt


Here's an example from a script I use in dev:



pip3 install virtualenv;
virtualenv --no-site-packages --python=python3.6 app;
virtualenv --relocatable app;
source ./app/bin/activate;
pip3 install -r requirements.txt;






share|improve this answer












share|improve this answer



share|improve this answer










answered Nov 10 at 0:13









Charles Landau

1,3951214




1,3951214








  • 2




    Since a docker container is already an isolated environment it does not give any extra advantage to use a vurtual env there. Also your answer does not seem to handle the actual problem and is just a general advice.
    – Klaus D.
    Nov 10 at 0:25










  • Since the error is that the module can't be found, I actually expect that this approach may resolve the issue. It ensures that deps are in place and it ensures that they are available to the interpreter. @KlausD.
    – Charles Landau
    Nov 10 at 0:28














  • 2




    Since a docker container is already an isolated environment it does not give any extra advantage to use a vurtual env there. Also your answer does not seem to handle the actual problem and is just a general advice.
    – Klaus D.
    Nov 10 at 0:25










  • Since the error is that the module can't be found, I actually expect that this approach may resolve the issue. It ensures that deps are in place and it ensures that they are available to the interpreter. @KlausD.
    – Charles Landau
    Nov 10 at 0:28








2




2




Since a docker container is already an isolated environment it does not give any extra advantage to use a vurtual env there. Also your answer does not seem to handle the actual problem and is just a general advice.
– Klaus D.
Nov 10 at 0:25




Since a docker container is already an isolated environment it does not give any extra advantage to use a vurtual env there. Also your answer does not seem to handle the actual problem and is just a general advice.
– Klaus D.
Nov 10 at 0:25












Since the error is that the module can't be found, I actually expect that this approach may resolve the issue. It ensures that deps are in place and it ensures that they are available to the interpreter. @KlausD.
– Charles Landau
Nov 10 at 0:28




Since the error is that the module can't be found, I actually expect that this approach may resolve the issue. It ensures that deps are in place and it ensures that they are available to the interpreter. @KlausD.
– Charles Landau
Nov 10 at 0:28


















draft saved

draft discarded




















































Thanks for contributing an answer to Stack Overflow!


  • Please be sure to answer the question. Provide details and share your research!

But avoid



  • Asking for help, clarification, or responding to other answers.

  • Making statements based on opinion; back them up with references or personal experience.


To learn more, see our tips on writing great answers.





Some of your past answers have not been well-received, and you're in danger of being blocked from answering.


Please pay close attention to the following guidance:


  • Please be sure to answer the question. Provide details and share your research!

But avoid



  • Asking for help, clarification, or responding to other answers.

  • Making statements based on opinion; back them up with references or personal experience.


To learn more, see our tips on writing great answers.




draft saved


draft discarded














StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53234740%2fpython-dependencies-and-docker-and-crontab%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown





















































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown

































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown







Popular posts from this blog

Schultheiß

Verwaltungsgliederung Dänemarks

Liste der Kulturdenkmale in Wilsdruff