Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Please add cron support #438

Closed
TomashKhamlai opened this issue Dec 11, 2018 · 20 comments · May be fixed by #939
Closed

Please add cron support #438

TomashKhamlai opened this issue Dec 11, 2018 · 20 comments · May be fixed by #939

Comments

@TomashKhamlai
Copy link

ISSUE TYPE

  • Feature request

SUMMARY

Cron is often used by CMS

Goal

Magento Upgrade process is possible with cron. If this feature can have negative impact on the whole system then instructions for adding external cron docker in Magento tutorial would be enough to resolve this feature request

@cytopia
Copy link
Owner

cytopia commented Dec 15, 2018

@TomashKhamlai

For production images running Magento I also require cron, however I am using the one from the host system.

Either via docker

0 3 * * * root docker exec devilbox_php_1 php -d memory_limit=-1 /shared/httpd/magento/shell/indexer.php --reindex catalog_url

Or via Docker Compse:

0 3 * * * root cd /path/to/devilbox && docker-compose exec php php -d memory_limit=-1 /shared/httpd/magento/shell/indexer.php --reindex catalog_url

This works well for me. Let me know if this also works for you.

@cytopia cytopia self-assigned this Dec 15, 2018
@TomashKhamlai
Copy link
Author

@cytopia here how it looks now

*/5 * * * * tkhamlai cd ~/devilbox && docker-compose exec php php -d memory_limit=-1 /shared/httpd/magento2/magento2/bin/magento cron:run 2>&1 | grep -v "Ran jobs by schedule" >> /shared/httpd/magento2/magento2/var/log/magento.cron.log

*/5 * * * * tkhamlai cd ~/devilbox && docker-compose exec php php -d memory_limit=-1 /shared/httpd/magento2/magento2/update/cron.php >> /shared/httpd/magento2/magento2/var/log/update.cron.log
*/5 * * * * tkhamlai cd ~/devilbox && docker-compose exec php php -d memory_limit=-1 /shared/httpd/magento2/magento2/bin/magento setup:cron:run >> /shared/httpd/magento2/magento2/var/log/setup.cron.log

In the line where I call php bin/magento cron:run cron is not accessible in the container.

So this is not a solution for Magento 2

@cytopia
Copy link
Owner

cytopia commented Dec 23, 2018

@TomashKhamlai
Not sure what you mean by cron is not accessible in the container. The suggestion I proposed above is without using cron from the container.

If you are inside the PHP container, what is the exact command to run the Magento 2 cronjob?

@TomashKhamlai
Copy link
Author

Inside of PHP container I run php bin/magento cron:run twice. The first command schedules new changes to MySQL database and executes everything that already has been scheduled and the second one executes the schedule.

To check I use

mysql> select * from magento2.cron_schedule;
Empty set (0.02 sec)

So something wrong with my crontab or Magento 2 application wants to have acces to the cron executable.

@cytopia
Copy link
Owner

cytopia commented Dec 24, 2018

Does this work if you do a apt update && apt install -y cron inside the php container first?

@cytopia
Copy link
Owner

cytopia commented Dec 24, 2018

The next PHP-FPM release will have cron and crontab binaries installed. See above linked PR or here: devilbox/docker-php-fpm#78

@cytopia cytopia mentioned this issue Dec 25, 2018
4 tasks
@TomashKhamlai
Copy link
Author

TomashKhamlai commented Dec 26, 2018

Today, I tried to do:
git pull && ./update-docker.sh

Then I logged in to PHP container and issued this commands:

devilbox@php-7.1.24 in /shared/httpd/magento-dev/magento2 $ php bin/magento cron:install
Error during saving of crontab: sh: 7: crontab: not found
devilbox@php-7.1.24 in /shared/httpd/magento-dev/magento2 $ whereis cron
cron: /etc/cron.daily
devilbox@php-7.1.24 in /shared/httpd/magento-dev/magento2 $ 

And here what happens when it is installed by hands:

devilbox@php-7.1.24 in /shared/httpd/magento-dev/magento2 $ sudo apt install cron
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Suggested packages:
  anacron logrotate checksecurity
The following NEW packages will be installed:
  cron
0 upgraded, 1 newly installed, 0 to remove and 12 not upgraded.
Need to get 95.4 kB of archives.
After this operation, 257 kB of additional disk space will be used.
Get:1 http://deb.debian.org/debian stretch/main amd64 cron amd64 3.0pl1-128+deb9u1 [95.4 kB]
Fetched 95.4 kB in 0s (393 kB/s)
debconf: delaying package configuration, since apt-utils is not installed
Selecting previously unselected package cron.
(Reading database ... 37207 files and directories currently installed.)
Preparing to unpack .../cron_3.0pl1-128+deb9u1_amd64.deb ...
Unpacking cron (3.0pl1-128+deb9u1) ...
Setting up cron (3.0pl1-128+deb9u1) ...
Adding group `crontab' (GID 105) ...
Done.
update-rc.d: warning: start and stop actions are no longer supported; falling back to defaults
invoke-rc.d: could not determine current runlevel
invoke-rc.d: policy-rc.d denied execution of start.

I hope this helps.

@cytopia
Copy link
Owner

cytopia commented Dec 26, 2018

So once cron is installed, does the Magento cronjob command work then?

@cytopia
Copy link
Owner

cytopia commented Dec 31, 2018

@TomashKhamlai

Error during saving of crontab: sh: 7: crontab: not found

This has now been fixed the merging the above linked PR. Check out the latest master branch again.

@TomashKhamlai
Copy link
Author

@cytopia, what output is after issuing?

crontab -l

I see:

bash: crontab: command not found

I can see the changes in git log also I see the changes in code. Cron is still unavailable for me.

@cytopia
Copy link
Owner

cytopia commented Jan 3, 2019

what container version are you on? And what is your current Devilbox git hash?

@TomashKhamlai
Copy link
Author

commit 40d9bd1
Now I am updating docker-images, so my next try wiil be on devilbox/php-fpm:7.1-work-0.65

@TomashKhamlai
Copy link
Author

I have checked and I can verify that cron and crontab are not available inside of php container.

@cytopia
Copy link
Owner

cytopia commented Jan 3, 2019

OK, good to hear. Does the Magento cron command now succeed?

@TomashKhamlai
Copy link
Author

Nope. No cron, no crontab and php bin/magento cron:... also complains on that.

@cytopia
Copy link
Owner

cytopia commented Jan 3, 2019

It's both available in -0.65

root@php-7.1.25 in /shared/httpd $ cron
cron     crontab
root@php-7.1.25 in /shared/httpd $ which cron
/usr/sbin/cron
root@php-7.1.25 in /shared/httpd $ which crontab
/usr/bin/crontab
root@php-7.1.25 in /shared/httpd $

@cytopia cytopia added the tool label Mar 3, 2019
@SlimDeluxe
Copy link

SlimDeluxe commented Jun 6, 2019

I see support has been added but how do you overcome the fact that the crontab is not persistent? What use is it if it can't survive a restart.
Also, I tried adding a PHP command but it's never executed. The same command works fine if executed directly in CLI.
If try even this basic example
* * * * * php -v >> /shared/httpd/cron.log
nothing is added to cron.log, but if I try
* * * * * date >> /shared/httpd/cron.log
it works.

@SlimDeluxe
Copy link

I needed to follow these additional instructions in order to get it working

  1. Use absolute binary paths
  2. Add the -T parameter to docker-compose exec

So my command in the crontab would look like this:
* * * * * cd /home/omer/devilbox; /usr/local/bin/docker-compose exec -T php /usr/local/bin/php -v >> /home/omer/docker-scheduler.log

@masiorama
Copy link

Even following the previous suggestions I'm not able to make cronjob work.
I tried with crontab -e and added 2 * * * * date >> /shared/httpd/cron.log expecting some output but nothing so far.
Any help?

@martin-rueegg
Copy link

I found a solution that seems to work:

I created a file $HOME/devilbox/autostart/project-autostart.sh that will

  1. create functions to add and remove cron jobs
  2. start the cron daemon
  3. run all the .sh scripts in $HOST_PATH_HTTPD_DATADIR/*/.devilbox/autostart

In my project directory, I added the file .devilbox/autostart/cron.sh that adds the required cron jobs:

cron_add "* * * * * "$(which php)" /shared/httpd/my_project/htdocs/path/to/script.php some arguments >/dev/null 2>&1"

This can easily be tested with
cron_add "* * * * * date >> /shared/httpd/cron.log"

(which can again be removed later with cron_remove "* * * * * date >> /shared/httpd/cron.log")

I've also created some PRs to support this functionality out-of-the-box:

Hope this helps anyone.


File $HOME/devilbox/autostart/project-autostart.sh

#!/usr/bin/env bash


set -e
set -u
set -o pipefail

DEBUG_LEVEL=$(printenv DEBUG_ENTRYPOINT)
DEBUG_LEVEL=${DEBUG_LEVEL:-0}
export DEBUG_LEVEL

###########################################################
# Functions
############################################################

###
### Execute custom uesr-supplied scripts
###
execute_project_scripts() {
        local script_dir="${1}"
        local debug="${2}"

        if [ ! -d "${script_dir}" ]; then
                run "mkdir -p ${script_dir}" "${debug}"
        fi
        script_files="$( find -L "${script_dir}" -type f -iname '*.sh' | sort -n )"

        # loop over them line by line
        IFS='
        '
        for script_f in ${script_files}; do
                script_name="$( basename "${script_f}" )"
                log "info" "Executing project startup script: ${script_name}" "${debug}"
                if ! bash "${script_f}"; then
                        log "err" "Failed to execute script" "${debug}"
                        exit 1
                fi
        done
}

cron_add() {
  # save the entire line in one variable
  line="$*"

  # check if line already exists in crontab
  crontab -l -u devilbox | grep "$line" > /dev/null
  status=$?

  if [ $status -ne 0 ]
  then
    log "info" "cron: adding line '${line//\*/\*}' ..." "$DEBUG_LEVEL"
    (crontab -l -u devilbox; echo "$line";) | crontab -u devilbox -
  fi

  # make sure the cron service is running
  if ! service cron status >/dev/null
  then
    service cron start
  fi
}
export -f cron_add

cron_remove() {
  # save the entire line in one variable
  line=$*

  # check if line already exists in crontab
  crontab -l -u devilbox | grep "$line" > /dev/null
  status=$?

  if [ $status -eq 0 ]; then
    log "info" "cron: removing line '${line//\*/\*}' ..." "$DEBUG_LEVEL"
    (crontab -l -u devilbox | grep -v "$line";) | crontab -u devilbox -
  fi
}
export -f cron_remove

# this function is required, as long as it is not exported by /docker-entrypoint.d/100-base-libs.sh
log() {
        echo $1 $2
}


for project_dir in /shared/httpd/*
do
        if [ -d $project_dir/.devilbox/autostart ]; then
                project=$(basename "$project_dir")
                log "info" "Evaluating project startup scripts for '$project'" "$DEBUG_LEVEL"
                execute_project_scripts "$project_dir/.devilbox/autostart" "$DEBUG_LEVEL"
        fi
done

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants