Cron jobs

Cron wakes every minute, reads tables of jobs, and starts the ones whose schedule matches the clock. It does not load your shell profile. It does not use the timezone of the laptop you edited from. It does not mail you because a command exited non-zero. The job runs, or not, as one user, with a short PATH, under /bin/sh unless the table says otherwise. "It works when I type it" is almost always one of those.

The syntax here is Debian and Ubuntu cron, and cronie on RHEL and Fedora, except where they differ. The service is cron on Debian and Ubuntu, and crond on cronie. systemctl is-active cron answers whether the daemon is running. A container image that does not run it will not grow jobs from a file sitting in /etc/cron.d.

Who may install a user table is a separate switch. If /etc/cron.allow exists, only users named in it may run crontab. If it does not exist and /etc/cron.deny does, users named there may not. Root is not blocked by either file. You (deploy) are not allowed to use this program (crontab) is this list, not a broken schedule.

The five fields

Five schedule fields, then the command: minute, hour, day of month, month, day of week.

15 3 * * * /usr/local/bin/backup-shop

That is 03:15 every day, in the daemon's timezone. 0 9-17 * * 1-5 is every hour from 09:00 to 17:00, Monday to Friday. */15 * * * * is every fifteen minutes. mon and jan are the three-letter names, any case. Sunday is 0, 7, or sun. Some schedulers and hosted panels number Sunday as 1 and Saturday as 7. That is a different dialect. On Linux cron, 1 is Monday. A star means every value of that field. */15 in the minute field is minutes 0, 15, 30, and 45. It starts on the clock, not fifteen minutes after you saved the table. 1-59/15 would skip minute 0. People who wanted "about every fifteen minutes, but not on the hour" need the range written out.

If day-of-month and day-of-week are both restricted, Vixie cron and cronie run the job when either field matches. 0 0 1 * 1 is midnight on the first of the month and midnight every Monday. It is not "Mondays that are the first". A first Monday has to be tested in the script, with the table firing every Monday.

@daily is 0 0 * * *, @hourly is 0 * * * *, @weekly is 0 0 * * 0, @monthly is 0 0 1 * *. @reboot runs when the daemon starts, including a later restart of the service. It does not wait for the network or for a slow mount. A job that depends on either wants a systemd unit with an After= line.

The command is the rest of the line, handed to sh -c. Redirects and pipes work. A percent sign does not reach the shell unchanged. That is its own section, because it is how date in a crontab stops meaning date.

A user crontab

crontab -e edits your table. crontab -l prints it. sudo crontab -e -u deploy edits deploy's, which is what you want when the job must not be root. crontab -r deletes the table with no prompt. Save a copy before you experiment:

crontab -l > "$HOME/crontab.bak"

Debian stores the table at /var/spool/cron/crontabs/<user>. cronie uses /var/spool/cron/<user>. Let crontab write it. There is no username column. A username typed out of habit becomes the program cron tries to run.

Lines of NAME=value at the top set the environment for the jobs below them. A line further down does not apply to a job above it. No export, and no spaces around =. Cron also sets HOME, LOGNAME, and SHELL from the account. It does not import variables you exported in an SSH session. A backup that works because DATABASE_URL was exported at the prompt will not see it here. Put that assignment in the table, or in a file the script reads by an absolute path.

crontab -e opens $VISUAL or $EDITOR, then installs the table only if it parses. A bad line produces errors in crontab file, can't install and leaves the previous table in place. Read that message. A save that appeared to succeed in the editor has not necessarily been installed. The file also has to end with a newline. A last job with no newline after it is dropped or rejected, depending on the cron build, and the symptom is the final line quietly doing nothing.

MAILTO=""
CRON_TZ=Europe/London
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

15 3 * * * /usr/local/bin/backup-shop >> /home/deploy/log/backup-shop.log 2>&1
*/15 * * * * /usr/bin/flock -n /home/deploy/log/backup-shop.lock /usr/local/bin/backup-shop >> /home/deploy/log/backup-shop.log 2>&1

Create /home/deploy/log first. Cron will not. A missing directory makes the shell print No such file or directory, and that sentence is the mail unless MAILTO is empty. /var/log is the other usual miss: root can write it, deploy often cannot. Put the log somewhere the user owns, or create the file and chown it.

flock -n skips the start when a previous run still holds the lock, and it prints nothing. A fifteen-minute table whose command takes twenty minutes otherwise stacks copies. The lock file has to be writable by that user.

The server clock

The numbers are the daemon's timezone. timedatectl prints Time zone. A cloud image on UTC and a laptop on Europe/London make 0 9 * * 1-5 into 09:00 UTC: 09:00 GMT in winter and 10:00 BST in summer. The job did not drift. The clock you compared it with is a different zone.

CRON_TZ=Europe/London, above the jobs, makes Debian cron and cronie read the numbers in that zone, including British Summer Time. A setting covers the lines after it.

The spring clock change skips an hour, so 02:30 does not occur and that job does not run. The autumn change repeats the hour, so 02:30 can run twice. Use 03:15 or later for a daily job in a zone that does this, or schedule in UTC on purpose. date inside the job prints the zone the job has. date in your SSH session prints the zone the session has. A difference is a TZ variable in one of them.

stdout, stderr, and MAILTO

Cron collects stdout and stderr. Non-empty output is mailed to the table's owner, or to MAILTO if that is set. MAILTO="" discards it. Empty output sends nothing, and a non-zero exit that prints nothing sends nothing either. Mail means "there was text", not "the job failed".

Write the failure somewhere you will look. This appends both streams:

15 3 * * * /usr/local/bin/backup-shop >> /home/deploy/log/backup-shop.log 2>&1

2>&1 belongs after the append. With both streams in the file, cron has an empty buffer and sends no mail. A machine with no mailer was not going to deliver it anyway. Relying on cron's mail needs output, a MAILTO that resolves, and a working mailer.

The shell is /bin/sh unless SHELL=/bin/bash. .bashrc is not read. source is not POSIX. nvm, pyenv, and a PATH that only exists in an interactive shell are absent. Call binaries by absolute path even if you set SHELL.

/etc/cron.d and the periodic directories

A user table's command starts at field six and runs as that user. /etc/crontab and /etc/cron.d/* put the username between the schedule and the command.

SHELL=/bin/sh
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
MAILTO=""

15 3 * * * deploy /usr/local/bin/backup-shop >> /home/deploy/log/backup-shop.log 2>&1

Save it as /etc/cron.d/backup-shop, owned by root, not writable by group or others. Mode 644 is fine. It need not be executable, and you do not load it with crontab. On Debian and Ubuntu the name may use only letters, digits, underscores, and hyphens. backup.cron and shop.daily are skipped, silently. cronie cares less about a dot and just as much about owner and mode. Avoid the dot.

/etc/cron.hourly and the daily, weekly, and monthly directories are run-parts. The script must be executable. On Debian a dot in the name is ignored, so backup.sh never runs and backup-shop does. run-parts --test /etc/cron.daily lists what would run. The clock time comes from /etc/crontab or from anacron, not from the script. With anacron installed, a daily script can wait until the machine is next on. User crontabs are not replayed. Miss 03:15 because the host was off, and that run is gone.

Test as the user, without your SSH environment:

sudo -u deploy env -i HOME=/home/deploy PATH=/usr/bin:/bin /bin/sh -c '/usr/local/bin/backup-shop'

If that cannot find a program or cannot read a file, the table cannot either. The daemon's own log only shows that it started a shell. Debian: grep CRON /var/log/syslog prints (deploy) CMD (/usr/local/bin/backup-shop ...). Journal hosts: journalctl -u cron --since "1 hour ago". Forked is not the same as succeeded. The job log is the other half.

systemd timers, briefly

A timer unit starts a service on a calendar. systemctl list-timers shows the next fire. The log is journalctl -u backup.service, not mail. Persistent=true runs a job the machine missed while it was off, which cron does not. Prefer a timer when you need that catch-up, the journal, or an order against other units. Prefer cron for one command on a host that already runs it.

PATH and the percent sign

A user crontab starts with PATH=/usr/bin:/bin on Debian, Ubuntu, and cronie. /usr/local/bin is not included, nor is /opt, nor a home directory. node: not found in the mail, straight after a successful SSH session, is this. .bashrc is what made the SSH session work.

Set PATH= in the table, or write absolute paths. command -v backup-shop in the session where it works prints the path to copy. nvm and pyenv only exist as shell setup. Point at the binary in the versioned directory.

% is special to crontab. Unescaped, each one becomes a newline, and everything after the first is standard input to the command. The shell never sees your line.

0 * * * * /usr/bin/date +%Y-%m-%d >> /home/deploy/log/tick.log

That runs /usr/bin/date + and feeds it the rest, including the redirect, on stdin. Quotes do not change the parse. Cron handles % before the shell starts. Escape every one:

0 * * * * /usr/bin/date +\%Y-\%m-\%d >> /home/deploy/log/tick.log

The same applies inside a format string that only looks quoted. A script file is ordinary text once cron has executed it: put date +%Y-%m-%d in /usr/local/bin/backup-shop and the percent signs are the shell's problem, which is to say they work.

Cron chdirs to the account's home first. Relative paths are relative to that home, not to wherever you happened to be sitting. Write absolute paths for the program, the log, and the files it opens.

An interactive git pull can work because an agent is loaded. Cron has no SSH_AUTH_SOCK and will not type a key passphrase. Give the job its own key. The SSH note is how that public line is installed, including command= and restrict so the key can do one thing. The laptop agent does not travel into the table.

Pitfalls

• User crontab PATH is /usr/bin:/bin. Add a PATH= line or absolute paths, and rehearse with env -i. nvm will not be there.
• % must be written \%. Quotes do not escape it. date +%Y-%m-%d in the table is not the command you run in a terminal.
• Day-of-month and day-of-week are OR when both are pinned down. 0 0 1 * 1 is not the first Monday.
• Hours follow the server zone, or CRON_TZ. A laptop on British Summer Time does not move a UTC table. Avoid 02:30 in a zone that skips or repeats that hour.
• A user table has no username field. /etc/cron.d requires one. Swap them and the username is executed, or the job runs as the wrong account.
• On Debian, a dot in a name under /etc/cron.d or /etc/cron.daily means the file is ignored. backup.sh does not run. Those files are owned by root and not group-writable. run-parts scripts also have to be executable.
• Cron mails output, not a non-zero status. MAILTO="" or a redirect leaves the log file. No mailer means mail was never the alert.
• The log directory must exist and be writable by the job's user.
• crontab -r deletes the table.
• A missed minute is not replayed. A systemd timer with Persistent=true is the one that catches up.

Run it once as the right user with an empty environment. Then check that cron forked it, and that the job log contains what the command printed.