Prevent Overlapping Cron Jobs That Outrun Their Schedule

In this article

The schedule was shorter than the work

Mautic needs periodic commands for segments, campaigns, broadcasts, imports, webhooks, reports, cleanup, and queued work. The repository grouped those commands into three lanes:

• a long segment-update lane;
• a quick-update lane for campaigns, imports, webhooks, reports, and cleanup;
• an email lane for broadcasts and queued messages.

Each wrapper was scheduled once per minute. Before doing anything, it looked for a matching Mautic process. If one existed, the wrapper logged that it was already running and exited. If none existed, the wrapper started its lane. That meant the next segment cycle could begin within a minute of the previous one finishing without my guessing whether the job needed four hours today or six.

The grouping was useful. A long segment calculation did not have to prevent email from leaving. The flaw was in how each lane claimed ownership.

pgrep is an observation, not a lock

The segment wrapper begins with a process search:

IS_RUNNING=$(pgrep -fa "console mautic:(segment)")

if [ -z "$IS_RUNNING" ]; then
  php "$MAUTIC_BASE/bin/console" mautic:segment:update
fi

That check can tell me a matching process existed at one instant. It cannot reserve the right to start the next process. Two cron invocations can both run pgrep before either reaches PHP, both see an empty result, and both start the job. This is the familiar check-then-act race.

The match is also broader than the wrapper’s ownership. A command started manually can block the scheduled job. Another Mautic installation with a similar command line can match. A harmless change to the console command can stop matching and quietly remove the protection.

An advisory file lock makes acquisition one operation. The non-blocking -n option tells flock to fail immediately when another process owns the lock. A file descriptor keeps the lock attached to the running shell:

exec 9>>/run/lock/mautic/segments.lock || exit 1
if flock -n -E 200 9; then
  :
else
  status=$?
  if (( status == 200 )); then
    exit 0  # Another invocation owns this lock.
  fi
  exit "$status"  # A lock error must not look like a skipped run.
fi

This Bash fragment requires an existing lock directory writable by the job’s account. Exit code 200 is reserved here for contention; other failures remain failures. The full wrapper below logs which path occurred.

Give each lane its own stable lock file on a local filesystem. Every invocation, including a manual run, must cooperate by using that same lock. A separate host with its own local file is not covered. The lock is released when the last descriptor holding it closes; a child that inherits the descriptor can keep it alive after the wrapper exits. Do not delete an active lock file: another invocation could create a different file at the same path and run concurrently. Check filesystem support before relying on this across network storage.

The old script could fail and still look successful

The quick-update wrapper runs several PHP commands one after another. It appends each command’s standard output to a log, but it does not test the exit status before continuing. Bash normally returns the status of the last command executed. If an early Mautic command fails and a later command succeeds, cron can receive zero.

Standard error is not appended to the same file either:

php $MAUTIC_BASE/bin/console mautic:campaigns:update \
  $MAUTIC_OPTS $CAMPAIGNS_BATCH_LIMIT >> $LOGFILE

Depending on cron’s mail configuration, the useful error may arrive somewhere else or nowhere anyone checks. The local log can show the start message and ordinary output without the failure that explains why work stopped.

I would make the shell strict, quote paths, redirect both streams once, and explicitly record the result:

#!/usr/bin/env bash
set -Eeuo pipefail
export PATH=/usr/bin:/bin
umask 027

: "${MAUTIC_BASE:?Set MAUTIC_BASE to the absolute Mautic installation path}"
readonly php_bin="${PHP_BIN:-/usr/bin/php}"
readonly log="${LOG_FILE:-/var/log/mautic/segments.log}"
readonly lock="${LOCK_FILE:-/run/lock/mautic/segments.lock}"
exec >>"$log" 2>&1

exec 9>>"$lock"
if flock -n -E 200 9; then
  :
else
  lock_status=$?
  if (( lock_status == 200 )); then
    printf '%s already running; trigger skipped\n' "$(date -Is)"
    exit 0
  fi
  printf '%s lock failed status=%s\n' "$(date -Is)" "$lock_status"
  exit "$lock_status"
fi

started=$SECONDS
printf '%s starting segment update\n' "$(date -Is)"
if "$php_bin" "$MAUTIC_BASE/bin/console" mautic:segments:update \
    --no-interaction --no-ansi --batch-limit=300; then
  status=0
else
  status=$?
fi
printf '%s finished status=%s duration_seconds=%s\n' \
  "$(date -Is)" "$status" "$((SECONDS - started))"
exit "$status"

Before scheduling this, set MAUTIC_BASE to the absolute installation path in the job’s environment; cron will not normally load your interactive shell profile. Set PHP_BIN if the application needs a different PHP executable. The default paths assume a local /run/lock/mautic directory and /var/log/mautic, provisioned for the service account with restricted permissions. Arrange for the runtime directory to be recreated after reboot, for example through your service manager or tmpfiles configuration. Alternatively, set LOCK_FILE and LOG_FILE to appropriate absolute paths. Keep the wrapper and its configuration outside directories writable by untrusted users.

Run it as the application’s service account, not root. Confirm the console path, PHP version, command options, directory permissions, and log rotation before enabling a schedule. This example redirects errors to its log, so cron mail alone is not a failure alert. A monitoring check must inspect failures and successful-completion freshness. Opening a missing log directory fails before redirection, so that error goes to the caller’s stderr.

The if is deliberate. It lets the script capture a failure while running with set -e. The final exit makes the Mautic command’s result the wrapper’s result. A zero from the duplicate-run path is also deliberate: the lock owner is doing the work, so the new trigger has nothing to do.

I would not use an IP substring as the primary-host election

The repository was designed for a pair of Mautic servers that shared data. It could be deployed to both, but only the server holding a configured virtual IP should run scheduled work. The check was:

ip a | grep $MAUTIC_VIP

That searches human-formatted command output with an unquoted regular expression. It can match a substring, an unexpected interface line, or regex punctuation. More importantly, it turns a deployment role into a fact inferred at runtime.

I would now install the cron or systemd timer only on the designated scheduler host. If the same artifact must exist everywhere, I would use a root-owned role file or exact configuration value and fail closed when it is absent. A high-availability scheduler needs a real lease or leader-election mechanism; grepping an address is not one.

This distinction matters because these commands mutate shared application state and can send email. “Only one server happened to match” is weaker than “the deployment names one scheduler owner.”

Skipped triggers do not queue up

With a non-blocking lock, an every-minute cron entry means “start within one minute whenever this lane becomes idle.” It does not mean the command runs once per minute. Contention discards that invocation; it does not save a run for later. This works for a backlog processor only when the application durably retains the pending work. A task that must execute once for every scheduled interval needs different scheduling semantics.

It can also create a permanent loop. If a successful segment update takes four hours, the next update begins almost immediately. There is no idle window for maintenance, no cooldown after a suspiciously fast failure, and no distinction between catching up and continuously recomputing.

I decide this per lane:

Current Mautic documentation recommends staggering required jobs rather than launching them in the same minute. It documents batch and maximum-contact controls for segment and campaign work. For long-lived message consumers, it recommends a memory, message, or time limit. Those controls turn “keep running” into a bounded operating decision.

The command names are part of the maintenance burden

The repository’s segment command is mautic:segment:update. Current Mautic documentation uses mautic:segments:update. The email path has also changed across Mautic generations, with current queued delivery centered on a bounded Messenger consumer.

I would not silently update those names in an article and pretend the historical deployment used today’s release. I would first record the installed Mautic version, run bin/console list mautic on a safe instance, and compare every configured command and option with the documentation for that exact version.

A wrapper that runs reliably can still automate a command that no longer exists. Version compatibility belongs in the deployment check.

What I would monitor

A timestamped text log is better than an invisible crontab, but it is not enough to prove freshness. I want each lane to emit four values: start time, finish time, exit status, and duration. I also want a separate freshness check that alerts when no successful completion has appeared within the expected window.

The duration is not just a graph. It changes the schedule. A four-hour median with a six-hour tail needs a different interval from a job that occasionally hangs for a day. A run that finishes in two seconds may be more suspicious than one that takes five hours.

For a destructive cleanup command, the original wrapper’s dry run before the real execution was a good instinct. I would keep it, make the deletion result independently visible, and verify the relevant database backup before allowing the real cleanup to run.

My corrected deployment checklist

1. Name one owner for each scheduled lane.
2. Use an atomic lock with a lane-specific identity.
3. Set batch, message, memory, contact, or time boundaries where the command supports them.
4. Capture stdout and stderr in one owned log destination.
5. Return the real application command’s exit status.
6. Record duration and alert on stale successful completion.
7. Test command names and options against the installed application version.
8. Keep email, segment, cleanup, and quick-update locks separate unless shared state requires serialization.
9. Test the duplicate trigger, failed command, terminated process, log rotation, and next successful run.

The old scripts solved a real operational problem: stop guessing a four-hour interval and keep independent work moving. The replacement I would trust keeps that shape, but acquires ownership atomically and reports failures separately from skipped triggers.

The script and host evidence

The source review uses the current public mautic-cron-scripts repository. The scheduling and command guidance is cross-checked against Mautic’s official cron-job documentation. Lock behavior comes from the util-linux flock(1) manual, and exit-status behavior comes from the official GNU Bash manual.

The 30-plus-hour and roughly four-to-six-hour runtimes are historical measurements recorded in the repository, not fresh benchmarks. I inspected the shell source and current documentation. Isolated shell fixtures exercise successful and failed dummy commands, real lock contention, a simulated lock-command error, missing configuration and directories, a missing executable, and reacquiring the lock. They do not exercise Mautic or a production scheduler. I did not run Mautic, send email, execute cleanup, install the crontab, test a network filesystem lock, or deploy the replacement wrapper for this article. The replacement is an explained pattern, not a drop-in promise for every Mautic version.

Disclosure: I maintain the linked repository.