Prevent Overlapping Cron Jobs That Outrun Their Schedule

When scheduled work can outlive its interval, use an atomic lock instead of a process-name search. Preserve meaningful exit status, record duration, define which host owns the job, and treat every-minute cron as a queue trigger rather than a promise of frequency.

The schedule was shorter than the work

Mautic needs periodic commands for segments, campaigns, broadcasts, imports, webhooks, reports, cleanup, and queued work. The repository grouped those commands into three lanes:

• a long segment-update lane;
• a quick-update lane for campaigns, imports, webhooks, reports, and cleanup;
• an email lane for broadcasts and queued messages.

Each wrapper was scheduled once per minute. Before doing anything, it looked for a matching Mautic process. If one existed, the wrapper logged that it was already running and exited. If none existed, the wrapper started its lane. That meant the next segment cycle could begin within a minute of the previous one finishing without my guessing whether the job needed four hours today or six.

The grouping was useful. A long segment calculation did not have to prevent email from leaving. The flaw was in how each lane claimed ownership.

pgrep is an observation, not a lock

The segment wrapper begins with a process search:

IS_RUNNING=$(pgrep -fa "console mautic:(segment)")

if [ -z "$IS_RUNNING" ]; then
  php "$MAUTIC_BASE/bin/console" mautic:segment:update
fi

That check can tell me a matching process existed at one instant. It cannot reserve the right to start the next process. Two cron invocations can both run pgrep before either reaches PHP, both see an empty result, and both start the job. This is the familiar check-then-act race.

The match is also broader than the wrapper’s ownership. A command started manually can block the scheduled job. Another Mautic installation with a similar command line can match. A harmless change to the console command can stop matching and quietly remove the protection.

An advisory file lock makes acquisition one operation. The non-blocking -n option tells flock to fail immediately when another process owns the lock. A file descriptor keeps the lock attached to the running shell:

exec 9>/run/lock/mautic-segments.lock
if ! flock -n 9; then
  exit 0
fi

The lock path is the identity. I can give segments, quick updates, and email separate locks, preserving the useful parallelism of the original design. The lock disappears when the process exits and closes the descriptor, including most crash paths. I still need to check whether the target filesystem supports flock; the util-linux manual warns that some network filesystems have limited or failed lock semantics.

The old script could fail and still look successful

The quick-update wrapper runs several PHP commands one after another. It appends each command’s standard output to a log, but it does not test the exit status before continuing. Bash normally returns the status of the last command executed. If an early Mautic command fails and a later command succeeds, cron can receive zero.

Standard error is not appended to the same file either:

php $MAUTIC_BASE/bin/console mautic:campaigns:update \
  $MAUTIC_OPTS $CAMPAIGNS_BATCH_LIMIT >> $LOGFILE

Depending on cron’s mail configuration, the useful error may arrive somewhere else or nowhere anyone checks. The local log can show the start message and ordinary output without the failure that explains why work stopped.

I would make the shell strict, quote paths, redirect both streams once, and explicitly record the result:

#!/usr/bin/env bash
set -Eeuo pipefail

readonly log=/var/log/mautic/segments.log
exec >>"$log" 2>&1

exec 9>/run/lock/mautic-segments.lock
if ! flock -n 9; then
  printf '%s already running\n' "$(date -Is)"
  exit 0
fi

started=$SECONDS
printf '%s starting segment update\n' "$(date -Is)"

if php "$MAUTIC_BASE/bin/console" mautic:segments:update \
    --no-interaction --no-ansi --batch-limit=300; then
  status=0
else
  status=$?
fi

printf '%s finished status=%s duration_seconds=%s\n' \
  "$(date -Is)" "$status" "$((SECONDS - started))"
exit "$status"

The if is deliberate. It lets the script capture a failure while running with set -e. The final exit makes the Mautic command’s result the wrapper’s result. A zero from the duplicate-run path is also deliberate: the lock owner is doing the work, so the new trigger has nothing to do.

I would not use an IP substring as the primary-host election

The repository was designed for a pair of Mautic servers that shared data. It could be deployed to both, but only the server holding a configured virtual IP should run scheduled work. The check was:

ip a | grep $MAUTIC_VIP

That searches human-formatted command output with an unquoted regular expression. It can match a substring, an unexpected interface line, or regex punctuation. More importantly, it turns a deployment role into a fact inferred at runtime.

I would now install the cron or systemd timer only on the designated scheduler host. If the same artifact must exist everywhere, I would use a root-owned role file or exact configuration value and fail closed when it is absent. A high-availability scheduler needs a real lease or leader-election mechanism; grepping an address is not one.

This distinction matters because these commands mutate shared application state and can send email. “Only one server happened to match” is weaker than “the deployment names one scheduler owner.”

Every-minute triggering is a queue, not a frequency

With a non-blocking lock, an every-minute cron entry means “start within one minute whenever this lane becomes idle.” It does not mean the command runs once per minute. That can be exactly right for a backlog processor.

It can also create a permanent loop. If a successful segment update takes four hours, the next update begins almost immediately. There is no idle window for maintenance, no cooldown after a suspiciously fast failure, and no distinction between catching up and continuously recomputing.

I decide this per lane:

Current Mautic documentation recommends staggering required jobs rather than launching them in the same minute. It documents batch and maximum-contact controls for segment and campaign work. For long-lived message consumers, it recommends a memory, message, or time limit. Those controls turn “keep running” into a bounded operating decision.

The command names are part of the maintenance burden

The repository’s segment command is mautic:segment:update. Current Mautic documentation uses mautic:segments:update. The email path has also changed across Mautic generations, with current queued delivery centered on a bounded Messenger consumer.

I would not silently update those names in an article and pretend the historical deployment used today’s release. I would first record the installed Mautic version, run bin/console list mautic on a safe instance, and compare every configured command and option with the documentation for that exact version.

A wrapper that runs reliably can still automate a command that no longer exists. Version compatibility belongs in the deployment check.

What I would monitor

A timestamped text log is better than an invisible crontab, but it is not enough to prove freshness. I want each lane to emit four values: start time, finish time, exit status, and duration. I also want a separate freshness check that alerts when no successful completion has appeared within the expected window.

The duration is not just a graph. It changes the schedule. A four-hour median with a six-hour tail needs a different interval from a job that occasionally hangs for a day. A run that finishes in two seconds may be more suspicious than one that takes five hours.

For a destructive cleanup command, the original wrapper’s dry run before the real execution was a good instinct. I would keep it, make the deletion result independently visible, and verify the relevant database backup before allowing the real cleanup to run.

My corrected deployment checklist

1. Name one owner for each scheduled lane.
2. Use an atomic lock with a lane-specific identity.
3. Set batch, message, memory, contact, or time boundaries where the command supports them.
4. Capture stdout and stderr in one owned log destination.
5. Return the real application command’s exit status.
6. Record duration and alert on stale successful completion.
7. Test command names and options against the installed application version.
8. Keep email, segment, cleanup, and quick-update locks separate unless shared state requires serialization.
9. Test the duplicate trigger, failed command, terminated process, log rotation, and next successful run.

The old scripts solved a real operational problem: stop guessing a four-hour interval and keep independent work moving. The replacement I would trust keeps that shape, but makes ownership and failure atomic and visible.

The script and host evidence

The source review uses the current public mautic-cron-scripts repository. The scheduling and command guidance is cross-checked against Mautic’s official cron-job documentation. Lock behavior comes from the util-linux flock(1) manual, and exit-status behavior comes from the official GNU Bash manual.

The 30-plus-hour and roughly four-to-six-hour runtimes are historical measurements recorded in the repository, not fresh benchmarks. I inspected the shell source and current documentation. I did not run Mautic, send email, execute cleanup, install the crontab, test a network filesystem lock, or deploy the replacement wrapper for this article. The replacement is an explained pattern, not a drop-in promise for every Mautic version.

Disclosure: I maintain the linked repository.