A Weekly Cron Died Silently for 10 Weeks. Here Is What We Changed.

The Bug Nobody Saw

For roughly 10 weeks, a scheduled publishing job produced zero output. No error email. No Slack ping. No dashboard turning red. The job ran on schedule every week, hit a failure condition early in the pipeline, logged a message to the console, and returned. That was it.

The failure path looked something like this:

async function runPublishingJob() {
  const items = await fetchScheduledContent();

  if (!items || items.length === 0) {
    console.log('No items to publish. Exiting.');
    return; // silent exit, no alert
  }

  await publishAll(items);
}

That console.log and early return felt reasonable when someone wrote it. If there is nothing to publish, just exit cleanly. The problem is that fetchScheduledContent() was supposed to return items every single week. When it started returning nothing, because of an upstream MongoDB query that had quietly broken, the job treated a critical failure as a normal no-op.

Ten weeks of content never went out. Nobody noticed until a stakeholder asked why the publishing cadence had gone quiet.

Why This Happens

Scheduled jobs are easy to forget about once they are running. Unlike an HTTP endpoint that a user hits and immediately reports as broken, a cron job runs in the background. If it does not produce visible output, the only way to know it is working is to go looking. Most teams do not go looking.

The deeper issue is that silence gets interpreted as success. The job ran. No exception was thrown. No process crashed. From the outside, everything looked fine.

This is a monitoring design flaw, not a code quality flaw. The developer who wrote that early return was not being careless. The system just had no concept of "this job ran but did nothing, and that is suspicious."

The Two Things Every Scheduled Job Needs

After this incident, we established a rule that applies to every scheduled job we build or maintain:

1. A positive heartbeat. The job must emit a signal when it completes successfully, not just when it fails. We use a lightweight ping to a health-check service (we use Better Uptime for this, though Cronitor and Healthchecks.io work the same way). If the ping does not arrive within the expected window, the service fires an alert. The job does not get credit for running unless it explicitly checks in.

2. A failure alert routed to a human. A console.log is not an alert. An alert is a message that lands somewhere a person actually reads, within a timeframe that matters. For us, that means a dedicated ops channel in Slack.

Here is the revised pattern:

async function runPublishingJob() {
  try {
    const items = await fetchScheduledContent();

    if (!items || items.length === 0) {
      await sendAlert({
        severity: 'critical',
        message: 'Publishing job ran but fetched zero items. Expected at least 1.',
        job: 'weekly-publisher',
      });
      return;
    }

    const published = await publishAll(items);

    if (published.count === 0) {
      await sendAlert({
        severity: 'critical',
        message: `Publishing job completed but zero items were published. Items fetched: ${items.length}`,
        job: 'weekly-publisher',
      });
      return;
    }

    await heartbeat.ping('weekly-publisher-success');
    console.log(`Published ${published.count} items.`);
  } catch (err) {
    await sendAlert({
      severity: 'critical',
      message: `Publishing job threw an unhandled error: ${err.message}`,
      job: 'weekly-publisher',
    });
  }
}

The key addition is the "zero published" alert. Even if the job runs without throwing, if it publishes nothing, that is loud now, not invisible.

How We Route Alerts

We consolidate all scheduled job alerts into one ops Slack channel. The routing rules are:

• Immediate criticals post to the channel the moment they fire. Zero published, unhandled exceptions, missed heartbeats. These cannot wait.
• Non-critical warnings (slower-than-expected run times, retry counts above threshold) go into a digest that posts three times per day: 8am, 1pm, and 6pm.

This structure matters. If every alert is immediate, engineers start ignoring the channel. If every alert is batched, a critical failure sits unread for hours. The two-tier system keeps the channel signal-to-noise ratio high enough that people actually look at it.

The digest is a simple Lambda function (we run most of our backend infrastructure on AWS) that queries a DynamoDB table of alert events, formats them, and posts to Slack via webhook. Nothing exotic.

The "Zero Published" Alert Is the Specific Fix

Most monitoring guides tell you to alert on errors. That is necessary but not sufficient. You also need to alert on the absence of expected output.

For a weekly publishing job, the expected output is at least one published item per run. If that does not happen, something is wrong, whether or not an exception was thrown. The fetchScheduledContent function returning an empty array was not an error in the JavaScript sense. It was a logic failure that only became visible when you asked "did this job do what it was supposed to do?"

This is sometimes called an outcome-based alert, as opposed to an error-based alert. Both are necessary. Error-based alerts catch crashes. Outcome-based alerts catch silent failures.

For any scheduled job, ask: what is the minimum acceptable output of a successful run? Then alert if the job produces less than that, even if it exits cleanly.

What We Audit Now

After this incident, we audited every scheduled job across our systems. The checklist we used:

• Does this job have a heartbeat ping configured?
• Is the heartbeat monitored with an expected-interval check?
• Does the job alert on zero or below-threshold output, not just on exceptions?
• Does the alert route to the ops channel, not just to logs?
• Has someone tested the failure path in the last 90 days?

Several jobs failed this checklist. None of them were broken, but they would have been invisible if they had broken.

At Savage Digital Solutions (savagesolutions.io), this audit is now part of our standard deployment review for any background job, whether it is a Next.js API route triggered by Vercel Cron, a standalone Node.js process, or a Python script running on a schedule in AWS.

Silence Is Not Success

The phrase we use internally now is: silence is not success. A job that runs and produces no output should be treated as suspicious until proven otherwise. The burden of proof is on the job to demonstrate it worked, not on the engineer to go check.

This is a small cultural shift, but it changes how you write monitoring code. Instead of "alert when something goes wrong," the mindset becomes "confirm that something went right, and alert if you cannot confirm it."

Ten weeks is a long time for a publishing pipeline to be down. The fix took about two hours to implement. The audit took a day. The policy change took one team conversation.

Key Takeaways

• A console.log followed by a return is not an alert. It is invisible in production.
• Every scheduled job needs a positive heartbeat, a ping that fires on success, monitored for expected arrival.
• Alert on zero or below-threshold output, not only on thrown exceptions. Silent no-ops are often the most dangerous failures.
• Route all job alerts to one ops channel. Use immediate delivery for criticals and a digest (we use 3x/day) for warnings.
• Audit existing scheduled jobs against a checklist: heartbeat configured, failure path tested, output threshold alert in place.
• Silence is not success. A job that does nothing should be loud, not invisible.