Run

Monitoring and on-call

Knowing the site is down before your customer calls to tell you.

Most of the outages we inherit were not caught by an alert: they were reported by an unhappy customer, often days later. The opposite exists too, more insidiously: monitoring that sends so many alerts nobody reads them any more. An alert should be rare, accurate, and say what to do. We watch uptime, application errors, server resources, and above all what nobody watches: the overnight jobs that were supposed to run and no longer do.

The method

How we work

  1. 01

    We decide what deserves to wake someone up

    An alert ignored three times is a dead alert, and it silences all the others. We separate what needs immediate action from what can wait until tomorrow.

  2. 02

    We watch from outside and from inside

    From outside: does the site answer, is the certificate valid. From inside: application errors, memory, disk, queues. The two do not catch the same failures.

  3. 03

    We centralise the logs

    All logs in one place, searchable over several weeks. Without that, diagnosing yesterday's outage means hoping the file was not overwritten.

  4. 04

    We verify that what should have run, ran

    Backups, imports, scheduled sends. An overnight job that stops raises no visible error: you find out weeks later, when the data is missing.

What you get

Deliverables

  • check_circleThe probes in place, with thresholds written down and discussed
  • check_circleCentralised, searchable logs
  • check_circleThe alerting channel: who is notified, how, within what delay
  • check_circleA written report after each incident: what happened, what will stop it recurring

Frequently asked questions

Do you respond at night and at weekends?
Depending on the contract. We would rather be straight about it: genuinely round-the-clock on-call requires an organisation a small firm cannot seriously promise. What we can hold, we put in writing; what we cannot, we do not sell.
How long are logs kept?
One month by default, which comfortably covers incident diagnosis. Longer where a regulatory obligation requires it — in which case the duration is decided alongside the processing register, not at random.
Will you monitor an application you did not write?
Yes. Uptime, certificates, resources and scheduled jobs can be watched from outside, whatever the technology. Fine-grained application error monitoring does require access to the code.

Let's talk about your project

Thirty minutes is enough to tell whether we are the right fit. We reply within 48 hours, and we say no when it is not for us.