Incident management, in plain words.
The language of incident management, defined plainly: what each term means, and where it touches what Pindot does.
A
- Air-gapped
- A system with no network connection to the internet or to any network outside its own. Software for an air-gapped site has to be installed, updated and run entirely from inside it.
- In Pindot: Pindot can run fully air-gapped, with its AI on your own hardware and no outbound connection.
- Alert fatigue
- What happens when people receive so many alerts, many of them unimportant or duplicated, that they start to ignore or mute them, and miss the ones that matter.
- Anomaly detection
- Finding behaviour that departs from what is normal for a system, such as an error count or traffic volume well above its usual level for that time, rather than relying only on fixed thresholds.
- In Pindot: Pindot learns each source's usual level and flags the one that moves away from it.
- Audit trail
- A chronological record of who did what, when, and with what result, kept so that actions can be reviewed later. A trustworthy audit trail is only ever added to, never edited.
- In Pindot: Every question asked in Pindot leaves an audit record of where it looked, which records it used and what it answered.
B
- Blast radius
- How much is affected when something fails: which services, users, regions or transactions. Reducing blast radius is about containing failures so one fault does not spread.
- Bridge call
- A live call, often by phone or video, where the people needed to resolve a serious incident work on it together. Also called a war room or incident call.
C
- Change advisory boardCAB
- A group that reviews and approves proposed changes to production systems, weighing their risk and timing. Common in ITIL-based change management.
- Change record
- The entry in a service desk or change management system that describes a change to production: what changed, when, who approved it and how risky it was rated.
- In Pindot: Pindot can read change records from your service desk and line them up against what failed.
- Contributing factor
- Something that did not cause an incident on its own but made it more likely, more harmful or longer lasting, such as a late alert or a misleading runbook.
- Correlation
- Linking events from different systems that happened together or in sequence, to see which may be related. Correlation suggests where to look; it does not by itself prove cause.
E
- Error budget
- The amount of unreliability a service is allowed within a period, derived from its SLO. If the objective is 99.9% availability, the remaining 0.1% is the error budget.
I
- Incident
- An unplanned interruption to a service, or a reduction in its quality, that needs a response. Incidents are usually given a severity that sets how urgently they are handled.
- Incident lead
- The person who coordinates the response to an incident: deciding what to try next, assigning work and declaring when it is resolved. Also called the incident commander or incident manager.
M
- Mean time to acknowledgeMTTA
- The average time between an alert firing and a person acknowledging it. A measure of how quickly on-call responders start work.
- Mean time to detectMTTD
- The average time between a problem starting and it being noticed, by monitoring or by a person.
- Mean time to resolveMTTR
- The average time from an incident starting, or being detected, to it being resolved. MTTR is also used for mean time to repair, recover or respond, so always say which one you mean.
O
- Observability
- How well you can understand what a system is doing from the data it produces, such as logs, metrics and traces, including for problems nobody anticipated.
- On-call
- The arrangement in which named people are available outside normal hours to respond to alerts and incidents, usually on a rotating schedule.
P
- Postmortem
- A written review after an incident covering what happened, why, how it was handled and what will change. Also called an incident review or post-incident review.
R
- Read-only access
- Permission to view data without being able to change it. For production systems it should be enforced by the system itself, so any attempted write is refused.
- In Pindot: Pindot connects to every source with read-only access and never writes to them.
- Root cause analysisRCA
- The process of working out why an incident happened, beyond its immediate trigger, so the underlying problem can be fixed. Most incidents have several contributing causes rather than one.
- In Pindot: Pindot names the probable cause of a problem, cites the exact records behind it and says how sure it is.
- Runbook
- A documented set of steps for handling a known situation, such as restarting a service or failing over a database, so the response does not depend on one person's memory.
S
- Service level agreementSLA
- A commitment to a customer about the level of service they will receive, often with consequences if it is missed. Usually less strict than the internal SLO behind it.
- Service level indicatorSLI
- A measurement of one aspect of a service's performance, such as the share of requests that succeed or that complete within a time limit.
- Service level objectiveSLO
- A target for an SLI over a period, such as 99.9% of requests succeeding over 30 days. The SLO sets the error budget.
- Severity
- A label for how serious an incident or event is, which sets how urgently it is handled and who is involved. Scales differ between organisations, so define yours in writing.
T
- Telemetry
- The data systems emit about their own behaviour, such as logs, metrics, events and traces, collected for monitoring and diagnosis.
- Timeline
- An ordered record of what happened during an incident, with times and sources. Written during the incident, it keeps the response focused; afterwards, it is the backbone of the postmortem.
Stop digging through logs at 2am.
Bring your own 02:14, a real incident from your systems. We will show you how Pindot finds the cause, and where every piece of evidence came from.