Monitoring Concepts
Monitoring is collecting measurements from a running system and watching them, so you learn about problems and trends before or as users do. It answers three questions: is it up, is it fast enough, and is it behaving normally?
What gets monitored
- Availability: can users reach the service, and does it respond correctly? This is often checked by a health check endpoint or a synthetic request made on a schedule.
- Performance: how long requests take. Averages hide problems, so teams look at percentiles, such as the time within which 95 percent of requests complete.
- Errors: the rate of failed requests or exceptions.
- Saturation and resources: CPU, memory, disk, database connections and queue lengths show how close the system is to its limits.
- Business signals such as orders per minute or sign-ups. A drop can reveal a problem that technical metrics miss.
Logs, metrics and traces
Observability is usually described through three kinds of data:
| Kind | What it is | Good for |
|---|---|---|
| Logs | Event records with details | Finding out what happened; see Logging Fundamentals |
| Metrics | Numbers over time | Dashboards, trends, alerting |
| Traces | The path of one request across services | Finding where time is spent or where it failed |
Alerts and targets
An alert notifies a person (or system) when a measurement crosses a threshold. Good alerts are actionable and tied to user impact. Noisy alerts get ignored, so every alert should deserve a response.
Teams also agree service levels: an SLO (objective) is the target, such as “99.9 percent of requests succeed in a month”, and an SLA (agreement) is a promise made to customers, sometimes with penalties. Both are in the glossary.
As a developer you will often be asked to add useful logs, metrics or health checks to your own code, and to look at dashboards after a release. When monitoring shows a problem, the response follows Incident Basics, and the investigation uses Troubleshooting Methodology.
Try it
Find the health check or status page of a service you use or work on (many applications expose a path such as /health). Request it in a browser or with curl, and note what it reports. Then pick three metrics you would want on a dashboard for that service, and decide when you would raise an alert on one of them.
Common mistakes
- Looking only at averages, which hide slow requests.
- Creating noisy alerts that people learn to ignore.
- Watching technical metrics but missing business signals like a drop in orders.
Check yourself
- What are logs, metrics and traces each good for?
- Why do teams use percentiles instead of averages?
- What is the difference between an SLO and an SLA?