Prometheus: snmp_exporter and OpenBSD

In a previous post, I showed how to run the Prometheus node_exporter on a number of different operating systems, including OpenBSD. Many OpenBSD installs are used as, or to replace, network appliances (e.g. peering routers, firewalls, VPN concentrators). Traditionally, you would monitor networking equipment using SNMP. OpenBSDs snmpd(8) can expose a number of metrics that cover carp(4), pf(4), relayd(8) and more. Prometheus and SNMP The snmp_exporter is used so that Prometheus can monitor devices via SNMP. [Read More]

Prometheus: Consul Service Discovery for blackbox and snmp exporter

In a previous post I covered how to use Consul for service discovery of standard exporters, allowing Prometheus to automatically discover what services to monitor. However, this configuration didn’t cater to exporters like the snmp_exporter or blackbox_exporter. What is interesting about both of the above is that rather than generating metrics for a local application, they are a proxy for other services. For example, you can use BlackBox exporter to do ICMP checks or HTTPS checks, without running an exporter on the services themselves. [Read More]

Prometheus: Discover services with DNS

In a previous post I covered how to use Consul for service discovery, allowing Prometheus to automatically discover what services to monitor. There are some cases where either setting up Consul (or similar) is not viable, or adds complexity that is not required. If you are already running your own DNS nameservers, you could make use of DNS SRV records. Common DNS record types The most common DNS records are A, AAAA and PTR. [Read More]

Prometheus: Monitoring node statistics on other operating systems

In my previous posts on Prometheus, most of the monitoring has been geared to either getting metrics from Linux hosts. There have been a couple of exceptions, like the blackbox_exporter (for ICMP/Ping, HTTP(S) and TCP socket monitoring) or the snmp_exporter (more commonly used for networking gear). Linux is not the only operating system you can monitor node metrics for though. Other operating systems have either their own port of the node_exporter, or via other exporters entirely. [Read More]

Prometheus: Discovering Services with Consul

In my previous post, I detailed moving my home monitoring over to Prometheus. I’ve gained huge insights into my home network (and a few external services I rely on), and have been very happy with it. Adding new endpoints has been pretty straightforward. I have been using Ansible to generate the prometheus.yml configuration file, using variables to generate each section of the scrape configuration. This has worked equally well for both services exposing native Prometheus endpoints (eg Cadvisor or Traefik) and for the numerous exporters I am running. [Read More]

Prometheus: Embracing the Exporter Life

A couple of months ago I decided to start monitoring my home network. This was in part due to frequent dropouts of an Amazon Firestick. This usually happened in the middle of the kids watching Peppa Pig or Paw Patrol. Nothing like family-driven monitoring! I originally setup Nagios and Zabbix. My home “network” mostly consists of a few Raspberry Pis, a couple of other random ARM boards, and a mish mash of cheap switches (see: unmanaged low end TP Link/Netgear, not old enteprise kit) and routers. [Read More]