Building Windows AWS AMIs using Packer and Ansible

Like many other companies that are deploying their applications to the cloud, the majority of our estate uses Linux. However we do need to use Windows for a couple of purposes. This could be for application testing, or for specific Windows features. We also recently adopted Packer to build our machine images, to allow them to be defined in code (and therefore within version control). In Amazon, these machine images are called AMIs. [Read More]
devops  aws  packer  windows  aws  ami 

Prometheus: snmp_exporter and OpenBSD

In a previous post, I showed how to run the Prometheus node_exporter on a number of different operating systems, including OpenBSD. Many OpenBSD installs are used as, or to replace, network appliances (e.g. peering routers, firewalls, VPN concentrators). Traditionally, you would monitor networking equipment using SNMP. OpenBSDs snmpd(8) can expose a number of metrics that cover carp(4), pf(4), relayd(8) and more. Prometheus and SNMP The snmp_exporter is used so that Prometheus can monitor devices via SNMP. [Read More]

Prometheus: Consul Service Discovery for blackbox and snmp exporter

In a previous post I covered how to use Consul for service discovery of standard exporters, allowing Prometheus to automatically discover what services to monitor. However, this configuration didn’t cater to exporters like the snmp_exporter or blackbox_exporter. What is interesting about both of the above is that rather than generating metrics for a local application, they are a proxy for other services. For example, you can use BlackBox exporter to do ICMP checks or HTTPS checks, without running an exporter on the services themselves. [Read More]

Prometheus: Discover services with DNS

In a previous post I covered how to use Consul for service discovery, allowing Prometheus to automatically discover what services to monitor. There are some cases where either setting up Consul (or similar) is not viable, or adds complexity that is not required. If you are already running your own DNS nameservers, you could make use of DNS SRV records. Common DNS record types The most common DNS records are A, AAAA and PTR. [Read More]

Configuration Seasoning: Getting started with Saltstack

Configuration management is the practice of deploying and managing your application and infrastructure configuration through automated tooling, rather than managing all of your infrastructure manually. This can cover everything from Linux servers, to network equipment, installing packages to updating existing services. The primary benefits are that you can manage more infrastructure without the operational burden increasing significantly, and that your configuration is consistent across your estate. There are already a number of tools which achieve this: - [Read More]

Prometheus: Monitoring node statistics on other operating systems

In my previous posts on Prometheus, most of the monitoring has been geared to either getting metrics from Linux hosts. There have been a couple of exceptions, like the blackbox_exporter (for ICMP/Ping, HTTP(S) and TCP socket monitoring) or the snmp_exporter (more commonly used for networking gear). Linux is not the only operating system you can monitor node metrics for though. Other operating systems have either their own port of the node_exporter, or via other exporters entirely. [Read More]

Prometheus: Discovering Services with Consul

In my previous post, I detailed moving my home monitoring over to Prometheus. I’ve gained huge insights into my home network (and a few external services I rely on), and have been very happy with it. Adding new endpoints has been pretty straightforward. I have been using Ansible to generate the prometheus.yml configuration file, using variables to generate each section of the scrape configuration. This has worked equally well for both services exposing native Prometheus endpoints (eg Cadvisor or Traefik) and for the numerous exporters I am running. [Read More]

Prometheus: Embracing the Exporter Life

A couple of months ago I decided to start monitoring my home network. This was in part due to frequent dropouts of an Amazon Firestick. This usually happened in the middle of the kids watching Peppa Pig or Paw Patrol. Nothing like family-driven monitoring! I originally setup Nagios and Zabbix. My home “network” mostly consists of a few Raspberry Pis, a couple of other random ARM boards, and a mish mash of cheap switches (see: unmanaged low end TP Link/Netgear, not old enteprise kit) and routers. [Read More]

Triggering ArgoCD to deploy to Kubernetes with a Jenkins Pipeline

In the world of DevOps, automation is one of the primary goals. This includes automating how you deploy your software. Rather than relying on someone to rsync/FTP/write their software on the machine it is being deployed upon, there is the concept of CI/CD. CI, or Continuous Integration, is the step of creating an artifact from code commits. This could be a Docker image, deployed using commits in the Master branch of a Git repository. [Read More]

AWS Cognito as an Oauth2 Provider for Kubernetes Apps - Part 1

Recently I have been integrating a number of apps in Kubernetes to use AWS Cognito as an Oauth2 provider. For those unaware, Oauth2 is a protocol that can be used to authenticate users against a number of different services. Whenever you see “Login with Google” or “Login with Facebook”, this is using Oauth2 behind the scenes. It’s worth pointing out that Oauth2 is a Framework for how to implement authorization. Open ID Connect (OIDC) extends Oauth2, but also simplifies it. [Read More]