[CheckMK] add tls checks for ACME certs

[VickieKarasic]

What maintenance needs to be done?

We need to review the VMs on our loadbalancer and add checks in CheckMK for ACME-generate certs. This follows work done in #1211 to add checks for VMs with manually generated certs.

Level of urgency

• High
• Moderate
• Low

Why is this maintenance needed?

• If we can monitor certificate expirations, we will not be "surprised" if any errors in autorenewal (or manual renewal) occur. This adds robustness to our monitoring system.

Acceptance criteria

• Make a list of TLS certs on load-balanced machines
• For ACME generated certs in the list above, create a different check with additional rules criteria (e.g. check that chron jobs run)
• In CheckMK HTTP service check for the above machines, set Age: Warning to at or below to 4 days and Critical at or below to 1 day.

Implementation notes, if any

• Depending on how many certs there are in the list above, we should think about whether there is a more streamlined way to add these checks (e.g. can CheckMK Support help us with this)?
• Used this video as our point of reference to add TLS checks: Monitoring websites and their certificates with Checkmk
• Our version of these steps is:

Part I

We use Active Checks
Service must be assigned to a host
Search HTTP
Select Check HTTP Service
Create a new rule
We don't need rule properties
web page name must match service name
overwrite the name of the host (add virtual host - lib-adc?)
Use SSL with autonegotiation
Attach this service to host and select localhost

Part II

Check for a string on the webpage
Copy string from destination page
Fixed string in content

Part III

Clone a HTTP rule above
Check mode of the role and Check Certificate Age
Added a new name to cloned rule
Modify the cert check from the default 60 seconds
Search interval in the Setup Search.
Create a new Rule for how frequent we check for certs