Added new OpenSSH break glass admin management operations guide

docs/pages/admin-guides/management/operations/breakglass-access.mdx

@@ -0,0 +1,199 @@
+---
+title: Configuring Break Glass Access for Disaster Recovery
+description: Guide to set up break glass emergency access for critical systems if Teleport becomes unavailable.
+---
+
+This guide will walk you through the steps required to configure emergency "break glass" access for critical agents and servers via OpenSSH using Teleport-issued certificates, in the following scenarios:
+
+1. The Teleport Agent on a server has crashed, gone offline, or become unusable.
+2. The Teleport control plane is down and cannot be used to access systems, and this procedure is necessary to fix it.
+
+## How it works
+
+Teleport's CA can issue short-lived, signed certificates that can be used to grant access to OpenSSH servers in emergency disaster recovery scenarios. By configuring the OpenSSH server on the Teleport Agent to trust Teleport’s CA, users with valid Teleport-issued certificates can authenticate to the server without requiring static SSH keys or passwords even if Teleport itself is down.
+
+Below, we’ll detail the steps to accomplish the following objectives:
+
+1. Configure the Teleport Agent's OpenSSH server to trust Teleport's CA
+2. Create a `breakglass` system user on the remote host
+3. Restrict cert-based SSH access to this `breakglass` user only
+4. Create a `breakglass` Role & User in Teleport
+5. Generate `breakglass` SSH Key and Cert using Teleport's CA
+6. Access the remote server using a Teleport issued cert even if Teleport is down or unresponsive
+
+## Prerequisites
+
+- OpenSSH client version 7.4 or above on your local machine.
+- A Linux host with OpenSSH server (`sshd`) version 7.4 or above.
+
+(!docs/pages/includes/commercial-prereqs-tabs.mdx!)
+- A Teleport user with ability to add new `impersonate` role permissions.
+
+## Step 1/6. Configure sshd to trust the Teleport CA
+
+For break glass access, the OpenSSH server must be configured to trust client certificates issued by the Teleport Certificate Authority (CA).
+
+1. **Export the Teleport CA public key:**
+
+On the OpenSSH server, run the following command. Replace `proxy.example.com` with the address of your Teleport Proxy Service:
+
+```bash
+export KEY=$(curl 'https://proxy.example.com/webapi/auth/export?type=openssh' | sed "s/cert-authority\ //")
+```
+
+2. Make the public key accessible to sshd:
+
+```bash
+sudo bash -c "echo \"$KEY\" > /etc/ssh/teleport_openssh_ca.pub"
+sudo bash -c "echo 'TrustedUserCAKeys /etc/ssh/teleport_openssh_ca.pub' >> /etc/ssh/sshd_config"
+```
+
+3. Restart the sshd service:
+
+```bash
+sudo systemctl restart sshd
+```
+
+Now, `sshd` will trust users who present a Teleport-issued SSH certificate.
+
+<Admonition title="Critical Security Warning" type="warning">
+ Adding the Teleport CA to the server's `TrustedUserCAKeys` will allow any user with a valid Teleport cert to authenticate to this node via OpenSSH unless further restrictions are implemented.  
+
+ You **MUST** complete Steps 4 and 5 below to ensure only the `brekaglass` user is allowed to log in with a valid Teleport certificate and is configured with proper access restrictions to prevent user switching once logged in. 
+</Admonition>
+
+## Step 2/6. Create a `breakglass` user on the Teleport Agent server
+
+Log into the Teleport Agent server and run the following commands.
+
+1. Create the `breakglass` user:
+
+```bash
+sudo adduser breakglass
+```
+
+Set a password for `breakglass` when prompted.
+
+**Note:** As a general security principle, it is highly recommended to limit `root` permissions through `sudo`. Because `breakglass` will need ability to gain elevated access to the system to troubleshoot and recover access, however, any use of break glass access should always be audited via sshd access logs. 
+
+## Step 3/6. Restrict cert-based SSH access to the `breakglass` user
+
+The `/etc/ssh/authorized_principals` directory is used in conjunction with OpenSSH's certificate-based authentication mechanism to configure fine-grained control over which users or roles are allowed to authenticate using SSH certificates.
+We will use this mechanism to restrict cert-based SSH access only to the `breakglass` user to prevent other Teleport users from being able to generate client certificates and gain access to the server outside of Teleport.
+
+1. Ensure the following line is added and/or uncommented in your `/etc/ssh/sshd_config` file:
+
+```
+AuthorizedPrincipalsFile /etc/ssh/authorized_principals/%u
+```
+
+2. Create an authorized_principals file for the `breakglass` user 
+
+```bash
+echo "breakglass" | sudo tee /etc/ssh/authorized_principals/breakglass
+```
+
+3. Restart the sshd service:
+
+```bash
+sudo systemctl restart sshd
+```
+
+## Step 4/6. Create `breakglass` Role & User in Teleport for impersonation
+
+1. Create `breakglass` Teleport role:
+
+Define the role in a file named breakglass-role.yaml
+
+```yaml
+kind: role
+metadata:
+  name: breakglass
+spec:
+  allow:
+    logins:
+    - breakglass
+    node_labels:
+      '*': '*'
+  deny: {}
+  options:
+    cert_format: standard
+    forward_agent: false
+    max_session_ttl: 24h0m0s
+    port_forwarding: true
+    ssh_file_copy: true
+version: v7
+```
+
+Create the role by applying with:
+
+```bash
+tctl create -f breakglass-role.yaml
+```
+
+2. Create the `breakglass` Teleport user:
+
+```bash
+tctl users add --roles=breakglass breakglass
+```
+
+We don't need to worry about setting a password for this user as it'll only ever be used in the context of OpenSSH break glass authentication via impersonation.
+
+3. Grant `impersonate` permissions for `breakglass` User & Role:
+
+To avoid having to directly log into Teleport as `breakglass`, grant `impersonation` permissions to an admin or automation user by adding the following to the impersonating user's matching role:
+
+```yaml
+impersonate:
+    roles:
+    - breakglass
+    users:
+    - breakglass
+```
+
+## Step 5/6. Generate a `breakglass` ssh key and cert
+
+Run the following after logging into Teleport via `tsh login` as the impersonating user:
+
+```bash
+tctl auth sign --ttl 24h --user=breakglass --out=/safe/path/breakglass --format openssh
+```
+
+This command will create an ssh private key and a Teleport CA-signed certificate to be used by your OpenSSh client when authenticating with the OpenSSH server.
+
+Store the private key securely in a vault or other secure location, and only access if needed to implement break glass procedures. 
+
+<Admonition type="tip">
+If you've followed this guide, the resulting `breakglass` Teleport CA-signed cert will be valid only for 24 hours. To ensure that you have a valid and up-to-date certificate in case of an outage, regularly rotate both the key and certificate before they expire. Use [Machine ID](../../../../pages/enroll-resources/machine-id/machine-id.mdx) or automate renewal with a cron job or systemd timer.
+
+**Example:** Create a cron job to rotate certs every 20 hours and push them to a secure vault: 
+
+```bash
+0 */20 * * * tctl auth sign --ttl 24h --user=breakglass --format openssh | vault kv put secret/breakglass breakglass=- >> /var/log/vault-cron.log 2>&1
+```
+</Admonition>
+
+
+## Step 6/6. Access the Teleport Agent using OpenSSH
+
+1. Add the following to your `~/.ssh/config` (or equivalent), making sure to update with your `HostName` and cert paths:
+
+```bash
+Host sshd-server
+    HostName sshd-server.example.com
+    User breakglass
+    IdentityFile /secure/path/breakglass
+    CertificateFile /secure/path/breakglass-cert.pub
+```
+
+2. Access the server
+
+Now, securely access the Teleport Agent using the `breakglass` user and Teleport-issued certificate:
+
+```bash
+ssh sshd-server
+```
+
+---
+
+The method described above ensures you can securely access your servers and Teleport Agents in emergency scenarios using Teleport CA signed certificates, even if Teleport's agents or control plane are down. This process can be used for any Teleport Agent running on server which supports OpenSSH regardless of Teleport protocol. 

docs/pages/admin-guides/management/operations/operations.mdx

@@ -16,3 +16,4 @@ the [Cluster Administration Guides](../admin/admin.mdx) section.
 - [TLS Routing Migration](tls-routing.mdx): Migrating your Teleport cluster to single-port TLS routing mode.
 - [Proxy Peering Migration](proxy-peering.mdx): Migrating your Teleport cluster to Proxy Peering mode.
 - [Database CA Migrations](db-ca-migrations.mdx): Completing Teleport's Database CA migrations.
+- [Disaster Recovery Break-Glass Access](breakglass-access.mdx): Set up break-glass access for Teleport.