Skip to content

Commit

Permalink
initial work
Browse files Browse the repository at this point in the history
explain how to use ebs volumes with the standalone k8s snap
  • Loading branch information
eaudetcobello committed Nov 13, 2024
1 parent 13689bd commit 30e6be5
Showing 1 changed file with 379 additions and 0 deletions.
379 changes: 379 additions & 0 deletions docs/src/snap/howto/storage/cloud.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,379 @@
# How to use cloud storage

{{product}} offers a local-storage option to quickly set up and run a
cluster, especially for single-node support. This guide walks you through
enabling and configuring this feature.

## What you'll need

This guide assumes the following:

- You have root or sudo access to an Amazon EC2 instance
- You can create roles and policies in AWS

## Set your host name

The cloud controller manager uses the node name to correctly associate the node with an EC2 instance. In Canonical K8s, the node name is derived from the hostname of the machine. Therefore, before bootstrapping the cluster, we must first set an appropriate host name.

Check failure on line 16 in docs/src/snap/howto/storage/cloud.md

View workflow job for this annotation

GitHub Actions / markdown-lint

Line length

docs/src/snap/howto/storage/cloud.md:16:81 MD013/line-length Line length [Expected: 80; Actual: 267] https://github.com/DavidAnson/markdownlint/blob/v0.34.0/doc/md013.md

Check failure on line 16 in docs/src/snap/howto/storage/cloud.md

View workflow job for this annotation

GitHub Actions / markdown-lint

Line length

docs/src/snap/howto/storage/cloud.md:16:81 MD013/line-length Line length [Expected: 80; Actual: 267] https://github.com/DavidAnson/markdownlint/blob/v0.34.0/doc/md013.md

```
echo "$(sudo cloud-init query ds.meta_data.local-hostname)" | sudo tee /etc/hostname
```

Then, reboot the machine.

When the machine is up, use `hostname -f` to check the host name. It should look like:

Check failure on line 24 in docs/src/snap/howto/storage/cloud.md

View workflow job for this annotation

GitHub Actions / markdown-lint

Line length

docs/src/snap/howto/storage/cloud.md:24:81 MD013/line-length Line length [Expected: 80; Actual: 86] https://github.com/DavidAnson/markdownlint/blob/v0.34.0/doc/md013.md

Check failure on line 24 in docs/src/snap/howto/storage/cloud.md

View workflow job for this annotation

GitHub Actions / markdown-lint

Line length

docs/src/snap/howto/storage/cloud.md:24:81 MD013/line-length Line length [Expected: 80; Actual: 86] https://github.com/DavidAnson/markdownlint/blob/v0.34.0/doc/md013.md

```
ip-172-31-11-86.us-east-2.compute.internal
```

This format is called IP-based naming and is specific to AWS.

```
{note} Don't rely on the PS1 prompt to know if your host name was changed successfully. The PS1 prompt only displays the hostname up to the first `.`.
```

## Set IAM Policies

Your instance will need a few IAM policies to be able to communciate with the AWS APIs. These policies are quite open and can be scaled back if desired.

Check failure on line 38 in docs/src/snap/howto/storage/cloud.md

View workflow job for this annotation

GitHub Actions / markdown-lint

Line length

docs/src/snap/howto/storage/cloud.md:38:81 MD013/line-length Line length [Expected: 80; Actual: 152] https://github.com/DavidAnson/markdownlint/blob/v0.34.0/doc/md013.md

Check failure on line 38 in docs/src/snap/howto/storage/cloud.md

View workflow job for this annotation

GitHub Actions / markdown-lint

Line length

docs/src/snap/howto/storage/cloud.md:38:81 MD013/line-length Line length [Expected: 80; Actual: 152] https://github.com/DavidAnson/markdownlint/blob/v0.34.0/doc/md013.md

For a control plane node:
```json

Check failure on line 41 in docs/src/snap/howto/storage/cloud.md

View workflow job for this annotation

GitHub Actions / markdown-lint

Fenced code blocks should be surrounded by blank lines

docs/src/snap/howto/storage/cloud.md:41 MD031/blanks-around-fences Fenced code blocks should be surrounded by blank lines [Context: "```json"] https://github.com/DavidAnson/markdownlint/blob/v0.34.0/doc/md031.md

Check failure on line 41 in docs/src/snap/howto/storage/cloud.md

View workflow job for this annotation

GitHub Actions / markdown-lint

Fenced code blocks should be surrounded by blank lines

docs/src/snap/howto/storage/cloud.md:41 MD031/blanks-around-fences Fenced code blocks should be surrounded by blank lines [Context: "```json"] https://github.com/DavidAnson/markdownlint/blob/v0.34.0/doc/md031.md
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"autoscaling:DescribeAutoScalingGroups",
"autoscaling:DescribeLaunchConfigurations",
"autoscaling:DescribeTags",
"ec2:DescribeInstances",
"ec2:DescribeRegions",
"ec2:DescribeRouteTables",
"ec2:DescribeSecurityGroups",
"ec2:DescribeSubnets",
"ec2:DescribeVolumes",
"ec2:DescribeAvailabilityZones",
"ec2:CreateSecurityGroup",
"ec2:CreateTags",
"ec2:CreateVolume",
"ec2:ModifyInstanceAttribute",
"ec2:ModifyVolume",
"ec2:AttachVolume",
"ec2:AuthorizeSecurityGroupIngress",
"ec2:CreateRoute",
"ec2:DeleteRoute",
"ec2:DeleteSecurityGroup",
"ec2:DeleteVolume",
"ec2:DetachVolume",
"ec2:RevokeSecurityGroupIngress",
"ec2:DescribeVpcs",
"ec2:DescribeInstanceTopology",
"elasticloadbalancing:AddTags",
"elasticloadbalancing:AttachLoadBalancerToSubnets",
"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer",
"elasticloadbalancing:CreateLoadBalancer",
"elasticloadbalancing:CreateLoadBalancerPolicy",
"elasticloadbalancing:CreateLoadBalancerListeners",
"elasticloadbalancing:ConfigureHealthCheck",
"elasticloadbalancing:DeleteLoadBalancer",
"elasticloadbalancing:DeleteLoadBalancerListeners",
"elasticloadbalancing:DescribeLoadBalancers",
"elasticloadbalancing:DescribeLoadBalancerAttributes",
"elasticloadbalancing:DetachLoadBalancerFromSubnets",
"elasticloadbalancing:DeregisterInstancesFromLoadBalancer",
"elasticloadbalancing:ModifyLoadBalancerAttributes",
"elasticloadbalancing:RegisterInstancesWithLoadBalancer",
"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer",
"elasticloadbalancing:AddTags",
"elasticloadbalancing:CreateListener",
"elasticloadbalancing:CreateTargetGroup",
"elasticloadbalancing:DeleteListener",
"elasticloadbalancing:DeleteTargetGroup",
"elasticloadbalancing:DescribeListeners",
"elasticloadbalancing:DescribeLoadBalancerPolicies",
"elasticloadbalancing:DescribeTargetGroups",
"elasticloadbalancing:DescribeTargetHealth",
"elasticloadbalancing:ModifyListener",
"elasticloadbalancing:ModifyTargetGroup",
"elasticloadbalancing:RegisterTargets",
"elasticloadbalancing:DeregisterTargets",
"elasticloadbalancing:SetLoadBalancerPoliciesOfListener",
"iam:CreateServiceLinkedRole",
"kms:DescribeKey"
],
"Resource": [
"*"
]
}
]
}
```

For a worker node:
```json

Check failure on line 115 in docs/src/snap/howto/storage/cloud.md

View workflow job for this annotation

GitHub Actions / markdown-lint

Fenced code blocks should be surrounded by blank lines

docs/src/snap/howto/storage/cloud.md:115 MD031/blanks-around-fences Fenced code blocks should be surrounded by blank lines [Context: "```json"] https://github.com/DavidAnson/markdownlint/blob/v0.34.0/doc/md031.md

Check failure on line 115 in docs/src/snap/howto/storage/cloud.md

View workflow job for this annotation

GitHub Actions / markdown-lint

Fenced code blocks should be surrounded by blank lines

docs/src/snap/howto/storage/cloud.md:115 MD031/blanks-around-fences Fenced code blocks should be surrounded by blank lines [Context: "```json"] https://github.com/DavidAnson/markdownlint/blob/v0.34.0/doc/md031.md
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ec2:DescribeInstances",
"ec2:DescribeRegions",
"ecr:GetAuthorizationToken",
"ecr:BatchCheckLayerAvailability",
"ecr:GetDownloadUrlForLayer",
"ecr:GetRepositoryPolicy",
"ecr:DescribeRepositories",
"ecr:ListImages",
"ecr:BatchGetImage"
],
"Resource": "*"
}
]
}
```

## Bootstrap Canonical K8s

Now that your machine has an appropriate host name, you are ready to bootstrap Canonical K8s.

Check failure on line 140 in docs/src/snap/howto/storage/cloud.md

View workflow job for this annotation

GitHub Actions / markdown-lint

Line length

docs/src/snap/howto/storage/cloud.md:140:81 MD013/line-length Line length [Expected: 80; Actual: 93] https://github.com/DavidAnson/markdownlint/blob/v0.34.0/doc/md013.md

Check failure on line 140 in docs/src/snap/howto/storage/cloud.md

View workflow job for this annotation

GitHub Actions / markdown-lint

Line length

docs/src/snap/howto/storage/cloud.md:140:81 MD013/line-length Line length [Expected: 80; Actual: 93] https://github.com/DavidAnson/markdownlint/blob/v0.34.0/doc/md013.md

First, create a bootstrap configuration file that sets the cloud-provider configuration to "external".

Check failure on line 142 in docs/src/snap/howto/storage/cloud.md

View workflow job for this annotation

GitHub Actions / markdown-lint

Line length

docs/src/snap/howto/storage/cloud.md:142:81 MD013/line-length Line length [Expected: 80; Actual: 102] https://github.com/DavidAnson/markdownlint/blob/v0.34.0/doc/md013.md

Check failure on line 142 in docs/src/snap/howto/storage/cloud.md

View workflow job for this annotation

GitHub Actions / markdown-lint

Line length

docs/src/snap/howto/storage/cloud.md:142:81 MD013/line-length Line length [Expected: 80; Actual: 102] https://github.com/DavidAnson/markdownlint/blob/v0.34.0/doc/md013.md

```
echo "cluster-config:
cloud-provider: external" > bootstrap-config.yaml
```

Then, bootstrap the cluster:

```
sudo k8s bootstrap --file ./bootstrap-config.yaml
sudo k8s status --wait-ready
```

## Deploy the cloud controller manager

Now that you have an appropriate host name, policies, and a Canonical K8s cluster, you have everything you need to deploy the cloud controller manager.

Check failure on line 158 in docs/src/snap/howto/storage/cloud.md

View workflow job for this annotation

GitHub Actions / markdown-lint

Line length

docs/src/snap/howto/storage/cloud.md:158:81 MD013/line-length Line length [Expected: 80; Actual: 151] https://github.com/DavidAnson/markdownlint/blob/v0.34.0/doc/md013.md

Check failure on line 158 in docs/src/snap/howto/storage/cloud.md

View workflow job for this annotation

GitHub Actions / markdown-lint

Line length

docs/src/snap/howto/storage/cloud.md:158:81 MD013/line-length Line length [Expected: 80; Actual: 151] https://github.com/DavidAnson/markdownlint/blob/v0.34.0/doc/md013.md

Here is a YAML definition file that sets appropriate defaults for you, it configures the necessary service accounts, roles, and daemonsets:

Check failure on line 160 in docs/src/snap/howto/storage/cloud.md

View workflow job for this annotation

GitHub Actions / markdown-lint

Line length

docs/src/snap/howto/storage/cloud.md:160:81 MD013/line-length Line length [Expected: 80; Actual: 139] https://github.com/DavidAnson/markdownlint/blob/v0.34.0/doc/md013.md

Check failure on line 160 in docs/src/snap/howto/storage/cloud.md

View workflow job for this annotation

GitHub Actions / markdown-lint

Line length

docs/src/snap/howto/storage/cloud.md:160:81 MD013/line-length Line length [Expected: 80; Actual: 139] https://github.com/DavidAnson/markdownlint/blob/v0.34.0/doc/md013.md

```
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: aws-cloud-controller-manager
namespace: kube-system
labels:
k8s-app: aws-cloud-controller-manager
spec:
selector:
matchLabels:
k8s-app: aws-cloud-controller-manager
updateStrategy:
type: RollingUpdate
template:
metadata:
labels:
k8s-app: aws-cloud-controller-manager
spec:
nodeSelector:
node-role.kubernetes.io/control-plane: ""
tolerations:
- key: node.cloudprovider.kubernetes.io/uninitialized
value: "true"
effect: NoSchedule
- effect: NoSchedule
key: node-role.kubernetes.io/control-plane
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: node-role.kubernetes.io/control-plane
operator: Exists
serviceAccountName: cloud-controller-manager
containers:
- name: aws-cloud-controller-manager
image: registry.k8s.io/provider-aws/cloud-controller-manager:v1.28.3
args:
- --v=2
- --cloud-provider=aws
- --use-service-account-credentials=true
- --configure-cloud-routes=false
resources:
requests:
cpu: 200m
hostNetwork: true
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: cloud-controller-manager
namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: cloud-controller-manager:apiserver-authentication-reader
namespace: kube-system
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: extension-apiserver-authentication-reader
subjects:
- apiGroup: ""
kind: ServiceAccount
name: cloud-controller-manager
namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: system:cloud-controller-manager
rules:
- apiGroups:
- ""
resources:
- events
verbs:
- create
- patch
- update
- apiGroups:
- ""
resources:
- nodes
verbs:
- '*'
- apiGroups:
- ""
resources:
- nodes/status
verbs:
- patch
- apiGroups:
- ""
resources:
- services
verbs:
- list
- patch
- update
- watch
- apiGroups:
- ""
resources:
- services/status
verbs:
- list
- patch
- update
- watch
- apiGroups:
- ""
resources:
- serviceaccounts
verbs:
- create
- get
- list
- watch
- apiGroups:
- ""
resources:
- persistentvolumes
verbs:
- get
- list
- update
- watch
- apiGroups:
- ""
resources:
- endpoints
verbs:
- create
- get
- list
- watch
- update
- apiGroups:
- coordination.k8s.io
resources:
- leases
verbs:
- create
- get
- list
- watch
- update
- apiGroups:
- ""
resources:
- serviceaccounts/token
verbs:
- create
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: system:cloud-controller-manager
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: system:cloud-controller-manager
subjects:
- apiGroup: ""
kind: ServiceAccount
name: cloud-controller-manager
namespace: kube-system
```

## Deploy the EBS CSI Driver

The easiest way to deploy the EBS CSI Driver is with the helm chart. Luckily,
Canonical K8s has a built-in helm command you can leverage.

If you want to create encrypted drives, you need to add the statement to the policy you are using for the instance.

Check failure on line 340 in docs/src/snap/howto/storage/cloud.md

View workflow job for this annotation

GitHub Actions / markdown-lint

Line length

docs/src/snap/howto/storage/cloud.md:340:81 MD013/line-length Line length [Expected: 80; Actual: 115] https://github.com/DavidAnson/markdownlint/blob/v0.34.0/doc/md013.md

Check failure on line 340 in docs/src/snap/howto/storage/cloud.md

View workflow job for this annotation

GitHub Actions / markdown-lint

Line length

docs/src/snap/howto/storage/cloud.md:340:81 MD013/line-length Line length [Expected: 80; Actual: 115] https://github.com/DavidAnson/markdownlint/blob/v0.34.0/doc/md013.md

```
{
"Effect": "Allow",
"Action": [
"kms:Decrypt",
"kms:GenerateDataKeyWithoutPlaintext",
"kms:CreateGrant"
],
"Resource": "*"
}
```

Then, add the helm repo for the EBS CSI Driver.

```
sudo k8s helm repo add aws-ebs-csi-driver https://kubernetes-sigs.github.io/aws-ebs-csi-driver
sudo k8s helm repo update
```

Finally, install the Helm chart, making sure to set the correct region as an argument.

```
sudo k8s helm upgrade --install aws-ebs-csi-driver \
--namespace kube-system \
aws-ebs-csi-driver/aws-ebs-csi-driver \
--set controller.region=us-east-2
```

Once the command completes, you can verify the pods are successfully deployed:

```
kubectl get pods -n kube-system -l app.kubernetes.io/name=aws-ebs-csi-driver
```

The status of all pods should be "Running".

<!-- LINKS -->
[getting-started-guide]: /snap/tutorial/getting-started.md

0 comments on commit 30e6be5

Please sign in to comment.