-
Notifications
You must be signed in to change notification settings - Fork 14
Commit
explain how to use ebs volumes with the standalone k8s snap
- Loading branch information
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,379 @@ | ||
# How to use cloud storage | ||
|
||
{{product}} offers a local-storage option to quickly set up and run a | ||
cluster, especially for single-node support. This guide walks you through | ||
enabling and configuring this feature. | ||
|
||
## What you'll need | ||
|
||
This guide assumes the following: | ||
|
||
- You have root or sudo access to an Amazon EC2 instance | ||
- You can create roles and policies in AWS | ||
|
||
## Set your host name | ||
|
||
The cloud controller manager uses the node name to correctly associate the node with an EC2 instance. In Canonical K8s, the node name is derived from the hostname of the machine. Therefore, before bootstrapping the cluster, we must first set an appropriate host name. | ||
Check failure on line 16 in docs/src/snap/howto/storage/cloud.md GitHub Actions / markdown-lintLine length
Check failure on line 16 in docs/src/snap/howto/storage/cloud.md GitHub Actions / markdown-lintLine length
|
||
|
||
``` | ||
echo "$(sudo cloud-init query ds.meta_data.local-hostname)" | sudo tee /etc/hostname | ||
``` | ||
|
||
Then, reboot the machine. | ||
|
||
When the machine is up, use `hostname -f` to check the host name. It should look like: | ||
Check failure on line 24 in docs/src/snap/howto/storage/cloud.md GitHub Actions / markdown-lintLine length
Check failure on line 24 in docs/src/snap/howto/storage/cloud.md GitHub Actions / markdown-lintLine length
|
||
|
||
``` | ||
ip-172-31-11-86.us-east-2.compute.internal | ||
``` | ||
|
||
This format is called IP-based naming and is specific to AWS. | ||
|
||
``` | ||
{note} Don't rely on the PS1 prompt to know if your host name was changed successfully. The PS1 prompt only displays the hostname up to the first `.`. | ||
``` | ||
|
||
## Set IAM Policies | ||
|
||
Your instance will need a few IAM policies to be able to communciate with the AWS APIs. These policies are quite open and can be scaled back if desired. | ||
Check failure on line 38 in docs/src/snap/howto/storage/cloud.md GitHub Actions / markdown-lintLine length
Check failure on line 38 in docs/src/snap/howto/storage/cloud.md GitHub Actions / markdown-lintLine length
|
||
|
||
For a control plane node: | ||
```json | ||
Check failure on line 41 in docs/src/snap/howto/storage/cloud.md GitHub Actions / markdown-lintFenced code blocks should be surrounded by blank lines
Check failure on line 41 in docs/src/snap/howto/storage/cloud.md GitHub Actions / markdown-lintFenced code blocks should be surrounded by blank lines
|
||
{ | ||
"Version": "2012-10-17", | ||
"Statement": [ | ||
{ | ||
"Effect": "Allow", | ||
"Action": [ | ||
"autoscaling:DescribeAutoScalingGroups", | ||
"autoscaling:DescribeLaunchConfigurations", | ||
"autoscaling:DescribeTags", | ||
"ec2:DescribeInstances", | ||
"ec2:DescribeRegions", | ||
"ec2:DescribeRouteTables", | ||
"ec2:DescribeSecurityGroups", | ||
"ec2:DescribeSubnets", | ||
"ec2:DescribeVolumes", | ||
"ec2:DescribeAvailabilityZones", | ||
"ec2:CreateSecurityGroup", | ||
"ec2:CreateTags", | ||
"ec2:CreateVolume", | ||
"ec2:ModifyInstanceAttribute", | ||
"ec2:ModifyVolume", | ||
"ec2:AttachVolume", | ||
"ec2:AuthorizeSecurityGroupIngress", | ||
"ec2:CreateRoute", | ||
"ec2:DeleteRoute", | ||
"ec2:DeleteSecurityGroup", | ||
"ec2:DeleteVolume", | ||
"ec2:DetachVolume", | ||
"ec2:RevokeSecurityGroupIngress", | ||
"ec2:DescribeVpcs", | ||
"ec2:DescribeInstanceTopology", | ||
"elasticloadbalancing:AddTags", | ||
"elasticloadbalancing:AttachLoadBalancerToSubnets", | ||
"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer", | ||
"elasticloadbalancing:CreateLoadBalancer", | ||
"elasticloadbalancing:CreateLoadBalancerPolicy", | ||
"elasticloadbalancing:CreateLoadBalancerListeners", | ||
"elasticloadbalancing:ConfigureHealthCheck", | ||
"elasticloadbalancing:DeleteLoadBalancer", | ||
"elasticloadbalancing:DeleteLoadBalancerListeners", | ||
"elasticloadbalancing:DescribeLoadBalancers", | ||
"elasticloadbalancing:DescribeLoadBalancerAttributes", | ||
"elasticloadbalancing:DetachLoadBalancerFromSubnets", | ||
"elasticloadbalancing:DeregisterInstancesFromLoadBalancer", | ||
"elasticloadbalancing:ModifyLoadBalancerAttributes", | ||
"elasticloadbalancing:RegisterInstancesWithLoadBalancer", | ||
"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer", | ||
"elasticloadbalancing:AddTags", | ||
"elasticloadbalancing:CreateListener", | ||
"elasticloadbalancing:CreateTargetGroup", | ||
"elasticloadbalancing:DeleteListener", | ||
"elasticloadbalancing:DeleteTargetGroup", | ||
"elasticloadbalancing:DescribeListeners", | ||
"elasticloadbalancing:DescribeLoadBalancerPolicies", | ||
"elasticloadbalancing:DescribeTargetGroups", | ||
"elasticloadbalancing:DescribeTargetHealth", | ||
"elasticloadbalancing:ModifyListener", | ||
"elasticloadbalancing:ModifyTargetGroup", | ||
"elasticloadbalancing:RegisterTargets", | ||
"elasticloadbalancing:DeregisterTargets", | ||
"elasticloadbalancing:SetLoadBalancerPoliciesOfListener", | ||
"iam:CreateServiceLinkedRole", | ||
"kms:DescribeKey" | ||
], | ||
"Resource": [ | ||
"*" | ||
] | ||
} | ||
] | ||
} | ||
``` | ||
|
||
For a worker node: | ||
```json | ||
Check failure on line 115 in docs/src/snap/howto/storage/cloud.md GitHub Actions / markdown-lintFenced code blocks should be surrounded by blank lines
Check failure on line 115 in docs/src/snap/howto/storage/cloud.md GitHub Actions / markdown-lintFenced code blocks should be surrounded by blank lines
|
||
{ | ||
"Version": "2012-10-17", | ||
"Statement": [ | ||
{ | ||
"Effect": "Allow", | ||
"Action": [ | ||
"ec2:DescribeInstances", | ||
"ec2:DescribeRegions", | ||
"ecr:GetAuthorizationToken", | ||
"ecr:BatchCheckLayerAvailability", | ||
"ecr:GetDownloadUrlForLayer", | ||
"ecr:GetRepositoryPolicy", | ||
"ecr:DescribeRepositories", | ||
"ecr:ListImages", | ||
"ecr:BatchGetImage" | ||
], | ||
"Resource": "*" | ||
} | ||
] | ||
} | ||
``` | ||
|
||
## Bootstrap Canonical K8s | ||
|
||
Now that your machine has an appropriate host name, you are ready to bootstrap Canonical K8s. | ||
Check failure on line 140 in docs/src/snap/howto/storage/cloud.md GitHub Actions / markdown-lintLine length
Check failure on line 140 in docs/src/snap/howto/storage/cloud.md GitHub Actions / markdown-lintLine length
|
||
|
||
First, create a bootstrap configuration file that sets the cloud-provider configuration to "external". | ||
Check failure on line 142 in docs/src/snap/howto/storage/cloud.md GitHub Actions / markdown-lintLine length
Check failure on line 142 in docs/src/snap/howto/storage/cloud.md GitHub Actions / markdown-lintLine length
|
||
|
||
``` | ||
echo "cluster-config: | ||
cloud-provider: external" > bootstrap-config.yaml | ||
``` | ||
|
||
Then, bootstrap the cluster: | ||
|
||
``` | ||
sudo k8s bootstrap --file ./bootstrap-config.yaml | ||
sudo k8s status --wait-ready | ||
``` | ||
|
||
## Deploy the cloud controller manager | ||
|
||
Now that you have an appropriate host name, policies, and a Canonical K8s cluster, you have everything you need to deploy the cloud controller manager. | ||
Check failure on line 158 in docs/src/snap/howto/storage/cloud.md GitHub Actions / markdown-lintLine length
Check failure on line 158 in docs/src/snap/howto/storage/cloud.md GitHub Actions / markdown-lintLine length
|
||
|
||
Here is a YAML definition file that sets appropriate defaults for you, it configures the necessary service accounts, roles, and daemonsets: | ||
Check failure on line 160 in docs/src/snap/howto/storage/cloud.md GitHub Actions / markdown-lintLine length
Check failure on line 160 in docs/src/snap/howto/storage/cloud.md GitHub Actions / markdown-lintLine length
|
||
|
||
``` | ||
--- | ||
apiVersion: apps/v1 | ||
kind: DaemonSet | ||
metadata: | ||
name: aws-cloud-controller-manager | ||
namespace: kube-system | ||
labels: | ||
k8s-app: aws-cloud-controller-manager | ||
spec: | ||
selector: | ||
matchLabels: | ||
k8s-app: aws-cloud-controller-manager | ||
updateStrategy: | ||
type: RollingUpdate | ||
template: | ||
metadata: | ||
labels: | ||
k8s-app: aws-cloud-controller-manager | ||
spec: | ||
nodeSelector: | ||
node-role.kubernetes.io/control-plane: "" | ||
tolerations: | ||
- key: node.cloudprovider.kubernetes.io/uninitialized | ||
value: "true" | ||
effect: NoSchedule | ||
- effect: NoSchedule | ||
key: node-role.kubernetes.io/control-plane | ||
affinity: | ||
nodeAffinity: | ||
requiredDuringSchedulingIgnoredDuringExecution: | ||
nodeSelectorTerms: | ||
- matchExpressions: | ||
- key: node-role.kubernetes.io/control-plane | ||
operator: Exists | ||
serviceAccountName: cloud-controller-manager | ||
containers: | ||
- name: aws-cloud-controller-manager | ||
image: registry.k8s.io/provider-aws/cloud-controller-manager:v1.28.3 | ||
args: | ||
- --v=2 | ||
- --cloud-provider=aws | ||
- --use-service-account-credentials=true | ||
- --configure-cloud-routes=false | ||
resources: | ||
requests: | ||
cpu: 200m | ||
hostNetwork: true | ||
--- | ||
apiVersion: v1 | ||
kind: ServiceAccount | ||
metadata: | ||
name: cloud-controller-manager | ||
namespace: kube-system | ||
--- | ||
apiVersion: rbac.authorization.k8s.io/v1 | ||
kind: RoleBinding | ||
metadata: | ||
name: cloud-controller-manager:apiserver-authentication-reader | ||
namespace: kube-system | ||
roleRef: | ||
apiGroup: rbac.authorization.k8s.io | ||
kind: Role | ||
name: extension-apiserver-authentication-reader | ||
subjects: | ||
- apiGroup: "" | ||
kind: ServiceAccount | ||
name: cloud-controller-manager | ||
namespace: kube-system | ||
--- | ||
apiVersion: rbac.authorization.k8s.io/v1 | ||
kind: ClusterRole | ||
metadata: | ||
name: system:cloud-controller-manager | ||
rules: | ||
- apiGroups: | ||
- "" | ||
resources: | ||
- events | ||
verbs: | ||
- create | ||
- patch | ||
- update | ||
- apiGroups: | ||
- "" | ||
resources: | ||
- nodes | ||
verbs: | ||
- '*' | ||
- apiGroups: | ||
- "" | ||
resources: | ||
- nodes/status | ||
verbs: | ||
- patch | ||
- apiGroups: | ||
- "" | ||
resources: | ||
- services | ||
verbs: | ||
- list | ||
- patch | ||
- update | ||
- watch | ||
- apiGroups: | ||
- "" | ||
resources: | ||
- services/status | ||
verbs: | ||
- list | ||
- patch | ||
- update | ||
- watch | ||
- apiGroups: | ||
- "" | ||
resources: | ||
- serviceaccounts | ||
verbs: | ||
- create | ||
- get | ||
- list | ||
- watch | ||
- apiGroups: | ||
- "" | ||
resources: | ||
- persistentvolumes | ||
verbs: | ||
- get | ||
- list | ||
- update | ||
- watch | ||
- apiGroups: | ||
- "" | ||
resources: | ||
- endpoints | ||
verbs: | ||
- create | ||
- get | ||
- list | ||
- watch | ||
- update | ||
- apiGroups: | ||
- coordination.k8s.io | ||
resources: | ||
- leases | ||
verbs: | ||
- create | ||
- get | ||
- list | ||
- watch | ||
- update | ||
- apiGroups: | ||
- "" | ||
resources: | ||
- serviceaccounts/token | ||
verbs: | ||
- create | ||
--- | ||
kind: ClusterRoleBinding | ||
apiVersion: rbac.authorization.k8s.io/v1 | ||
metadata: | ||
name: system:cloud-controller-manager | ||
roleRef: | ||
apiGroup: rbac.authorization.k8s.io | ||
kind: ClusterRole | ||
name: system:cloud-controller-manager | ||
subjects: | ||
- apiGroup: "" | ||
kind: ServiceAccount | ||
name: cloud-controller-manager | ||
namespace: kube-system | ||
``` | ||
|
||
## Deploy the EBS CSI Driver | ||
|
||
The easiest way to deploy the EBS CSI Driver is with the helm chart. Luckily, | ||
Canonical K8s has a built-in helm command you can leverage. | ||
|
||
If you want to create encrypted drives, you need to add the statement to the policy you are using for the instance. | ||
Check failure on line 340 in docs/src/snap/howto/storage/cloud.md GitHub Actions / markdown-lintLine length
Check failure on line 340 in docs/src/snap/howto/storage/cloud.md GitHub Actions / markdown-lintLine length
|
||
|
||
``` | ||
{ | ||
"Effect": "Allow", | ||
"Action": [ | ||
"kms:Decrypt", | ||
"kms:GenerateDataKeyWithoutPlaintext", | ||
"kms:CreateGrant" | ||
], | ||
"Resource": "*" | ||
} | ||
``` | ||
|
||
Then, add the helm repo for the EBS CSI Driver. | ||
|
||
``` | ||
sudo k8s helm repo add aws-ebs-csi-driver https://kubernetes-sigs.github.io/aws-ebs-csi-driver | ||
sudo k8s helm repo update | ||
``` | ||
|
||
Finally, install the Helm chart, making sure to set the correct region as an argument. | ||
|
||
``` | ||
sudo k8s helm upgrade --install aws-ebs-csi-driver \ | ||
--namespace kube-system \ | ||
aws-ebs-csi-driver/aws-ebs-csi-driver \ | ||
--set controller.region=us-east-2 | ||
``` | ||
|
||
Once the command completes, you can verify the pods are successfully deployed: | ||
|
||
``` | ||
kubectl get pods -n kube-system -l app.kubernetes.io/name=aws-ebs-csi-driver | ||
``` | ||
|
||
The status of all pods should be "Running". | ||
|
||
<!-- LINKS --> | ||
[getting-started-guide]: /snap/tutorial/getting-started.md |