Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add CIDR for microcluster Network Interface Configuration #449

Merged
merged 6 commits into from
Jun 5, 2024

Conversation

mateoflorido
Copy link
Member

@mateoflorido mateoflorido commented May 29, 2024

Overview

Infer the address for the microcluster by specifying a CIDR in the address flag.

Rationale

This allows us to have configuration in the ClusterAPI resources (where individual node IPs are not known yet) to choose the network interface that will be used by microcluster.

Testing

Bootstrapping

root@kcp-1:/k8s# k8s bootstrap --address 10.20.30.0/24
Bootstrapping the cluster. This may take a few seconds, please wait.
Bootstrapped a new Kubernetes cluster with node address "10.20.30.241:6400".
The node will be 'Ready' to host workloads after the CNI is deployed successfully.

root@kcp-1:/k8s# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
55: eth0@if56: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 00:16:3e:5d:e3:df brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 10.45.248.89/24 metric 100 brd 10.45.248.255 scope global dynamic eth0
       valid_lft 3503sec preferred_lft 3503sec
    inet6 fe80::216:3eff:fe5d:e3df/64 scope link
       valid_lft forever preferred_lft forever
57: eth1@if58: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 00:16:3e:68:18:bb brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 10.20.30.241/24 brd 10.20.30.255 scope global dynamic eth1
       valid_lft 3522sec preferred_lft 3522sec
    inet6 fe80::216:3eff:fe68:18bb/64 scope link
       valid_lft forever preferred_lft forever

Joining

root@kcp-3:~# k8s join-cluster eyJuYW1lIjoia2NwLTMiLCJzZWNyZXQiOiI5NzE0MmIxZDU3MGFkZTlmYzU2ZGE4ZGU4M2Q5Y2NlZDI2ZWM3ZjdkMjg0MGE0ZTZmZmM5ZTcyNWI0YzQ0OGExIiwiZmluZ2VycHJpbnQiOiI0MWJjYTVhZGVhMTZlZDJkNjgzYTY2MmY2M2EyYTJiNTEzMjlmNDkwNzllMjJiZjY0M2ZlN2NmNTlhOWZkZmVlIiwiam9pbl9hZGRyZXNzZXMiOlsiMTAuMjAuMzAuMjQxOjY0MDAiLCIxMC4yMC4zMC4xMTI6NjQwMCJdfQ== --address 10.20.30.0/24
Joining the cluster. This may take a few seconds, please wait.
Cluster services have started on "kcp-3".
Please allow some time for initial Kubernetes node registration.

microcluster running on the specified CIDR:

Netid  State   Recv-Q  Send-Q          Local Address:Port      Peer Address:Port  Process
tcp    LISTEN  0       4096             10.20.30.126:6400           0.0.0.0:*      users:(("k8sd",pid=1705,fd=12))

@mateoflorido mateoflorido requested a review from a team as a code owner May 29, 2024 01:20
@mateoflorido
Copy link
Member Author

Test with CIDR and ports:

root@kcp-1:~# k8s bootstrap --address 10.20.30.0/24:9875
Bootstrapping the cluster. This may take a few seconds, please wait.
Bootstrapped a new Kubernetes cluster with node address "10.20.30.86:9875".
The node will be 'Ready' to host workloads after the CNI is deployed successfully.
root@kcp-1:~# ss -tulpn | grep k8sd
tcp   LISTEN 0      4096         10.20.30.86:9875       0.0.0.0:*    users:(("k8sd",pid=1643,fd=11))
root@kcp-2:~# k8s join-cluster eyJuYW1lIjoia2NwLTIiLCJzZWNyZXQiOiJjZTk1ZmJjN2Y1YTM1YmE0ODg1YTlkNzA1ZTE5MmUyNDFiMDM2OTkyODU2NGU4YzA5NzcyZDkzZTMzNDI5ZDIxIiwiZmluZ2VycHJpbnQiOiIxMDA2NWEzM2FlNjA3ZmJlMWVkMTE3OGVjMjU5OWVlZGU0MGVhMWY5OTZhYTM4MmRjNWM5OGQwNGQ5Yjg0ZWU2Iiwiam9pbl9hZGRyZXNzZXMiOlsiMTAuMjAuMzAuODY6OTg3NSJdfQ== --address 10.20.30.0/24:9875
Joining the cluster. This may take a few seconds, please wait.
Cluster services have started on "kcp-2".
Please allow some time for initial Kubernetes node registration.

root@kcp-2:~# k8s kubectl get no -A -o wide
NAME    STATUS   ROLES                  AGE     VERSION   INTERNAL-IP    EXTERNAL-IP   OS-IMAGE             KERNEL-VERSION     CONTAINER-RUNTIME
kcp-1   Ready    control-plane,worker   11m     v1.30.1   10.20.30.86    <none>        Ubuntu 22.04.4 LTS   6.5.0-35-generic   containerd://1.6.32
kcp-2   Ready    control-plane,worker   5m41s   v1.30.1   10.20.30.138   <none>        Ubuntu 22.04.4 LTS   6.5.0-35-generic   containerd://1.6.32
root@kcp-2:~# ss -tulpn | grep k8sd
tcp   LISTEN 0      4096         10.20.30.138:9875       0.0.0.0:*    users:(("k8sd",pid=1681,fd=12))

Copy link
Contributor

@neoaggelos neoaggelos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent work, almost there!

src/k8s/pkg/utils/cidr.go Outdated Show resolved Hide resolved
src/k8s/pkg/utils/cidr.go Outdated Show resolved Hide resolved
src/k8s/pkg/utils/cidr.go Outdated Show resolved Hide resolved
@mateoflorido mateoflorido force-pushed the KU-882/microcluster-cidr branch from d104f2f to 4d83456 Compare May 31, 2024 14:56
Copy link
Contributor

@neoaggelos neoaggelos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a final change and this should be good to go! Nice work @mateoflorido !

src/k8s/pkg/utils/cidr.go Outdated Show resolved Hide resolved
Copy link
Contributor

@neoaggelos neoaggelos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

final note on style, then we're good to merge

src/k8s/cmd/k8s/k8s_join_cluster.go Outdated Show resolved Hide resolved
src/k8s/cmd/k8s/k8s_bootstrap.go Outdated Show resolved Hide resolved
Copy link
Contributor

@bschimke95 bschimke95 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work Mateo! Only add some minor unittests for the new util function and we are good to go!

src/k8s/pkg/utils/cidr.go Show resolved Hide resolved
Copy link
Contributor

@bschimke95 bschimke95 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, nicely done!

@mateoflorido mateoflorido merged commit 5adaf4a into main Jun 5, 2024
14 checks passed
@mateoflorido mateoflorido deleted the KU-882/microcluster-cidr branch June 5, 2024 12:06
eaudetcobello pushed a commit that referenced this pull request Jun 6, 2024
Allow the user to provide an address, CIDR and/or a port using the `--address` flag in the bootstrap and join commands.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants