! This container is having trouble accessing https://registry.k8s.io

[apiszcz]

What Happened?

minikube start --embed-certs

! This container is having trouble accessing https://registry.k8s.io
W0114 09:42:59.673348 875426 out.go:239] * To pull new external images, you may need to configure a proxy: https://minikube.sigs.k8s.io/docs/reference/networking/proxy/

• To pull new external images, you may need to configure a proxy: https://minikube.sigs.k8s.io/docs/reference/networking/proxy/

Attach the log file

Docker driver.

Operating System

Ubuntu

Driver

Docker

[devarsh10]

Hey,
You might have not turned the internet on.
The same mistake as I do.

Here, I firstly wrote minikube start --embed-certs when I was not connected with internet.
Then I stopped minikube from Docker Desktop and I started it again after connecting with internet. And I got the issue solved(maybe!).

I hope, that I have solved this issue.
Regards.

[rbrt-stn]

I have the same issue.
@apiszcz, have you found a solution?

[apiszcz]

no.

[yang-hai-feng]

Same here

[david-hamilton-bah]

I'm also having the same issue. I do not have a proxy configured, and I can connect to the site in my browser.

[yang-hai-feng]

We found the issues only happens when I try to install Docker Engine directly in WSL2(Ubuntu), switch to Docker Desktop fixed this issue, major difference is when using minikube ssh -- cat /etc/resolv.conf, it shows different nameservers:

with docker desktop:

nameserver 192.168.65.2
options ndots:0

with docker engine:

nameserver 192.168.49.1
options ndots:0

[apiszcz]

Thank you!

[rbrt-stn]

We found the issues only happens when I try to install Docker Engine directly in WSL2(Ubuntu), switch to Docker Desktop fixed this issue, major difference is when using minikube ssh -- cat /etc/resolv.conf, it shows different nameservers:

with docker desktop:

nameserver 192.168.65.2 options ndots:0

with docker engine:

nameserver 192.168.49.1 options ndots:0

Unfortunately I am unable to install Docker Desktop on my machine (it's a work laptop and Docker Desktop requires licence which I don't have). So this solution isn't suitable for me.

Do you have any other recommendation?

[yang-hai-feng]

My recommendation is using Ubuntu VM directly, the systemd of WSL had some issues

[mehdi-eaton]

how to use ubuntu vm directly?

I have docker desktop, still getting same issue.

[johncuyle]

We found the issues only happens when I try to install Docker Engine directly in WSL2(Ubuntu), switch to Docker Desktop fixed this issue, major difference is when using minikube ssh -- cat /etc/resolv.conf, it shows different nameservers:

with docker desktop:

nameserver 192.168.65.2 options ndots:0

with docker engine:

nameserver 192.168.49.1 options ndots:0

This doesn't fully explain the issue, though. I have three development PCs, different hardware but the same OS configuration. All three are:

minikube version: v1.32.0
commit: 8220a6e
Docker version 24.0.7, build afdd53b
Distributor ID: Ubuntu
Description: Ubuntu 22.04.3 LTS
Release: 22.04
Codename: jammy
WSL version: 2.0.9.0
Kernel version: 5.15.133.1-1
WSLg version: 1.0.59
MSRDC version: 1.2.4677
Direct3D version: 1.611.1-81528511
DXCore version: 10.0.25131.1002-220531-1700.rs-onecore-base2-hyp
Windows version: 10.0.22631.2715

All three are running Docker engine under WSL. All three have identical docker network configurations for minikube:

[
{
"Name": "minikube",
"Id": "214e994e67324dc2187a1f329f299fa5f9e44f889e92ee7c3cc9f6d808ecef54",
"Created": "2023-11-25T08:38:41.585466316-08:00",
"Scope": "local",
"Driver": "bridge",
"EnableIPv6": false,
"IPAM": {
"Driver": "default",
"Options": {},
"Config": [
{
"Subnet": "192.168.49.0/24",
"Gateway": "192.168.49.1"
}
]
},
"Internal": false,
"Attachable": false,
"Ingress": false,
"ConfigFrom": {
"Network": ""
},
"ConfigOnly": false,
"Containers": {
"11d36138d6ec128c89c4ac174b061a6ca3f4294c2b8a6f49621e4d2c9ec242c2": {
"Name": "minikube",
"EndpointID": "5f11354c36ae2c26f8871eceb48ea1532e5420f0be1816d9f015dbe1d1a49de3",
"MacAddress": "02:42:c0:a8:31:02",
"IPv4Address": "192.168.49.2/24",
"IPv6Address": ""
}
},
"Options": {
"--icc": "",
"--ip-masq": "",
"com.docker.network.driver.mtu": "1500"
},
"Labels": {
"created_by.minikube.sigs.k8s.io": "true",
"name.minikube.sigs.k8s.io": "minikube"
}
}
]

You can see the gateway is 192.168.49.1.

If I remote into the minikube container, cat /etc/resolv.conf gives me:

nameserver 192.168.49.1
options ndots:0

This is the correct configuration for pointing the nameserver at the gateway. On two of the machines (the ones that work) if I then do nslookup registry.k8s.io I get:

Server: 192.168.49.1
Address: 192.168.49.1#53
Non-authoritative answer:
Name: registry.k8s.io
Address: 34.96.108.209
Name: registry.k8s.io
Address: 2600:1901:0:bbc4::

The issue on the machine that's displaying the title error is that when you do an nslookup from the host machine, it works, but not from the container:

...:~$ nslookup registry.k8s.io
Server: 172.17.176.1
Address: 172.17.176.1#53

Non-authoritative answer:
Name: registry.k8s.io
Address: 34.96.108.209
Name: registry.k8s.io
Address: 2600:1901:0:bbc4::

...:~$ docker exec -it minikube /bin/bash
root@minikube:/# nslookup registry.k8s.io
Server: 192.168.49.1
Address: 192.168.49.1#53

** server can't find registry.k8s.io: SERVFAIL

root@minikube:/#

The problem isn't that 192.168.49.1 is the wrong configuration. It's definitely the correct configuration. When running under WSL using desktop probably avoids the issue because docker desktop would provide DNS services via the host in a way that doesn't involve WSL. The actual issue appears to be something more along the lines of "DNS resolution fails when running minikube on Ubuntu 22.04 despite being correctly configured on some system configurations." All three of these machines have reasonably fresh installations of Ubuntu. The machine I'm having issues with is brand new. Since the WSL parition was brand new I blew it away and started from scratch. Same issue. It appears to somehow be machine specific, which is very strange since the issue looks like it's somewhere between Minikube and the Docker networking stack. It's not blocking all traffic or preventing access to the host, though. If I run a service that listens on port 5555 on the host and connect to minikube I can do this:

$ docker exec -it minikube /bin/bash
root@minikube:/# cat /etc/resolv.conf
nameserver 192.168.49.1
options ndots:0
root@minikube:/# curl 192.168.49.1:5555/ops/health
ok
root@minikube:/#

And that's working fine. Likewise nslookup doesn't appear to be failing to find dns services, it's just that resolution inexplicably fails. I'm not sure what additional diagnosis I can do on this issue, but given that I both have it working fine on two machines and have the issue on a third, I'd be happy to provide any additional information necessary to track the issue down. For licensing reasons, using Docker Desktop isn't an option for me, and I need this working on the new machine as it is supposed to replace one of the other machines (which is now 9 years old and needs to be retired.)

[SuranSandeepa]

I'm experiencing the same issue. I don't have a proxy set up, yet I can access the site through my browser.

[hharshas]

facing the same issue, changed the dns server so every nslookup works good inside the minikube, however dns only work for the domain name not https://registry.k8s.io/. don't know what to do. due to this i am unable to access the argocd rollout dashboard however the cli works fine. i guess cli work work on Kubernetes api but the dashboard require domain registry. i don't know please help !!!

[hharshas]

facing the same issue, changed the dns server so every nslookup works good inside the minikube, however dns only work for the domain name not https://registry.k8s.io/. don't know what to do. due to this i am unable to access the argocd rollout dashboard however the cli works fine. i guess cli work work on Kubernetes api but the dashboard require domain registry. i don't know please help !!!

scaled down to one version, however the dashboard is still not running.