* * ==> Audit <== * |---------|--------------------------------|----------|--------------|---------|---------------------|---------------------| | Command | Args | Profile | User | Version | Start Time | End Time | |---------|--------------------------------|----------|--------------|---------|---------------------|---------------------| | start | --driver=qemu2 | minikube | caerulescens | v1.32.0 | 31 Jan 24 13:41 EST | | | | --network=builtin --nodes=3 | | | | | | | | --alsologtostderr | | | | | | | delete | | minikube | caerulescens | v1.32.0 | 31 Jan 24 13:43 EST | 31 Jan 24 13:43 EST | |---------|--------------------------------|----------|--------------|---------|---------------------|---------------------| * * ==> Last Start <== * Log file created at: 2024/01/31 13:41:31 Running on machine: hpc Binary: Built with gc go1.21.3 for linux/amd64 Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg I0131 13:41:31.317010 230266 out.go:296] Setting OutFile to fd 1 ... I0131 13:41:31.317839 230266 out.go:348] isatty.IsTerminal(1) = true I0131 13:41:31.317857 230266 out.go:309] Setting ErrFile to fd 2... I0131 13:41:31.317871 230266 out.go:348] isatty.IsTerminal(2) = true I0131 13:41:31.318834 230266 root.go:338] Updating PATH: /home/caerulescens/.minikube/bin W0131 13:41:31.319331 230266 root.go:314] Error reading config file at /home/caerulescens/.minikube/config/config.json: open /home/caerulescens/.minikube/config/config.json: no such file or directory I0131 13:41:31.320119 230266 out.go:303] Setting JSON to false I0131 13:41:31.332625 230266 start.go:128] hostinfo: {"hostname":"hpc","uptime":21327,"bootTime":1706705165,"procs":533,"os":"linux","platform":"debian","platformFamily":"debian","platformVersion":"12.4","kernelVersion":"6.1.0-16-amd64","kernelArch":"x86_64","virtualizationSystem":"kvm","virtualizationRole":"host","hostId":"43f27311-08f8-4675-bc7c-07b3d71ac9a3"} I0131 13:41:31.332726 230266 start.go:138] virtualization: kvm host I0131 13:41:31.336421 230266 out.go:177] ๐Ÿ˜„ minikube v1.32.0 on Debian 12.4 W0131 13:41:31.342205 230266 preload.go:295] Failed to list preload files: open /home/caerulescens/.minikube/cache/preloaded-tarball: no such file or directory I0131 13:41:31.342268 230266 notify.go:220] Checking for updates... I0131 13:41:31.342381 230266 driver.go:378] Setting default libvirt URI to qemu:///system I0131 13:41:31.345216 230266 out.go:177] โœจ Using the qemu2 driver based on user configuration I0131 13:41:31.350753 230266 start.go:298] selected driver: qemu2 I0131 13:41:31.350772 230266 start.go:902] validating driver "qemu2" against I0131 13:41:31.350789 230266 start.go:913] status for qemu2: {Installed:true Healthy:true Running:true NeedsImprovement:false Error: Reason: Fix: Doc: Version:} I0131 13:41:31.352165 230266 start_flags.go:309] no existing cluster config was found, will generate one from the flags W0131 13:41:31.352240 230266 out.go:239] โ— You are using the QEMU driver without a dedicated network, which doesn't support `minikube service` & `minikube tunnel` commands. I0131 13:41:31.354829 230266 start_flags.go:394] Using suggested 5333MB memory alloc based on sys=64144MB, container=0MB I0131 13:41:31.355001 230266 start_flags.go:913] Wait components to verify : map[apiserver:true system_pods:true] I0131 13:41:31.355053 230266 cni.go:84] Creating CNI manager for "" I0131 13:41:31.355065 230266 cni.go:136] 0 nodes found, recommending kindnet I0131 13:41:31.355078 230266 start_flags.go:318] Found "CNI" CNI - setting NetworkPlugin=cni I0131 13:41:31.355097 230266 start_flags.go:323] config: {Name:minikube KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.42@sha256:d35ac07dfda971cabee05e0deca8aeac772f885a5348e1a0c0b0a36db20fcfc0 Memory:5333 CPUs:2 DiskSize:20000 VMDriver: Driver:qemu2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.28.3 ClusterName:minikube Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop: ExposedPorts:[] ListenAddress: Network:builtin Subnet: MultiNodeRequested:true ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/caerulescens:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 AutoPauseInterval:1m0s GPUs:} I0131 13:41:31.355373 230266 iso.go:125] acquiring lock: {Name:mk1740523c59e03d8c54140cae0a0a85e92a5211 Clock:{} Delay:500ms Timeout:10m0s Cancel:} I0131 13:41:31.358510 230266 out.go:177] ๐Ÿ’ฟ Downloading VM boot image ... I0131 13:41:31.361511 230266 download.go:107] Downloading: https://storage.googleapis.com/minikube/iso/minikube-v1.32.1-amd64.iso?checksum=file:https://storage.googleapis.com/minikube/iso/minikube-v1.32.1-amd64.iso.sha256 -> /home/caerulescens/.minikube/cache/iso/amd64/minikube-v1.32.1-amd64.iso I0131 13:41:36.011109 230266 out.go:177] ๐Ÿ‘ Starting control plane node minikube in cluster minikube I0131 13:41:36.013936 230266 preload.go:132] Checking if preload exists for k8s version v1.28.3 and runtime docker I0131 13:41:36.083491 230266 preload.go:119] Found remote preload: https://storage.googleapis.com/minikube-preloaded-volume-tarballs/v18/v1.28.3/preloaded-images-k8s-v18-v1.28.3-docker-overlay2-amd64.tar.lz4 I0131 13:41:36.083509 230266 cache.go:56] Caching tarball of preloaded images I0131 13:41:36.083595 230266 preload.go:132] Checking if preload exists for k8s version v1.28.3 and runtime docker I0131 13:41:36.086514 230266 out.go:177] ๐Ÿ’พ Downloading Kubernetes v1.28.3 preload ... I0131 13:41:36.092374 230266 preload.go:238] getting checksum for preloaded-images-k8s-v18-v1.28.3-docker-overlay2-amd64.tar.lz4 ... I0131 13:41:36.207611 230266 download.go:107] Downloading: https://storage.googleapis.com/minikube-preloaded-volume-tarballs/v18/v1.28.3/preloaded-images-k8s-v18-v1.28.3-docker-overlay2-amd64.tar.lz4?checksum=md5:82104bbf889ff8b69d5c141ce86c05ac -> /home/caerulescens/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.3-docker-overlay2-amd64.tar.lz4 I0131 13:41:42.164770 230266 preload.go:249] saving checksum for preloaded-images-k8s-v18-v1.28.3-docker-overlay2-amd64.tar.lz4 ... I0131 13:41:42.164861 230266 preload.go:256] verifying checksum of /home/caerulescens/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.3-docker-overlay2-amd64.tar.lz4 ... I0131 13:41:42.776589 230266 cache.go:59] Finished verifying existence of preloaded tar for v1.28.3 on docker I0131 13:41:42.776874 230266 profile.go:148] Saving config to /home/caerulescens/.minikube/profiles/minikube/config.json ... I0131 13:41:42.776897 230266 lock.go:35] WriteFile acquiring /home/caerulescens/.minikube/profiles/minikube/config.json: {Name:mkb4c835d3ef9a84824c464ad333bc4b49016801 Clock:{} Delay:500ms Timeout:1m0s Cancel:} I0131 13:41:42.777027 230266 start.go:365] acquiring machines lock for minikube: {Name:mk730b7b7d5d10067abf794b343f53c46eaa903c Clock:{} Delay:500ms Timeout:13m0s Cancel:} I0131 13:41:42.777052 230266 start.go:369] acquired machines lock for "minikube" in 14.928ยตs I0131 13:41:42.777071 230266 start.go:93] Provisioning new machine with config: &{Name:minikube KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube/iso/minikube-v1.32.1-amd64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.42@sha256:d35ac07dfda971cabee05e0deca8aeac772f885a5348e1a0c0b0a36db20fcfc0 Memory:5333 CPUs:2 DiskSize:20000 VMDriver: Driver:qemu2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.28.3 ClusterName:minikube Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.28.3 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop: ExposedPorts:[] ListenAddress: Network:builtin Subnet: MultiNodeRequested:true ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/caerulescens:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 AutoPauseInterval:1m0s GPUs:} &{Name: IP: Port:8443 KubernetesVersion:v1.28.3 ContainerRuntime:docker ControlPlane:true Worker:true} I0131 13:41:42.777140 230266 start.go:125] createHost starting for "" (driver="qemu2") I0131 13:41:42.794228 230266 out.go:204] ๐Ÿ”ฅ Creating qemu2 VM (CPUs=2, Memory=5333MB, Disk=20000MB) ... I0131 13:41:42.794405 230266 start.go:159] libmachine.API.Create for "minikube" (driver="qemu2") I0131 13:41:42.794436 230266 client.go:168] LocalClient.Create starting I0131 13:41:42.794534 230266 main.go:141] libmachine: Creating CA: /home/caerulescens/.minikube/certs/ca.pem I0131 13:41:43.053212 230266 main.go:141] libmachine: Creating client certificate: /home/caerulescens/.minikube/certs/cert.pem I0131 13:41:43.102027 230266 main.go:141] libmachine: port range: 0 -> 65535 I0131 13:41:43.102196 230266 main.go:141] libmachine: Downloading /home/caerulescens/.minikube/cache/boot2docker.iso from file:///home/caerulescens/.minikube/cache/iso/amd64/minikube-v1.32.1-amd64.iso... I0131 13:41:43.312172 230266 main.go:141] libmachine: Creating SSH key... I0131 13:41:43.430871 230266 main.go:141] libmachine: Creating Disk image... I0131 13:41:43.430884 230266 main.go:141] libmachine: Creating 20000 MB hard disk image... I0131 13:41:43.430995 230266 main.go:141] libmachine: executing: qemu-img convert -f raw -O qcow2 /home/caerulescens/.minikube/machines/minikube/disk.qcow2.raw /home/caerulescens/.minikube/machines/minikube/disk.qcow2 I0131 13:41:43.463724 230266 main.go:141] libmachine: STDOUT: I0131 13:41:43.463745 230266 main.go:141] libmachine: STDERR: I0131 13:41:43.463793 230266 main.go:141] libmachine: executing: qemu-img resize /home/caerulescens/.minikube/machines/minikube/disk.qcow2 +20000M I0131 13:41:43.480828 230266 main.go:141] libmachine: STDOUT: Image resized. I0131 13:41:43.480880 230266 main.go:141] libmachine: STDERR: I0131 13:41:43.480909 230266 main.go:141] libmachine: DONE writing to /home/caerulescens/.minikube/machines/minikube/disk.qcow2.raw and /home/caerulescens/.minikube/machines/minikube/disk.qcow2 I0131 13:41:43.480930 230266 main.go:141] libmachine: Starting QEMU VM... I0131 13:41:43.481057 230266 main.go:141] libmachine: executing: qemu-system-x86_64 -cpu max -display none -accel kvm -m 5333 -smp 2 -boot d -cdrom /home/caerulescens/.minikube/machines/minikube/boot2docker.iso -qmp unix:/home/caerulescens/.minikube/machines/minikube/monitor,server,nowait -pidfile /home/caerulescens/.minikube/machines/minikube/qemu.pid -nic user,model=virtio,hostfwd=tcp::39193-:22,hostfwd=tcp::35355-:2376,hostname=minikube -daemonize /home/caerulescens/.minikube/machines/minikube/disk.qcow2 I0131 13:41:43.558651 230266 main.go:141] libmachine: STDOUT: I0131 13:41:43.558684 230266 main.go:141] libmachine: STDERR: I0131 13:41:43.558702 230266 main.go:141] libmachine: Waiting for VM to start (ssh -p 39193 docker@127.0.0.1)... I0131 13:42:15.076695 230266 machine.go:88] provisioning docker machine ... I0131 13:42:15.076731 230266 buildroot.go:166] provisioning hostname "minikube" I0131 13:42:15.076818 230266 main.go:141] libmachine: Using SSH client type: native I0131 13:42:15.077267 230266 main.go:141] libmachine: &{{{ 0 [] [] []} docker [0x808a40] 0x80b720 [] 0s} localhost 39193 } I0131 13:42:15.077287 230266 main.go:141] libmachine: About to run SSH command: sudo hostname minikube && echo "minikube" | sudo tee /etc/hostname I0131 13:42:15.177771 230266 main.go:141] libmachine: SSH cmd err, output: : minikube I0131 13:42:15.177878 230266 main.go:141] libmachine: Using SSH client type: native I0131 13:42:15.178307 230266 main.go:141] libmachine: &{{{ 0 [] [] []} docker [0x808a40] 0x80b720 [] 0s} localhost 39193 } I0131 13:42:15.178366 230266 main.go:141] libmachine: About to run SSH command: if ! grep -xq '.*\sminikube' /etc/hosts; then if grep -xq '127.0.1.1\s.*' /etc/hosts; then sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 minikube/g' /etc/hosts; else echo '127.0.1.1 minikube' | sudo tee -a /etc/hosts; fi fi I0131 13:42:15.266543 230266 main.go:141] libmachine: SSH cmd err, output: : I0131 13:42:15.266572 230266 buildroot.go:172] set auth options {CertDir:/home/caerulescens/.minikube CaCertPath:/home/caerulescens/.minikube/certs/ca.pem CaPrivateKeyPath:/home/caerulescens/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/home/caerulescens/.minikube/machines/server.pem ServerKeyPath:/home/caerulescens/.minikube/machines/server-key.pem ClientKeyPath:/home/caerulescens/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/home/caerulescens/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/home/caerulescens/.minikube} I0131 13:42:15.266601 230266 buildroot.go:174] setting up certificates I0131 13:42:15.266618 230266 provision.go:83] configureAuth start I0131 13:42:15.266633 230266 provision.go:138] copyHostCerts I0131 13:42:15.266699 230266 exec_runner.go:151] cp: /home/caerulescens/.minikube/certs/ca.pem --> /home/caerulescens/.minikube/ca.pem (1094 bytes) I0131 13:42:15.266850 230266 exec_runner.go:151] cp: /home/caerulescens/.minikube/certs/cert.pem --> /home/caerulescens/.minikube/cert.pem (1135 bytes) I0131 13:42:15.266941 230266 exec_runner.go:151] cp: /home/caerulescens/.minikube/certs/key.pem --> /home/caerulescens/.minikube/key.pem (1675 bytes) I0131 13:42:15.267017 230266 provision.go:112] generating server cert: /home/caerulescens/.minikube/machines/server.pem ca-key=/home/caerulescens/.minikube/certs/ca.pem private-key=/home/caerulescens/.minikube/certs/ca-key.pem org=caerulescens.minikube san=[127.0.0.1 localhost localhost 127.0.0.1 minikube minikube] I0131 13:42:15.420134 230266 provision.go:172] copyRemoteCerts I0131 13:42:15.420202 230266 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker I0131 13:42:15.420217 230266 sshutil.go:53] new ssh client: &{IP:localhost Port:39193 SSHKeyPath:/home/caerulescens/.minikube/machines/minikube/id_rsa Username:docker} I0131 13:42:15.472496 230266 ssh_runner.go:362] scp /home/caerulescens/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1094 bytes) I0131 13:42:15.492391 230266 ssh_runner.go:362] scp /home/caerulescens/.minikube/machines/server.pem --> /etc/docker/server.pem (1224 bytes) I0131 13:42:15.513263 230266 ssh_runner.go:362] scp /home/caerulescens/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1675 bytes) I0131 13:42:15.532820 230266 provision.go:86] duration metric: configureAuth took 266.185073ms I0131 13:42:15.532848 230266 buildroot.go:189] setting minikube options for container-runtime I0131 13:42:15.533050 230266 config.go:182] Loaded profile config "minikube": Driver=qemu2, ContainerRuntime=docker, KubernetesVersion=v1.28.3 I0131 13:42:15.533135 230266 main.go:141] libmachine: Using SSH client type: native I0131 13:42:15.533610 230266 main.go:141] libmachine: &{{{ 0 [] [] []} docker [0x808a40] 0x80b720 [] 0s} localhost 39193 } I0131 13:42:15.533637 230266 main.go:141] libmachine: About to run SSH command: df --output=fstype / | tail -n 1 I0131 13:42:15.619374 230266 main.go:141] libmachine: SSH cmd err, output: : tmpfs I0131 13:42:15.619395 230266 buildroot.go:70] root file system type: tmpfs I0131 13:42:15.619530 230266 provision.go:309] Updating docker unit: /lib/systemd/system/docker.service ... I0131 13:42:15.619597 230266 main.go:141] libmachine: Using SSH client type: native I0131 13:42:15.619993 230266 main.go:141] libmachine: &{{{ 0 [] [] []} docker [0x808a40] 0x80b720 [] 0s} localhost 39193 } I0131 13:42:15.620087 230266 main.go:141] libmachine: About to run SSH command: sudo mkdir -p /lib/systemd/system && printf %!s(MISSING) "[Unit] Description=Docker Application Container Engine Documentation=https://docs.docker.com After=network.target minikube-automount.service docker.socket Requires= minikube-automount.service docker.socket StartLimitBurst=3 StartLimitIntervalSec=60 [Service] Type=notify Restart=on-failure # This file is a systemd drop-in unit that inherits from the base dockerd configuration. # The base configuration already specifies an 'ExecStart=...' command. The first directive # here is to clear out that command inherited from the base configuration. Without this, # the command from the base configuration and the command specified here are treated as # a sequence of commands, which is not the desired behavior, nor is it valid -- systemd # will catch this invalid input and refuse to start the service with an error like: # Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services. # NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other # container runtimes. If left unlimited, it may result in OOM issues with MySQL. ExecStart= ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=qemu2 --insecure-registry 10.96.0.0/12 ExecReload=/bin/kill -s HUP \$MAINPID # Having non-zero Limit*s causes performance problems due to accounting overhead # in the kernel. We recommend using cgroups to do container-local accounting. LimitNOFILE=infinity LimitNPROC=infinity LimitCORE=infinity # Uncomment TasksMax if your systemd version supports it. # Only systemd 226 and above support this version. TasksMax=infinity TimeoutStartSec=0 # set delegate yes so that systemd does not reset the cgroups of docker containers Delegate=yes # kill only the docker process, not all processes in the cgroup KillMode=process [Install] WantedBy=multi-user.target " | sudo tee /lib/systemd/system/docker.service.new I0131 13:42:15.720213 230266 main.go:141] libmachine: SSH cmd err, output: : [Unit] Description=Docker Application Container Engine Documentation=https://docs.docker.com After=network.target minikube-automount.service docker.socket Requires= minikube-automount.service docker.socket StartLimitBurst=3 StartLimitIntervalSec=60 [Service] Type=notify Restart=on-failure # This file is a systemd drop-in unit that inherits from the base dockerd configuration. # The base configuration already specifies an 'ExecStart=...' command. The first directive # here is to clear out that command inherited from the base configuration. Without this, # the command from the base configuration and the command specified here are treated as # a sequence of commands, which is not the desired behavior, nor is it valid -- systemd # will catch this invalid input and refuse to start the service with an error like: # Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services. # NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other # container runtimes. If left unlimited, it may result in OOM issues with MySQL. ExecStart= ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=qemu2 --insecure-registry 10.96.0.0/12 ExecReload=/bin/kill -s HUP $MAINPID # Having non-zero Limit*s causes performance problems due to accounting overhead # in the kernel. We recommend using cgroups to do container-local accounting. LimitNOFILE=infinity LimitNPROC=infinity LimitCORE=infinity # Uncomment TasksMax if your systemd version supports it. # Only systemd 226 and above support this version. TasksMax=infinity TimeoutStartSec=0 # set delegate yes so that systemd does not reset the cgroups of docker containers Delegate=yes # kill only the docker process, not all processes in the cgroup KillMode=process [Install] WantedBy=multi-user.target I0131 13:42:15.720351 230266 main.go:141] libmachine: Using SSH client type: native I0131 13:42:15.720796 230266 main.go:141] libmachine: &{{{ 0 [] [] []} docker [0x808a40] 0x80b720 [] 0s} localhost 39193 } I0131 13:42:15.720823 230266 main.go:141] libmachine: About to run SSH command: sudo diff -u /lib/systemd/system/docker.service /lib/systemd/system/docker.service.new || { sudo mv /lib/systemd/system/docker.service.new /lib/systemd/system/docker.service; sudo systemctl -f daemon-reload && sudo systemctl -f enable docker && sudo systemctl -f restart docker; } I0131 13:42:16.526109 230266 main.go:141] libmachine: SSH cmd err, output: : diff: can't stat '/lib/systemd/system/docker.service': No such file or directory Created symlink /etc/systemd/system/multi-user.target.wants/docker.service โ†’ /usr/lib/systemd/system/docker.service. I0131 13:42:16.526141 230266 machine.go:91] provisioned docker machine in 1.449425219s I0131 13:42:16.526157 230266 client.go:171] LocalClient.Create took 33.731707419s I0131 13:42:16.526182 230266 start.go:167] duration metric: libmachine.API.Create for "minikube" took 33.731774785s I0131 13:42:16.526196 230266 start.go:300] post-start starting for "minikube" (driver="qemu2") I0131 13:42:16.526212 230266 start.go:329] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs] I0131 13:42:16.526312 230266 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs I0131 13:42:16.526336 230266 sshutil.go:53] new ssh client: &{IP:localhost Port:39193 SSHKeyPath:/home/caerulescens/.minikube/machines/minikube/id_rsa Username:docker} I0131 13:42:16.579525 230266 ssh_runner.go:195] Run: cat /etc/os-release I0131 13:42:16.582800 230266 info.go:137] Remote host: Buildroot 2021.02.12 I0131 13:42:16.582831 230266 filesync.go:126] Scanning /home/caerulescens/.minikube/addons for local assets ... I0131 13:42:16.582922 230266 filesync.go:126] Scanning /home/caerulescens/.minikube/files for local assets ... I0131 13:42:16.582957 230266 start.go:303] post-start completed in 56.748198ms I0131 13:42:16.583373 230266 profile.go:148] Saving config to /home/caerulescens/.minikube/profiles/minikube/config.json ... I0131 13:42:16.583529 230266 start.go:128] duration metric: createHost completed in 33.806376906s I0131 13:42:16.583602 230266 main.go:141] libmachine: Using SSH client type: native I0131 13:42:16.584024 230266 main.go:141] libmachine: &{{{ 0 [] [] []} docker [0x808a40] 0x80b720 [] 0s} localhost 39193 } I0131 13:42:16.584040 230266 main.go:141] libmachine: About to run SSH command: date +%!s(MISSING).%!N(MISSING) I0131 13:42:16.677188 230266 main.go:141] libmachine: SSH cmd err, output: : 1706726536.674751601 I0131 13:42:16.677216 230266 fix.go:206] guest clock: 1706726536.674751601 I0131 13:42:16.677233 230266 fix.go:219] Guest: 2024-01-31 13:42:16.674751601 -0500 EST Remote: 2024-01-31 13:42:16.583540911 -0500 EST m=+45.318502123 (delta=91.21069ms) I0131 13:42:16.677275 230266 fix.go:190] guest clock delta is within tolerance: 91.21069ms I0131 13:42:16.677286 230266 start.go:83] releasing machines lock for "minikube", held for 33.900223937s I0131 13:42:16.677438 230266 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/ I0131 13:42:16.677487 230266 sshutil.go:53] new ssh client: &{IP:localhost Port:39193 SSHKeyPath:/home/caerulescens/.minikube/machines/minikube/id_rsa Username:docker} I0131 13:42:16.677711 230266 ssh_runner.go:195] Run: cat /version.json I0131 13:42:16.677735 230266 sshutil.go:53] new ssh client: &{IP:localhost Port:39193 SSHKeyPath:/home/caerulescens/.minikube/machines/minikube/id_rsa Username:docker} I0131 13:42:16.776874 230266 ssh_runner.go:195] Run: systemctl --version I0131 13:42:16.781873 230266 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*" W0131 13:42:16.786142 230266 cni.go:209] loopback cni configuration skipped: "/etc/cni/net.d/*loopback.conf*" not found I0131 13:42:16.786279 230266 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f ( ( -name *bridge* -or -name *podman* ) -and -not -name *.mk_disabled ) -printf "%!p(MISSING), " -exec sh -c "sudo mv {} {}.mk_disabled" ; I0131 13:42:16.798778 230266 cni.go:262] disabled [/etc/cni/net.d/87-podman-bridge.conflist] bridge cni config(s) I0131 13:42:16.798814 230266 start.go:472] detecting cgroup driver to use... I0131 13:42:16.799004 230266 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %!s(MISSING) "runtime-endpoint: unix:///run/containerd/containerd.sock " | sudo tee /etc/crictl.yaml" I0131 13:42:16.813403 230266 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)sandbox_image = .*$|\1sandbox_image = "registry.k8s.io/pause:3.9"|' /etc/containerd/config.toml" I0131 13:42:16.821683 230266 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)restrict_oom_score_adj = .*$|\1restrict_oom_score_adj = false|' /etc/containerd/config.toml" I0131 13:42:16.829548 230266 containerd.go:145] configuring containerd to use "cgroupfs" as cgroup driver... I0131 13:42:16.829661 230266 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)SystemdCgroup = .*$|\1SystemdCgroup = false|g' /etc/containerd/config.toml" I0131 13:42:16.837826 230266 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runtime.v1.linux"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml" I0131 13:42:16.845857 230266 ssh_runner.go:195] Run: sh -c "sudo sed -i '/systemd_cgroup/d' /etc/containerd/config.toml" I0131 13:42:16.853843 230266 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runc.v1"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml" I0131 13:42:16.861806 230266 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk" I0131 13:42:16.867984 230266 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)conf_dir = .*$|\1conf_dir = "/etc/cni/net.d"|g' /etc/containerd/config.toml" I0131 13:42:16.872811 230266 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables I0131 13:42:16.877195 230266 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward" I0131 13:42:16.881739 230266 ssh_runner.go:195] Run: sudo systemctl daemon-reload I0131 13:42:16.964793 230266 ssh_runner.go:195] Run: sudo systemctl restart containerd I0131 13:42:16.976681 230266 start.go:472] detecting cgroup driver to use... I0131 13:42:16.976796 230266 ssh_runner.go:195] Run: sudo systemctl cat docker.service I0131 13:42:16.987098 230266 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service containerd I0131 13:42:16.996005 230266 ssh_runner.go:195] Run: sudo systemctl stop -f containerd I0131 13:42:17.009119 230266 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service containerd I0131 13:42:17.017663 230266 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio I0131 13:42:17.025470 230266 ssh_runner.go:195] Run: sudo systemctl stop -f crio I0131 13:42:17.070071 230266 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio I0131 13:42:17.078244 230266 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %!s(MISSING) "runtime-endpoint: unix:///var/run/cri-dockerd.sock " | sudo tee /etc/crictl.yaml" I0131 13:42:17.088116 230266 ssh_runner.go:195] Run: which cri-dockerd I0131 13:42:17.090909 230266 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/cri-docker.service.d I0131 13:42:17.097623 230266 ssh_runner.go:362] scp memory --> /etc/systemd/system/cri-docker.service.d/10-cni.conf (189 bytes) I0131 13:42:17.108484 230266 ssh_runner.go:195] Run: sudo systemctl unmask docker.service I0131 13:42:17.197962 230266 ssh_runner.go:195] Run: sudo systemctl enable docker.socket I0131 13:42:17.276110 230266 docker.go:560] configuring docker to use "cgroupfs" as cgroup driver... I0131 13:42:17.276256 230266 ssh_runner.go:362] scp memory --> /etc/docker/daemon.json (130 bytes) I0131 13:42:17.284166 230266 ssh_runner.go:195] Run: sudo systemctl daemon-reload I0131 13:42:17.360510 230266 ssh_runner.go:195] Run: sudo systemctl restart docker I0131 13:42:18.710856 230266 ssh_runner.go:235] Completed: sudo systemctl restart docker: (1.350307641s) I0131 13:42:18.710955 230266 ssh_runner.go:195] Run: sudo systemctl enable cri-docker.socket I0131 13:42:18.798526 230266 ssh_runner.go:195] Run: sudo systemctl unmask cri-docker.socket I0131 13:42:18.898606 230266 ssh_runner.go:195] Run: sudo systemctl enable cri-docker.socket I0131 13:42:18.978674 230266 ssh_runner.go:195] Run: sudo systemctl daemon-reload I0131 13:42:19.063110 230266 ssh_runner.go:195] Run: sudo systemctl restart cri-docker.socket I0131 13:42:19.072993 230266 ssh_runner.go:195] Run: sudo systemctl daemon-reload I0131 13:42:19.154107 230266 ssh_runner.go:195] Run: sudo systemctl restart cri-docker I0131 13:42:19.196340 230266 start.go:519] Will wait 60s for socket path /var/run/cri-dockerd.sock I0131 13:42:19.196447 230266 ssh_runner.go:195] Run: stat /var/run/cri-dockerd.sock I0131 13:42:19.199834 230266 start.go:540] Will wait 60s for crictl version I0131 13:42:19.199925 230266 ssh_runner.go:195] Run: which crictl I0131 13:42:19.202028 230266 ssh_runner.go:195] Run: sudo /usr/bin/crictl version I0131 13:42:19.231709 230266 start.go:556] Version: 0.1.0 RuntimeName: docker RuntimeVersion: 24.0.7 RuntimeApiVersion: v1 I0131 13:42:19.231804 230266 ssh_runner.go:195] Run: docker version --format {{.Server.Version}} I0131 13:42:19.247558 230266 ssh_runner.go:195] Run: docker version --format {{.Server.Version}} I0131 13:42:19.278228 230266 out.go:204] ๐Ÿณ Preparing Kubernetes v1.28.3 on Docker 24.0.7 ... I0131 13:42:19.278380 230266 ssh_runner.go:195] Run: grep 10.0.2.2 host.minikube.internal$ /etc/hosts I0131 13:42:19.280867 230266 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "10.0.2.2 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts"" I0131 13:42:19.290157 230266 preload.go:132] Checking if preload exists for k8s version v1.28.3 and runtime docker I0131 13:42:19.290245 230266 ssh_runner.go:195] Run: docker images --format {{.Repository}}:{{.Tag}} I0131 13:42:19.301896 230266 docker.go:671] Got preloaded images: I0131 13:42:19.301922 230266 docker.go:677] registry.k8s.io/kube-apiserver:v1.28.3 wasn't preloaded I0131 13:42:19.301997 230266 ssh_runner.go:195] Run: sudo cat /var/lib/docker/image/overlay2/repositories.json I0131 13:42:19.309594 230266 ssh_runner.go:195] Run: which lz4 I0131 13:42:19.312719 230266 ssh_runner.go:195] Run: stat -c "%!s(MISSING) %!y(MISSING)" /preloaded.tar.lz4 I0131 13:42:19.315966 230266 ssh_runner.go:352] existence check for /preloaded.tar.lz4: stat -c "%!s(MISSING) %!y(MISSING)" /preloaded.tar.lz4: Process exited with status 1 stdout: stderr: stat: cannot statx '/preloaded.tar.lz4': No such file or directory I0131 13:42:19.315990 230266 ssh_runner.go:362] scp /home/caerulescens/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.3-docker-overlay2-amd64.tar.lz4 --> /preloaded.tar.lz4 (422944352 bytes) I0131 13:42:21.594312 230266 docker.go:635] Took 2.281631 seconds to copy over tarball I0131 13:42:21.594426 230266 ssh_runner.go:195] Run: sudo tar -I lz4 -C /var -xf /preloaded.tar.lz4 I0131 13:42:23.814633 230266 ssh_runner.go:235] Completed: sudo tar -I lz4 -C /var -xf /preloaded.tar.lz4: (2.220155053s) I0131 13:42:23.814695 230266 ssh_runner.go:146] rm: /preloaded.tar.lz4 I0131 13:42:23.839970 230266 ssh_runner.go:195] Run: sudo cat /var/lib/docker/image/overlay2/repositories.json I0131 13:42:23.845834 230266 ssh_runner.go:362] scp memory --> /var/lib/docker/image/overlay2/repositories.json (2629 bytes) I0131 13:42:23.855932 230266 ssh_runner.go:195] Run: sudo systemctl daemon-reload I0131 13:42:23.942193 230266 ssh_runner.go:195] Run: sudo systemctl restart docker I0131 13:42:26.219399 230266 ssh_runner.go:235] Completed: sudo systemctl restart docker: (2.277156856s) I0131 13:42:26.219547 230266 ssh_runner.go:195] Run: docker images --format {{.Repository}}:{{.Tag}} I0131 13:42:26.233034 230266 docker.go:671] Got preloaded images: -- stdout -- registry.k8s.io/kube-apiserver:v1.28.3 registry.k8s.io/kube-controller-manager:v1.28.3 registry.k8s.io/kube-scheduler:v1.28.3 registry.k8s.io/kube-proxy:v1.28.3 registry.k8s.io/etcd:3.5.9-0 registry.k8s.io/coredns/coredns:v1.10.1 registry.k8s.io/pause:3.9 gcr.io/k8s-minikube/storage-provisioner:v5 -- /stdout -- I0131 13:42:26.233069 230266 cache_images.go:84] Images are preloaded, skipping loading I0131 13:42:26.233141 230266 ssh_runner.go:195] Run: docker info --format {{.CgroupDriver}} I0131 13:42:26.255847 230266 cni.go:84] Creating CNI manager for "" I0131 13:42:26.255868 230266 cni.go:136] 1 nodes found, recommending kindnet I0131 13:42:26.255893 230266 kubeadm.go:87] Using pod CIDR: 10.244.0.0/16 I0131 13:42:26.255919 230266 kubeadm.go:176] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:10.0.2.15 APIServerPort:8443 KubernetesVersion:v1.28.3 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:minikube NodeName:minikube DNSDomain:cluster.local CRISocket:/var/run/cri-dockerd.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "10.0.2.15"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:10.0.2.15 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt StaticPodPath:/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[hairpinMode:hairpin-veth runtimeRequestTimeout:15m] PrependCriSocketUnix:true} I0131 13:42:26.256087 230266 kubeadm.go:181] kubeadm config: apiVersion: kubeadm.k8s.io/v1beta3 kind: InitConfiguration localAPIEndpoint: advertiseAddress: 10.0.2.15 bindPort: 8443 bootstrapTokens: - groups: - system:bootstrappers:kubeadm:default-node-token ttl: 24h0m0s usages: - signing - authentication nodeRegistration: criSocket: unix:///var/run/cri-dockerd.sock name: "minikube" kubeletExtraArgs: node-ip: 10.0.2.15 taints: [] --- apiVersion: kubeadm.k8s.io/v1beta3 kind: ClusterConfiguration apiServer: certSANs: ["127.0.0.1", "localhost", "10.0.2.15"] extraArgs: enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota" controllerManager: extraArgs: allocate-node-cidrs: "true" leader-elect: "false" scheduler: extraArgs: leader-elect: "false" certificatesDir: /var/lib/minikube/certs clusterName: mk controlPlaneEndpoint: control-plane.minikube.internal:8443 etcd: local: dataDir: /var/lib/minikube/etcd extraArgs: proxy-refresh-interval: "70000" kubernetesVersion: v1.28.3 networking: dnsDomain: cluster.local podSubnet: "10.244.0.0/16" serviceSubnet: 10.96.0.0/12 --- apiVersion: kubelet.config.k8s.io/v1beta1 kind: KubeletConfiguration authentication: x509: clientCAFile: /var/lib/minikube/certs/ca.crt cgroupDriver: cgroupfs hairpinMode: hairpin-veth runtimeRequestTimeout: 15m clusterDomain: "cluster.local" # disable disk resource management by default imageGCHighThresholdPercent: 100 evictionHard: nodefs.available: "0%!"(MISSING) nodefs.inodesFree: "0%!"(MISSING) imagefs.available: "0%!"(MISSING) failSwapOn: false staticPodPath: /etc/kubernetes/manifests --- apiVersion: kubeproxy.config.k8s.io/v1alpha1 kind: KubeProxyConfiguration clusterCIDR: "10.244.0.0/16" metricsBindAddress: 0.0.0.0:10249 conntrack: maxPerCore: 0 # Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established" tcpEstablishedTimeout: 0s # Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close" tcpCloseWaitTimeout: 0s I0131 13:42:26.256239 230266 kubeadm.go:976] kubelet [Unit] Wants=docker.socket [Service] ExecStart= ExecStart=/var/lib/minikube/binaries/v1.28.3/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --container-runtime-endpoint=unix:///var/run/cri-dockerd.sock --hostname-override=minikube --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=10.0.2.15 [Install] config: {KubernetesVersion:v1.28.3 ClusterName:minikube Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} I0131 13:42:26.256336 230266 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.28.3 I0131 13:42:26.263976 230266 binaries.go:44] Found k8s binaries, skipping transfer I0131 13:42:26.264097 230266 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube I0131 13:42:26.271414 230266 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (366 bytes) I0131 13:42:26.285685 230266 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes) I0131 13:42:26.298359 230266 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2082 bytes) I0131 13:42:26.310000 230266 ssh_runner.go:195] Run: grep 10.0.2.15 control-plane.minikube.internal$ /etc/hosts I0131 13:42:26.312419 230266 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "10.0.2.15 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts"" I0131 13:42:26.321320 230266 certs.go:56] Setting up /home/caerulescens/.minikube/profiles/minikube for IP: 10.0.2.15 I0131 13:42:26.321467 230266 certs.go:190] acquiring lock for shared ca certs: {Name:mk64b385ec688d7401f4709104c32dac3b04a23d Clock:{} Delay:500ms Timeout:1m0s Cancel:} I0131 13:42:26.321575 230266 certs.go:204] generating minikubeCA CA: /home/caerulescens/.minikube/ca.key I0131 13:42:26.454703 230266 crypto.go:156] Writing cert to /home/caerulescens/.minikube/ca.crt ... I0131 13:42:26.454723 230266 lock.go:35] WriteFile acquiring /home/caerulescens/.minikube/ca.crt: {Name:mkea392895a8bb3bc0ee1e2c8902e4da1727c85f Clock:{} Delay:500ms Timeout:1m0s Cancel:} I0131 13:42:26.454855 230266 crypto.go:164] Writing key to /home/caerulescens/.minikube/ca.key ... I0131 13:42:26.454864 230266 lock.go:35] WriteFile acquiring /home/caerulescens/.minikube/ca.key: {Name:mk3596b46616e55e3ac9aa9af1007cbb4b1cad1f Clock:{} Delay:500ms Timeout:1m0s Cancel:} I0131 13:42:26.454916 230266 certs.go:204] generating proxyClientCA CA: /home/caerulescens/.minikube/proxy-client-ca.key I0131 13:42:26.541269 230266 crypto.go:156] Writing cert to /home/caerulescens/.minikube/proxy-client-ca.crt ... I0131 13:42:26.541281 230266 lock.go:35] WriteFile acquiring /home/caerulescens/.minikube/proxy-client-ca.crt: {Name:mkb6e88edd27b248d7348503c93f5f9ad6fbfe43 Clock:{} Delay:500ms Timeout:1m0s Cancel:} I0131 13:42:26.541356 230266 crypto.go:164] Writing key to /home/caerulescens/.minikube/proxy-client-ca.key ... I0131 13:42:26.541365 230266 lock.go:35] WriteFile acquiring /home/caerulescens/.minikube/proxy-client-ca.key: {Name:mk1395edbb3c953f2ae8b6c386a19e7f024b8b1f Clock:{} Delay:500ms Timeout:1m0s Cancel:} I0131 13:42:26.541439 230266 certs.go:319] generating minikube-user signed cert: /home/caerulescens/.minikube/profiles/minikube/client.key I0131 13:42:26.541450 230266 crypto.go:68] Generating cert /home/caerulescens/.minikube/profiles/minikube/client.crt with IP's: [] I0131 13:42:26.652225 230266 crypto.go:156] Writing cert to /home/caerulescens/.minikube/profiles/minikube/client.crt ... I0131 13:42:26.652251 230266 lock.go:35] WriteFile acquiring /home/caerulescens/.minikube/profiles/minikube/client.crt: {Name:mka910e33bfcf0286bb5219c275f2bc84005f8e2 Clock:{} Delay:500ms Timeout:1m0s Cancel:} I0131 13:42:26.652362 230266 crypto.go:164] Writing key to /home/caerulescens/.minikube/profiles/minikube/client.key ... I0131 13:42:26.652385 230266 lock.go:35] WriteFile acquiring /home/caerulescens/.minikube/profiles/minikube/client.key: {Name:mk934c30dfef619fa88fad0531fe4872d903d0d5 Clock:{} Delay:500ms Timeout:1m0s Cancel:} I0131 13:42:26.652437 230266 certs.go:319] generating minikube signed cert: /home/caerulescens/.minikube/profiles/minikube/apiserver.key.49504c3e I0131 13:42:26.652455 230266 crypto.go:68] Generating cert /home/caerulescens/.minikube/profiles/minikube/apiserver.crt.49504c3e with IP's: [10.0.2.15 10.96.0.1 127.0.0.1 10.0.0.1] I0131 13:42:26.758707 230266 crypto.go:156] Writing cert to /home/caerulescens/.minikube/profiles/minikube/apiserver.crt.49504c3e ... I0131 13:42:26.758723 230266 lock.go:35] WriteFile acquiring /home/caerulescens/.minikube/profiles/minikube/apiserver.crt.49504c3e: {Name:mk9efce07b4d27f85d20bf0f4ae78c18c70e78c8 Clock:{} Delay:500ms Timeout:1m0s Cancel:} I0131 13:42:26.758822 230266 crypto.go:164] Writing key to /home/caerulescens/.minikube/profiles/minikube/apiserver.key.49504c3e ... I0131 13:42:26.758831 230266 lock.go:35] WriteFile acquiring /home/caerulescens/.minikube/profiles/minikube/apiserver.key.49504c3e: {Name:mk92bbe8d42e10892bf6f7ff921cc17b6aab0ae2 Clock:{} Delay:500ms Timeout:1m0s Cancel:} I0131 13:42:26.758878 230266 certs.go:337] copying /home/caerulescens/.minikube/profiles/minikube/apiserver.crt.49504c3e -> /home/caerulescens/.minikube/profiles/minikube/apiserver.crt I0131 13:42:26.758936 230266 certs.go:341] copying /home/caerulescens/.minikube/profiles/minikube/apiserver.key.49504c3e -> /home/caerulescens/.minikube/profiles/minikube/apiserver.key I0131 13:42:26.758980 230266 certs.go:319] generating aggregator signed cert: /home/caerulescens/.minikube/profiles/minikube/proxy-client.key I0131 13:42:26.758996 230266 crypto.go:68] Generating cert /home/caerulescens/.minikube/profiles/minikube/proxy-client.crt with IP's: [] I0131 13:42:26.984423 230266 crypto.go:156] Writing cert to /home/caerulescens/.minikube/profiles/minikube/proxy-client.crt ... I0131 13:42:26.984450 230266 lock.go:35] WriteFile acquiring /home/caerulescens/.minikube/profiles/minikube/proxy-client.crt: {Name:mk86a56137ce970ae22a6a1b3bad3a378cb36cc1 Clock:{} Delay:500ms Timeout:1m0s Cancel:} I0131 13:42:26.984564 230266 crypto.go:164] Writing key to /home/caerulescens/.minikube/profiles/minikube/proxy-client.key ... I0131 13:42:26.984573 230266 lock.go:35] WriteFile acquiring /home/caerulescens/.minikube/profiles/minikube/proxy-client.key: {Name:mk498e30f07b2cd7bc0c33a762e376a56a700723 Clock:{} Delay:500ms Timeout:1m0s Cancel:} I0131 13:42:26.984713 230266 certs.go:437] found cert: /home/caerulescens/.minikube/certs/home/caerulescens/.minikube/certs/ca-key.pem (1675 bytes) I0131 13:42:26.984744 230266 certs.go:437] found cert: /home/caerulescens/.minikube/certs/home/caerulescens/.minikube/certs/ca.pem (1094 bytes) I0131 13:42:26.984767 230266 certs.go:437] found cert: /home/caerulescens/.minikube/certs/home/caerulescens/.minikube/certs/cert.pem (1135 bytes) I0131 13:42:26.984789 230266 certs.go:437] found cert: /home/caerulescens/.minikube/certs/home/caerulescens/.minikube/certs/key.pem (1675 bytes) I0131 13:42:26.985215 230266 ssh_runner.go:362] scp /home/caerulescens/.minikube/profiles/minikube/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1399 bytes) I0131 13:42:27.006566 230266 ssh_runner.go:362] scp /home/caerulescens/.minikube/profiles/minikube/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1679 bytes) I0131 13:42:27.026600 230266 ssh_runner.go:362] scp /home/caerulescens/.minikube/profiles/minikube/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes) I0131 13:42:27.046357 230266 ssh_runner.go:362] scp /home/caerulescens/.minikube/profiles/minikube/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1675 bytes) I0131 13:42:27.066274 230266 ssh_runner.go:362] scp /home/caerulescens/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes) I0131 13:42:27.087673 230266 ssh_runner.go:362] scp /home/caerulescens/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1675 bytes) I0131 13:42:27.108129 230266 ssh_runner.go:362] scp /home/caerulescens/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes) I0131 13:42:27.128632 230266 ssh_runner.go:362] scp /home/caerulescens/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1675 bytes) I0131 13:42:27.149119 230266 ssh_runner.go:362] scp /home/caerulescens/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes) I0131 13:42:27.169306 230266 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes) I0131 13:42:27.184166 230266 ssh_runner.go:195] Run: openssl version I0131 13:42:27.189480 230266 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem" I0131 13:42:27.197885 230266 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem I0131 13:42:27.202258 230266 certs.go:480] hashing: -rw-r--r-- 1 root root 1111 Jan 31 18:42 /usr/share/ca-certificates/minikubeCA.pem I0131 13:42:27.202292 230266 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem I0131 13:42:27.208553 230266 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0" I0131 13:42:27.217421 230266 ssh_runner.go:195] Run: ls /var/lib/minikube/certs/etcd I0131 13:42:27.220626 230266 certs.go:353] certs directory doesn't exist, likely first start: ls /var/lib/minikube/certs/etcd: Process exited with status 2 stdout: stderr: ls: cannot access '/var/lib/minikube/certs/etcd': No such file or directory I0131 13:42:27.220677 230266 kubeadm.go:404] StartCluster: {Name:minikube KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube/iso/minikube-v1.32.1-amd64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.42@sha256:d35ac07dfda971cabee05e0deca8aeac772f885a5348e1a0c0b0a36db20fcfc0 Memory:5333 CPUs:2 DiskSize:20000 VMDriver: Driver:qemu2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:39577 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.28.3 ClusterName:minikube Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP:10.0.2.15 Port:8443 KubernetesVersion:v1.28.3 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop: ExposedPorts:[] ListenAddress: Network:builtin Subnet: MultiNodeRequested:true ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/caerulescens:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 AutoPauseInterval:1m0s GPUs:} I0131 13:42:27.220815 230266 ssh_runner.go:195] Run: docker ps --filter status=paused --filter=name=k8s_.*_(kube-system)_ --format={{.ID}} I0131 13:42:27.237359 230266 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd I0131 13:42:27.243088 230266 ssh_runner.go:195] Run: sudo cp /var/tmp/minikube/kubeadm.yaml.new /var/tmp/minikube/kubeadm.yaml I0131 13:42:27.248449 230266 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf I0131 13:42:27.253500 230266 kubeadm.go:152] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2 stdout: stderr: ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory I0131 13:42:27.253538 230266 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.28.3:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem" I0131 13:42:27.286096 230266 kubeadm.go:322] [init] Using Kubernetes version: v1.28.3 I0131 13:42:27.286218 230266 kubeadm.go:322] [preflight] Running pre-flight checks I0131 13:42:27.395705 230266 kubeadm.go:322] [preflight] Pulling images required for setting up a Kubernetes cluster I0131 13:42:27.395813 230266 kubeadm.go:322] [preflight] This might take a minute or two, depending on the speed of your internet connection I0131 13:42:27.395913 230266 kubeadm.go:322] [preflight] You can also perform this action in beforehand using 'kubeadm config images pull' I0131 13:42:27.599563 230266 kubeadm.go:322] [certs] Using certificateDir folder "/var/lib/minikube/certs" I0131 13:42:27.608869 230266 out.go:204] โ–ช Generating certificates and keys ... I0131 13:42:27.608985 230266 kubeadm.go:322] [certs] Using existing ca certificate authority I0131 13:42:27.609138 230266 kubeadm.go:322] [certs] Using existing apiserver certificate and key on disk I0131 13:42:27.805065 230266 kubeadm.go:322] [certs] Generating "apiserver-kubelet-client" certificate and key I0131 13:42:27.894513 230266 kubeadm.go:322] [certs] Generating "front-proxy-ca" certificate and key I0131 13:42:27.994354 230266 kubeadm.go:322] [certs] Generating "front-proxy-client" certificate and key I0131 13:42:28.330797 230266 kubeadm.go:322] [certs] Generating "etcd/ca" certificate and key I0131 13:42:28.498048 230266 kubeadm.go:322] [certs] Generating "etcd/server" certificate and key I0131 13:42:28.498200 230266 kubeadm.go:322] [certs] etcd/server serving cert is signed for DNS names [localhost minikube] and IPs [10.0.2.15 127.0.0.1 ::1] I0131 13:42:28.666055 230266 kubeadm.go:322] [certs] Generating "etcd/peer" certificate and key I0131 13:42:28.666208 230266 kubeadm.go:322] [certs] etcd/peer serving cert is signed for DNS names [localhost minikube] and IPs [10.0.2.15 127.0.0.1 ::1] I0131 13:42:28.756912 230266 kubeadm.go:322] [certs] Generating "etcd/healthcheck-client" certificate and key I0131 13:42:28.876097 230266 kubeadm.go:322] [certs] Generating "apiserver-etcd-client" certificate and key I0131 13:42:29.015794 230266 kubeadm.go:322] [certs] Generating "sa" key and public key I0131 13:42:29.015904 230266 kubeadm.go:322] [kubeconfig] Using kubeconfig folder "/etc/kubernetes" I0131 13:42:29.155598 230266 kubeadm.go:322] [kubeconfig] Writing "admin.conf" kubeconfig file I0131 13:42:29.293193 230266 kubeadm.go:322] [kubeconfig] Writing "kubelet.conf" kubeconfig file I0131 13:42:29.505646 230266 kubeadm.go:322] [kubeconfig] Writing "controller-manager.conf" kubeconfig file I0131 13:42:29.671043 230266 kubeadm.go:322] [kubeconfig] Writing "scheduler.conf" kubeconfig file I0131 13:42:29.671181 230266 kubeadm.go:322] [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests" I0131 13:42:29.673548 230266 kubeadm.go:322] [control-plane] Using manifest folder "/etc/kubernetes/manifests" I0131 13:42:29.690969 230266 out.go:204] โ–ช Booting up control plane ... I0131 13:42:29.691155 230266 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-apiserver" I0131 13:42:29.691315 230266 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-controller-manager" I0131 13:42:29.691397 230266 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-scheduler" I0131 13:42:29.691502 230266 kubeadm.go:322] [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env" I0131 13:42:29.691673 230266 kubeadm.go:322] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml" I0131 13:42:29.691754 230266 kubeadm.go:322] [kubelet-start] Starting the kubelet I0131 13:42:29.785243 230266 kubeadm.go:322] [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s I0131 13:42:36.286620 230266 kubeadm.go:322] [apiclient] All control plane components are healthy after 6.503804 seconds I0131 13:42:36.286818 230266 kubeadm.go:322] [upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace I0131 13:42:36.298906 230266 kubeadm.go:322] [kubelet] Creating a ConfigMap "kubelet-config" in namespace kube-system with the configuration for the kubelets in the cluster I0131 13:42:36.850380 230266 kubeadm.go:322] [upload-certs] Skipping phase. Please see --upload-certs I0131 13:42:36.850641 230266 kubeadm.go:322] [mark-control-plane] Marking the node minikube as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers] I0131 13:42:37.372052 230266 kubeadm.go:322] [bootstrap-token] Using token: utp5t5.x63wsyb5tzt18wfl I0131 13:42:37.375397 230266 out.go:204] โ–ช Configuring RBAC rules ... I0131 13:42:37.375573 230266 kubeadm.go:322] [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles I0131 13:42:37.385847 230266 kubeadm.go:322] [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes I0131 13:42:37.400989 230266 kubeadm.go:322] [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials I0131 13:42:37.406173 230266 kubeadm.go:322] [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token I0131 13:42:37.410871 230266 kubeadm.go:322] [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster I0131 13:42:37.415533 230266 kubeadm.go:322] [bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace I0131 13:42:37.436442 230266 kubeadm.go:322] [kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key I0131 13:42:37.650739 230266 kubeadm.go:322] [addons] Applied essential addon: CoreDNS I0131 13:42:37.793386 230266 kubeadm.go:322] [addons] Applied essential addon: kube-proxy I0131 13:42:37.794828 230266 kubeadm.go:322] I0131 13:42:37.794941 230266 kubeadm.go:322] Your Kubernetes control-plane has initialized successfully! I0131 13:42:37.794955 230266 kubeadm.go:322] I0131 13:42:37.795106 230266 kubeadm.go:322] To start using your cluster, you need to run the following as a regular user: I0131 13:42:37.795133 230266 kubeadm.go:322] I0131 13:42:37.795196 230266 kubeadm.go:322] mkdir -p $HOME/.kube I0131 13:42:37.795310 230266 kubeadm.go:322] sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config I0131 13:42:37.795403 230266 kubeadm.go:322] sudo chown $(id -u):$(id -g) $HOME/.kube/config I0131 13:42:37.795416 230266 kubeadm.go:322] I0131 13:42:37.795517 230266 kubeadm.go:322] Alternatively, if you are the root user, you can run: I0131 13:42:37.795532 230266 kubeadm.go:322] I0131 13:42:37.795616 230266 kubeadm.go:322] export KUBECONFIG=/etc/kubernetes/admin.conf I0131 13:42:37.795632 230266 kubeadm.go:322] I0131 13:42:37.795737 230266 kubeadm.go:322] You should now deploy a pod network to the cluster. I0131 13:42:37.795886 230266 kubeadm.go:322] Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at: I0131 13:42:37.796035 230266 kubeadm.go:322] https://kubernetes.io/docs/concepts/cluster-administration/addons/ I0131 13:42:37.796089 230266 kubeadm.go:322] I0131 13:42:37.796255 230266 kubeadm.go:322] You can now join any number of control-plane nodes by copying certificate authorities I0131 13:42:37.796405 230266 kubeadm.go:322] and service account keys on each node and then running the following as root: I0131 13:42:37.796422 230266 kubeadm.go:322] I0131 13:42:37.796597 230266 kubeadm.go:322] kubeadm join control-plane.minikube.internal:8443 --token utp5t5.x63wsyb5tzt18wfl \ I0131 13:42:37.796800 230266 kubeadm.go:322] --discovery-token-ca-cert-hash sha256:099eb32cca9e38030398eb0f9f7ebe0e851b9cef71131d1fb16a522f99b41bce \ I0131 13:42:37.796842 230266 kubeadm.go:322] --control-plane I0131 13:42:37.796853 230266 kubeadm.go:322] I0131 13:42:37.797003 230266 kubeadm.go:322] Then you can join any number of worker nodes by running the following on each as root: I0131 13:42:37.797019 230266 kubeadm.go:322] I0131 13:42:37.797167 230266 kubeadm.go:322] kubeadm join control-plane.minikube.internal:8443 --token utp5t5.x63wsyb5tzt18wfl \ I0131 13:42:37.797360 230266 kubeadm.go:322] --discovery-token-ca-cert-hash sha256:099eb32cca9e38030398eb0f9f7ebe0e851b9cef71131d1fb16a522f99b41bce I0131 13:42:37.798024 230266 kubeadm.go:322] [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service' I0131 13:42:37.798061 230266 cni.go:84] Creating CNI manager for "" I0131 13:42:37.798080 230266 cni.go:136] 1 nodes found, recommending kindnet I0131 13:42:37.801305 230266 out.go:177] ๐Ÿ”— Configuring CNI (Container Networking Interface) ... I0131 13:42:37.804309 230266 ssh_runner.go:195] Run: stat /opt/cni/bin/portmap I0131 13:42:37.808352 230266 cni.go:182] applying CNI manifest using /var/lib/minikube/binaries/v1.28.3/kubectl ... I0131 13:42:37.808371 230266 ssh_runner.go:362] scp memory --> /var/tmp/minikube/cni.yaml (2438 bytes) I0131 13:42:37.831781 230266 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.3/kubectl apply --kubeconfig=/var/lib/minikube/kubeconfig -f /var/tmp/minikube/cni.yaml I0131 13:42:38.480208 230266 ssh_runner.go:195] Run: /bin/bash -c "cat /proc/$(pgrep kube-apiserver)/oom_adj" I0131 13:42:38.480331 230266 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.3/kubectl create clusterrolebinding minikube-rbac --clusterrole=cluster-admin --serviceaccount=kube-system:default --kubeconfig=/var/lib/minikube/kubeconfig I0131 13:42:38.480334 230266 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.3/kubectl label nodes minikube.k8s.io/version=v1.32.0 minikube.k8s.io/commit=8220a6eb95f0a4d75f7f2d7b14cef975f050512d minikube.k8s.io/name=minikube minikube.k8s.io/updated_at=2024_01_31T13_42_38_0700 minikube.k8s.io/primary=true --all --overwrite --kubeconfig=/var/lib/minikube/kubeconfig I0131 13:42:38.493202 230266 ops.go:34] apiserver oom_adj: -16 I0131 13:42:38.575050 230266 kubeadm.go:1081] duration metric: took 94.827642ms to wait for elevateKubeSystemPrivileges. I0131 13:42:38.615046 230266 host.go:66] Checking if "minikube" exists ... I0131 13:42:38.616253 230266 main.go:141] libmachine: Using SSH client type: external I0131 13:42:38.616298 230266 main.go:141] libmachine: Using SSH private key: /home/caerulescens/.minikube/machines/minikube/id_rsa (-rw-------) I0131 13:42:38.616359 230266 main.go:141] libmachine: &{[-F /dev/null -o ConnectionAttempts=3 -o ConnectTimeout=10 -o ControlMaster=no -o ControlPath=none -o LogLevel=quiet -o PasswordAuthentication=no -o ServerAliveInterval=60 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null docker@localhost -o IdentitiesOnly=yes -i /home/caerulescens/.minikube/machines/minikube/id_rsa -p 39193] /usr/bin/ssh } I0131 13:42:38.616402 230266 main.go:141] libmachine: /usr/bin/ssh -F /dev/null -o ConnectionAttempts=3 -o ConnectTimeout=10 -o ControlMaster=no -o ControlPath=none -o LogLevel=quiet -o PasswordAuthentication=no -o ServerAliveInterval=60 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null docker@localhost -o IdentitiesOnly=yes -i /home/caerulescens/.minikube/machines/minikube/id_rsa -p 39193 -f -NTL 39577:localhost:8443 I0131 13:42:38.703835 230266 kubeadm.go:406] StartCluster complete in 11.48314366s I0131 13:42:38.703880 230266 settings.go:142] acquiring lock: {Name:mk8d27447c1c9e545903b1773b29083297adc301 Clock:{} Delay:500ms Timeout:1m0s Cancel:} I0131 13:42:38.703957 230266 settings.go:150] Updating kubeconfig: /home/caerulescens/.kube/config I0131 13:42:38.704786 230266 lock.go:35] WriteFile acquiring /home/caerulescens/.kube/config: {Name:mkd625f49e00c6b458a797492be28d65bff7f752 Clock:{} Delay:500ms Timeout:1m0s Cancel:} I0131 13:42:38.705047 230266 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.28.3/kubectl --kubeconfig=/var/lib/minikube/kubeconfig -n kube-system get configmap coredns -o yaml" I0131 13:42:38.705080 230266 addons.go:499] enable addons start: toEnable=map[ambassador:false auto-pause:false cloud-spanner:false csi-hostpath-driver:false dashboard:false default-storageclass:true efk:false freshpod:false gcp-auth:false gvisor:false headlamp:false helm-tiller:false inaccel:false ingress:false ingress-dns:false inspektor-gadget:false istio:false istio-provisioner:false kong:false kubeflow:false kubevirt:false logviewer:false metallb:false metrics-server:false nvidia-device-plugin:false nvidia-driver-installer:false nvidia-gpu-device-plugin:false olm:false pod-security-policy:false portainer:false registry:false registry-aliases:false registry-creds:false storage-provisioner:true storage-provisioner-gluster:false storage-provisioner-rancher:false volumesnapshots:false] I0131 13:42:38.705185 230266 addons.go:69] Setting storage-provisioner=true in profile "minikube" I0131 13:42:38.705218 230266 addons.go:231] Setting addon storage-provisioner=true in "minikube" I0131 13:42:38.705282 230266 host.go:66] Checking if "minikube" exists ... I0131 13:42:38.705211 230266 addons.go:69] Setting default-storageclass=true in profile "minikube" I0131 13:42:38.705311 230266 config.go:182] Loaded profile config "minikube": Driver=qemu2, ContainerRuntime=docker, KubernetesVersion=v1.28.3 I0131 13:42:38.705326 230266 addons_storage_classes.go:33] enableOrDisableStorageClasses default-storageclass=true on "minikube" I0131 13:42:38.712200 230266 out.go:177] โ–ช Using image gcr.io/k8s-minikube/storage-provisioner:v5 I0131 13:42:38.707797 230266 addons.go:231] Setting addon default-storageclass=true in "minikube" I0131 13:42:38.712278 230266 host.go:66] Checking if "minikube" exists ... I0131 13:42:38.713665 230266 addons.go:423] installing /etc/kubernetes/addons/storageclass.yaml I0131 13:42:38.716018 230266 addons.go:423] installing /etc/kubernetes/addons/storage-provisioner.yaml I0131 13:42:38.716026 230266 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storageclass.yaml (271 bytes) I0131 13:42:38.716041 230266 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storage-provisioner.yaml (2676 bytes) I0131 13:42:38.716059 230266 sshutil.go:53] new ssh client: &{IP:localhost Port:39193 SSHKeyPath:/home/caerulescens/.minikube/machines/minikube/id_rsa Username:docker} I0131 13:42:38.716069 230266 sshutil.go:53] new ssh client: &{IP:localhost Port:39193 SSHKeyPath:/home/caerulescens/.minikube/machines/minikube/id_rsa Username:docker} I0131 13:42:38.731009 230266 kapi.go:248] "coredns" deployment in "kube-system" namespace and "minikube" context rescaled to 1 replicas I0131 13:42:38.731051 230266 start.go:223] Will wait 6m0s for node &{Name: IP:10.0.2.15 Port:8443 KubernetesVersion:v1.28.3 ContainerRuntime:docker ControlPlane:true Worker:true} I0131 13:42:38.734185 230266 out.go:177] ๐Ÿ”Ž Verifying Kubernetes components... I0131 13:42:38.739650 230266 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service kubelet I0131 13:42:38.786414 230266 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.3/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml I0131 13:42:38.815019 230266 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.3/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml I0131 13:42:38.831739 230266 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.28.3/kubectl --kubeconfig=/var/lib/minikube/kubeconfig -n kube-system get configmap coredns -o yaml | sed -e '/^ forward . \/etc\/resolv.conf.*/i \ hosts {\n 10.0.2.2 host.minikube.internal\n fallthrough\n }' -e '/^ errors *$/i \ log' | sudo /var/lib/minikube/binaries/v1.28.3/kubectl --kubeconfig=/var/lib/minikube/kubeconfig replace -f -" I0131 13:42:38.832637 230266 api_server.go:52] waiting for apiserver process to appear ... I0131 13:42:38.832720 230266 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.* I0131 13:42:39.352267 230266 out.go:177] ๐ŸŒŸ Enabled addons: default-storageclass, storage-provisioner I0131 13:42:39.348879 230266 start.go:926] {"host.minikube.internal": 10.0.2.2} host record injected into CoreDNS's ConfigMap I0131 13:42:39.348961 230266 api_server.go:72] duration metric: took 617.871953ms to wait for apiserver process to appear ... I0131 13:42:39.358199 230266 api_server.go:88] waiting for apiserver healthz status ... I0131 13:42:39.358195 230266 addons.go:502] enable addons completed in 653.127866ms: enabled=[default-storageclass storage-provisioner] I0131 13:42:39.358224 230266 api_server.go:253] Checking apiserver healthz at https://localhost:39577/healthz ... I0131 13:42:39.368146 230266 api_server.go:279] https://localhost:39577/healthz returned 200: ok I0131 13:42:39.371438 230266 api_server.go:141] control plane version: v1.28.3 I0131 13:42:39.371457 230266 api_server.go:131] duration metric: took 13.245913ms to wait for apiserver health ... I0131 13:42:39.371467 230266 system_pods.go:43] waiting for kube-system pods to appear ... I0131 13:42:39.377998 230266 system_pods.go:59] 5 kube-system pods found I0131 13:42:39.378035 230266 system_pods.go:61] "etcd-minikube" [fbe0ba23-64a2-4f08-a61e-27a53a15a278] Running / Ready:ContainersNotReady (containers with unready status: [etcd]) / ContainersReady:ContainersNotReady (containers with unready status: [etcd]) I0131 13:42:39.378049 230266 system_pods.go:61] "kube-apiserver-minikube" [dd8c4b17-9c77-4ab4-b4e0-a793f9808864] Running / Ready:ContainersNotReady (containers with unready status: [kube-apiserver]) / ContainersReady:ContainersNotReady (containers with unready status: [kube-apiserver]) I0131 13:42:39.378067 230266 system_pods.go:61] "kube-controller-manager-minikube" [22ff3576-ec42-49f2-aa7e-21d660e5fd16] Running / Ready:ContainersNotReady (containers with unready status: [kube-controller-manager]) / ContainersReady:ContainersNotReady (containers with unready status: [kube-controller-manager]) I0131 13:42:39.378083 230266 system_pods.go:61] "kube-scheduler-minikube" [4e958c40-4382-40b4-9408-7094b5c68adf] Running / Ready:ContainersNotReady (containers with unready status: [kube-scheduler]) / ContainersReady:ContainersNotReady (containers with unready status: [kube-scheduler]) I0131 13:42:39.378098 230266 system_pods.go:61] "storage-provisioner" [f7d2e208-d1d7-408d-b109-7655dc682b7a] Pending: PodScheduled:Unschedulable (0/1 nodes are available: 1 node(s) had untolerated taint {node.kubernetes.io/not-ready: }. preemption: 0/1 nodes are available: 1 Preemption is not helpful for scheduling..) I0131 13:42:39.378114 230266 system_pods.go:74] duration metric: took 6.63854ms to wait for pod list to return data ... I0131 13:42:39.378131 230266 kubeadm.go:581] duration metric: took 647.048319ms to wait for : map[apiserver:true system_pods:true] ... I0131 13:42:39.378151 230266 node_conditions.go:102] verifying NodePressure condition ... I0131 13:42:39.382679 230266 node_conditions.go:122] node storage ephemeral capacity is 17784760Ki I0131 13:42:39.382708 230266 node_conditions.go:123] node cpu capacity is 2 I0131 13:42:39.382724 230266 node_conditions.go:105] duration metric: took 4.563106ms to run NodePressure ... I0131 13:42:39.382744 230266 start.go:228] waiting for startup goroutines ... I0131 13:42:39.382761 230266 start.go:233] waiting for cluster config update ... I0131 13:42:39.382777 230266 start.go:242] writing updated cluster config ... I0131 13:42:39.386189 230266 out.go:177] I0131 13:42:39.389483 230266 config.go:182] Loaded profile config "minikube": Driver=qemu2, ContainerRuntime=docker, KubernetesVersion=v1.28.3 I0131 13:42:39.389591 230266 profile.go:148] Saving config to /home/caerulescens/.minikube/profiles/minikube/config.json ... I0131 13:42:39.393150 230266 out.go:177] ๐Ÿ‘ Starting worker node minikube-m02 in cluster minikube I0131 13:42:39.396097 230266 preload.go:132] Checking if preload exists for k8s version v1.28.3 and runtime docker I0131 13:42:39.396124 230266 cache.go:56] Caching tarball of preloaded images I0131 13:42:39.396229 230266 preload.go:174] Found /home/caerulescens/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.3-docker-overlay2-amd64.tar.lz4 in cache, skipping download I0131 13:42:39.396246 230266 cache.go:59] Finished verifying existence of preloaded tar for v1.28.3 on docker I0131 13:42:39.396340 230266 profile.go:148] Saving config to /home/caerulescens/.minikube/profiles/minikube/config.json ... I0131 13:42:39.396561 230266 start.go:365] acquiring machines lock for minikube-m02: {Name:mk730b7b7d5d10067abf794b343f53c46eaa903c Clock:{} Delay:500ms Timeout:13m0s Cancel:} I0131 13:42:39.396607 230266 start.go:369] acquired machines lock for "minikube-m02" in 22.863ยตs I0131 13:42:39.396631 230266 start.go:93] Provisioning new machine with config: &{Name:minikube KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube/iso/minikube-v1.32.1-amd64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.42@sha256:d35ac07dfda971cabee05e0deca8aeac772f885a5348e1a0c0b0a36db20fcfc0 Memory:5333 CPUs:2 DiskSize:20000 VMDriver: Driver:qemu2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:39577 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.28.3 ClusterName:minikube Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP:10.0.2.15 Port:8443 KubernetesVersion:v1.28.3 ContainerRuntime:docker ControlPlane:true Worker:true} {Name:m02 IP: Port:0 KubernetesVersion:v1.28.3 ContainerRuntime:docker ControlPlane:false Worker:true}] Addons:map[default-storageclass:true storage-provisioner:true] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop: ExposedPorts:[] ListenAddress: Network:builtin Subnet: MultiNodeRequested:true ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/caerulescens:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 AutoPauseInterval:1m0s GPUs:} &{Name:m02 IP: Port:0 KubernetesVersion:v1.28.3 ContainerRuntime:docker ControlPlane:false Worker:true} I0131 13:42:39.396743 230266 start.go:125] createHost starting for "m02" (driver="qemu2") I0131 13:42:39.400026 230266 out.go:204] ๐Ÿ”ฅ Creating qemu2 VM (CPUs=2, Memory=5333MB, Disk=20000MB) ... I0131 13:42:39.400145 230266 start.go:159] libmachine.API.Create for "minikube" (driver="qemu2") I0131 13:42:39.400199 230266 client.go:168] LocalClient.Create starting I0131 13:42:39.400267 230266 main.go:141] libmachine: Reading certificate data from /home/caerulescens/.minikube/certs/ca.pem I0131 13:42:39.400301 230266 main.go:141] libmachine: Decoding PEM data... I0131 13:42:39.400322 230266 main.go:141] libmachine: Parsing certificate... I0131 13:42:39.400384 230266 main.go:141] libmachine: Reading certificate data from /home/caerulescens/.minikube/certs/cert.pem I0131 13:42:39.400411 230266 main.go:141] libmachine: Decoding PEM data... I0131 13:42:39.400428 230266 main.go:141] libmachine: Parsing certificate... I0131 13:42:39.400641 230266 main.go:141] libmachine: port range: 0 -> 65535 I0131 13:42:39.400824 230266 main.go:141] libmachine: Downloading /home/caerulescens/.minikube/cache/boot2docker.iso from file:///home/caerulescens/.minikube/cache/iso/amd64/minikube-v1.32.1-amd64.iso... I0131 13:42:39.633999 230266 main.go:141] libmachine: Creating SSH key... I0131 13:42:39.689564 230266 main.go:141] libmachine: Creating Disk image... I0131 13:42:39.689575 230266 main.go:141] libmachine: Creating 20000 MB hard disk image... I0131 13:42:39.689700 230266 main.go:141] libmachine: executing: qemu-img convert -f raw -O qcow2 /home/caerulescens/.minikube/machines/minikube-m02/disk.qcow2.raw /home/caerulescens/.minikube/machines/minikube-m02/disk.qcow2 I0131 13:42:39.707307 230266 main.go:141] libmachine: STDOUT: I0131 13:42:39.707318 230266 main.go:141] libmachine: STDERR: I0131 13:42:39.707356 230266 main.go:141] libmachine: executing: qemu-img resize /home/caerulescens/.minikube/machines/minikube-m02/disk.qcow2 +20000M I0131 13:42:39.727101 230266 main.go:141] libmachine: STDOUT: Image resized. I0131 13:42:39.727126 230266 main.go:141] libmachine: STDERR: I0131 13:42:39.727171 230266 main.go:141] libmachine: DONE writing to /home/caerulescens/.minikube/machines/minikube-m02/disk.qcow2.raw and /home/caerulescens/.minikube/machines/minikube-m02/disk.qcow2 I0131 13:42:39.727181 230266 main.go:141] libmachine: Starting QEMU VM... I0131 13:42:39.727270 230266 main.go:141] libmachine: executing: qemu-system-x86_64 -cpu max -display none -accel kvm -m 5333 -smp 2 -boot d -cdrom /home/caerulescens/.minikube/machines/minikube-m02/boot2docker.iso -qmp unix:/home/caerulescens/.minikube/machines/minikube-m02/monitor,server,nowait -pidfile /home/caerulescens/.minikube/machines/minikube-m02/qemu.pid -nic user,model=virtio,hostfwd=tcp::34031-:22,hostfwd=tcp::40015-:2376,hostname=minikube-m02 -daemonize /home/caerulescens/.minikube/machines/minikube-m02/disk.qcow2 I0131 13:42:39.829657 230266 main.go:141] libmachine: STDOUT: I0131 13:42:39.829697 230266 main.go:141] libmachine: STDERR: I0131 13:42:39.829725 230266 main.go:141] libmachine: Waiting for VM to start (ssh -p 34031 docker@127.0.0.1)... I0131 13:43:11.228482 230266 machine.go:88] provisioning docker machine ... I0131 13:43:11.228534 230266 buildroot.go:166] provisioning hostname "minikube-m02" I0131 13:43:11.228621 230266 main.go:141] libmachine: Using SSH client type: external I0131 13:43:11.228654 230266 main.go:141] libmachine: Using SSH private key: /home/caerulescens/.minikube/machines/minikube-m02/id_rsa (-rw-------) I0131 13:43:11.228699 230266 main.go:141] libmachine: &{[-F /dev/null -o ConnectionAttempts=3 -o ConnectTimeout=10 -o ControlMaster=no -o ControlPath=none -o LogLevel=quiet -o PasswordAuthentication=no -o ServerAliveInterval=60 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null docker@localhost -o IdentitiesOnly=yes -i /home/caerulescens/.minikube/machines/minikube-m02/id_rsa -p 34031] /usr/bin/ssh } I0131 13:43:11.228722 230266 main.go:141] libmachine: About to run SSH command: sudo hostname minikube-m02 && echo "minikube-m02" | sudo tee /etc/hostname I0131 13:43:11.333468 230266 main.go:141] libmachine: SSH cmd err, output: : minikube-m02 I0131 13:43:11.333598 230266 main.go:141] libmachine: Using SSH client type: external I0131 13:43:11.333639 230266 main.go:141] libmachine: Using SSH private key: /home/caerulescens/.minikube/machines/minikube-m02/id_rsa (-rw-------) I0131 13:43:11.333693 230266 main.go:141] libmachine: &{[-F /dev/null -o ConnectionAttempts=3 -o ConnectTimeout=10 -o ControlMaster=no -o ControlPath=none -o LogLevel=quiet -o PasswordAuthentication=no -o ServerAliveInterval=60 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null docker@localhost -o IdentitiesOnly=yes -i /home/caerulescens/.minikube/machines/minikube-m02/id_rsa -p 34031] /usr/bin/ssh } I0131 13:43:11.333752 230266 main.go:141] libmachine: About to run SSH command: if ! grep -xq '.*\sminikube-m02' /etc/hosts; then if grep -xq '127.0.1.1\s.*' /etc/hosts; then sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 minikube-m02/g' /etc/hosts; else echo '127.0.1.1 minikube-m02' | sudo tee -a /etc/hosts; fi fi I0131 13:43:11.442113 230266 main.go:141] libmachine: SSH cmd err, output: : I0131 13:43:11.442157 230266 buildroot.go:172] set auth options {CertDir:/home/caerulescens/.minikube CaCertPath:/home/caerulescens/.minikube/certs/ca.pem CaPrivateKeyPath:/home/caerulescens/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/home/caerulescens/.minikube/machines/server.pem ServerKeyPath:/home/caerulescens/.minikube/machines/server-key.pem ClientKeyPath:/home/caerulescens/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/home/caerulescens/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/home/caerulescens/.minikube} I0131 13:43:11.442188 230266 buildroot.go:174] setting up certificates I0131 13:43:11.442205 230266 provision.go:83] configureAuth start I0131 13:43:11.442220 230266 provision.go:138] copyHostCerts I0131 13:43:11.442284 230266 exec_runner.go:144] found /home/caerulescens/.minikube/ca.pem, removing ... I0131 13:43:11.442298 230266 exec_runner.go:203] rm: /home/caerulescens/.minikube/ca.pem I0131 13:43:11.442405 230266 exec_runner.go:151] cp: /home/caerulescens/.minikube/certs/ca.pem --> /home/caerulescens/.minikube/ca.pem (1094 bytes) I0131 13:43:11.442542 230266 exec_runner.go:144] found /home/caerulescens/.minikube/cert.pem, removing ... I0131 13:43:11.442557 230266 exec_runner.go:203] rm: /home/caerulescens/.minikube/cert.pem I0131 13:43:11.442607 230266 exec_runner.go:151] cp: /home/caerulescens/.minikube/certs/cert.pem --> /home/caerulescens/.minikube/cert.pem (1135 bytes) I0131 13:43:11.442705 230266 exec_runner.go:144] found /home/caerulescens/.minikube/key.pem, removing ... I0131 13:43:11.442730 230266 exec_runner.go:203] rm: /home/caerulescens/.minikube/key.pem I0131 13:43:11.442782 230266 exec_runner.go:151] cp: /home/caerulescens/.minikube/certs/key.pem --> /home/caerulescens/.minikube/key.pem (1675 bytes) I0131 13:43:11.442870 230266 provision.go:112] generating server cert: /home/caerulescens/.minikube/machines/server.pem ca-key=/home/caerulescens/.minikube/certs/ca.pem private-key=/home/caerulescens/.minikube/certs/ca-key.pem org=caerulescens.minikube-m02 san=[127.0.0.1 localhost localhost 127.0.0.1 minikube minikube-m02] I0131 13:43:11.566911 230266 provision.go:172] copyRemoteCerts I0131 13:43:11.566973 230266 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker I0131 13:43:11.566989 230266 sshutil.go:53] new ssh client: &{IP:localhost Port:34031 SSHKeyPath:/home/caerulescens/.minikube/machines/minikube-m02/id_rsa Username:docker} I0131 13:43:11.610280 230266 ssh_runner.go:362] scp /home/caerulescens/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1094 bytes) I0131 13:43:11.630718 230266 ssh_runner.go:362] scp /home/caerulescens/.minikube/machines/server.pem --> /etc/docker/server.pem (1233 bytes) I0131 13:43:11.650766 230266 ssh_runner.go:362] scp /home/caerulescens/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1679 bytes) I0131 13:43:11.670735 230266 provision.go:86] duration metric: configureAuth took 228.514504ms I0131 13:43:11.670759 230266 buildroot.go:189] setting minikube options for container-runtime I0131 13:43:11.670969 230266 config.go:182] Loaded profile config "minikube": Driver=qemu2, ContainerRuntime=docker, KubernetesVersion=v1.28.3 I0131 13:43:11.671059 230266 main.go:141] libmachine: Using SSH client type: external I0131 13:43:11.671097 230266 main.go:141] libmachine: Using SSH private key: /home/caerulescens/.minikube/machines/minikube-m02/id_rsa (-rw-------) I0131 13:43:11.671154 230266 main.go:141] libmachine: &{[-F /dev/null -o ConnectionAttempts=3 -o ConnectTimeout=10 -o ControlMaster=no -o ControlPath=none -o LogLevel=quiet -o PasswordAuthentication=no -o ServerAliveInterval=60 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null docker@localhost -o IdentitiesOnly=yes -i /home/caerulescens/.minikube/machines/minikube-m02/id_rsa -p 34031] /usr/bin/ssh } I0131 13:43:11.671178 230266 main.go:141] libmachine: About to run SSH command: df --output=fstype / | tail -n 1 I0131 13:43:11.772867 230266 main.go:141] libmachine: SSH cmd err, output: : tmpfs I0131 13:43:11.772888 230266 buildroot.go:70] root file system type: tmpfs I0131 13:43:11.773018 230266 provision.go:309] Updating docker unit: /lib/systemd/system/docker.service ... I0131 13:43:11.773092 230266 main.go:141] libmachine: Using SSH client type: external I0131 13:43:11.773118 230266 main.go:141] libmachine: Using SSH private key: /home/caerulescens/.minikube/machines/minikube-m02/id_rsa (-rw-------) I0131 13:43:11.773167 230266 main.go:141] libmachine: &{[-F /dev/null -o ConnectionAttempts=3 -o ConnectTimeout=10 -o ControlMaster=no -o ControlPath=none -o LogLevel=quiet -o PasswordAuthentication=no -o ServerAliveInterval=60 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null docker@localhost -o IdentitiesOnly=yes -i /home/caerulescens/.minikube/machines/minikube-m02/id_rsa -p 34031] /usr/bin/ssh } I0131 13:43:11.773294 230266 main.go:141] libmachine: About to run SSH command: sudo mkdir -p /lib/systemd/system && printf %!s(MISSING) "[Unit] Description=Docker Application Container Engine Documentation=https://docs.docker.com After=network.target minikube-automount.service docker.socket Requires= minikube-automount.service docker.socket StartLimitBurst=3 StartLimitIntervalSec=60 [Service] Type=notify Restart=on-failure Environment="NO_PROXY=127.0.0.1" # This file is a systemd drop-in unit that inherits from the base dockerd configuration. # The base configuration already specifies an 'ExecStart=...' command. The first directive # here is to clear out that command inherited from the base configuration. Without this, # the command from the base configuration and the command specified here are treated as # a sequence of commands, which is not the desired behavior, nor is it valid -- systemd # will catch this invalid input and refuse to start the service with an error like: # Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services. # NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other # container runtimes. If left unlimited, it may result in OOM issues with MySQL. ExecStart= ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=qemu2 --insecure-registry 10.96.0.0/12 ExecReload=/bin/kill -s HUP \$MAINPID # Having non-zero Limit*s causes performance problems due to accounting overhead # in the kernel. We recommend using cgroups to do container-local accounting. LimitNOFILE=infinity LimitNPROC=infinity LimitCORE=infinity # Uncomment TasksMax if your systemd version supports it. # Only systemd 226 and above support this version. TasksMax=infinity TimeoutStartSec=0 # set delegate yes so that systemd does not reset the cgroups of docker containers Delegate=yes # kill only the docker process, not all processes in the cgroup KillMode=process [Install] WantedBy=multi-user.target " | sudo tee /lib/systemd/system/docker.service.new I0131 13:43:11.886774 230266 main.go:141] libmachine: SSH cmd err, output: : [Unit] Description=Docker Application Container Engine Documentation=https://docs.docker.com After=network.target minikube-automount.service docker.socket Requires= minikube-automount.service docker.socket StartLimitBurst=3 StartLimitIntervalSec=60 [Service] Type=notify Restart=on-failure Environment=NO_PROXY=127.0.0.1 # This file is a systemd drop-in unit that inherits from the base dockerd configuration. # The base configuration already specifies an 'ExecStart=...' command. The first directive # here is to clear out that command inherited from the base configuration. Without this, # the command from the base configuration and the command specified here are treated as # a sequence of commands, which is not the desired behavior, nor is it valid -- systemd # will catch this invalid input and refuse to start the service with an error like: # Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services. # NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other # container runtimes. If left unlimited, it may result in OOM issues with MySQL. ExecStart= ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=qemu2 --insecure-registry 10.96.0.0/12 ExecReload=/bin/kill -s HUP $MAINPID # Having non-zero Limit*s causes performance problems due to accounting overhead # in the kernel. We recommend using cgroups to do container-local accounting. LimitNOFILE=infinity LimitNPROC=infinity LimitCORE=infinity # Uncomment TasksMax if your systemd version supports it. # Only systemd 226 and above support this version. TasksMax=infinity TimeoutStartSec=0 # set delegate yes so that systemd does not reset the cgroups of docker containers Delegate=yes # kill only the docker process, not all processes in the cgroup KillMode=process [Install] WantedBy=multi-user.target I0131 13:43:11.886888 230266 main.go:141] libmachine: Using SSH client type: external I0131 13:43:11.886926 230266 main.go:141] libmachine: Using SSH private key: /home/caerulescens/.minikube/machines/minikube-m02/id_rsa (-rw-------) I0131 13:43:11.886973 230266 main.go:141] libmachine: &{[-F /dev/null -o ConnectionAttempts=3 -o ConnectTimeout=10 -o ControlMaster=no -o ControlPath=none -o LogLevel=quiet -o PasswordAuthentication=no -o ServerAliveInterval=60 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null docker@localhost -o IdentitiesOnly=yes -i /home/caerulescens/.minikube/machines/minikube-m02/id_rsa -p 34031] /usr/bin/ssh } I0131 13:43:11.886997 230266 main.go:141] libmachine: About to run SSH command: sudo diff -u /lib/systemd/system/docker.service /lib/systemd/system/docker.service.new || { sudo mv /lib/systemd/system/docker.service.new /lib/systemd/system/docker.service; sudo systemctl -f daemon-reload && sudo systemctl -f enable docker && sudo systemctl -f restart docker; } I0131 13:43:12.722554 230266 main.go:141] libmachine: SSH cmd err, output: : diff: can't stat '/lib/systemd/system/docker.service': No such file or directory Created symlink /etc/systemd/system/multi-user.target.wants/docker.service โ†’ /usr/lib/systemd/system/docker.service. I0131 13:43:12.722598 230266 machine.go:91] provisioned docker machine in 1.494093506s I0131 13:43:12.722613 230266 client.go:171] LocalClient.Create took 33.322401881s I0131 13:43:12.722638 230266 start.go:167] duration metric: libmachine.API.Create for "minikube" took 33.32249219s I0131 13:43:12.722651 230266 start.go:300] post-start starting for "minikube-m02" (driver="qemu2") I0131 13:43:12.722666 230266 start.go:329] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs] I0131 13:43:12.722758 230266 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs I0131 13:43:12.722781 230266 sshutil.go:53] new ssh client: &{IP:localhost Port:34031 SSHKeyPath:/home/caerulescens/.minikube/machines/minikube-m02/id_rsa Username:docker} I0131 13:43:12.770732 230266 ssh_runner.go:195] Run: cat /etc/os-release I0131 13:43:12.774146 230266 info.go:137] Remote host: Buildroot 2021.02.12 I0131 13:43:12.774169 230266 filesync.go:126] Scanning /home/caerulescens/.minikube/addons for local assets ... I0131 13:43:12.774254 230266 filesync.go:126] Scanning /home/caerulescens/.minikube/files for local assets ... I0131 13:43:12.774297 230266 start.go:303] post-start completed in 51.634344ms I0131 13:43:12.774763 230266 profile.go:148] Saving config to /home/caerulescens/.minikube/profiles/minikube/config.json ... I0131 13:43:12.774967 230266 start.go:128] duration metric: createHost completed in 33.378208623s I0131 13:43:12.775048 230266 main.go:141] libmachine: Using SSH client type: external I0131 13:43:12.775085 230266 main.go:141] libmachine: Using SSH private key: /home/caerulescens/.minikube/machines/minikube-m02/id_rsa (-rw-------) I0131 13:43:12.775140 230266 main.go:141] libmachine: &{[-F /dev/null -o ConnectionAttempts=3 -o ConnectTimeout=10 -o ControlMaster=no -o ControlPath=none -o LogLevel=quiet -o PasswordAuthentication=no -o ServerAliveInterval=60 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null docker@localhost -o IdentitiesOnly=yes -i /home/caerulescens/.minikube/machines/minikube-m02/id_rsa -p 34031] /usr/bin/ssh } I0131 13:43:12.775161 230266 main.go:141] libmachine: About to run SSH command: date +%!s(MISSING).%!N(MISSING) I0131 13:43:12.876160 230266 main.go:141] libmachine: SSH cmd err, output: : 1706726592.873074915 I0131 13:43:12.876186 230266 fix.go:206] guest clock: 1706726592.873074915 I0131 13:43:12.876202 230266 fix.go:219] Guest: 2024-01-31 13:43:12.873074915 -0500 EST Remote: 2024-01-31 13:43:12.774980753 -0500 EST m=+101.509941965 (delta=98.094162ms) I0131 13:43:12.876231 230266 fix.go:190] guest clock delta is within tolerance: 98.094162ms I0131 13:43:12.876244 230266 start.go:83] releasing machines lock for "minikube-m02", held for 33.479623619s I0131 13:43:12.885482 230266 out.go:177] ๐ŸŒ Found network options: I0131 13:43:12.888824 230266 out.go:177] โ–ช NO_PROXY=127.0.0.1 I0131 13:43:12.892067 230266 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/ I0131 13:43:12.892116 230266 sshutil.go:53] new ssh client: &{IP:localhost Port:34031 SSHKeyPath:/home/caerulescens/.minikube/machines/minikube-m02/id_rsa Username:docker} I0131 13:43:12.892285 230266 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*" I0131 13:43:12.892304 230266 sshutil.go:53] new ssh client: &{IP:localhost Port:34031 SSHKeyPath:/home/caerulescens/.minikube/machines/minikube-m02/id_rsa Username:docker} W0131 13:43:12.995168 230266 cni.go:209] loopback cni configuration skipped: "/etc/cni/net.d/*loopback.conf*" not found I0131 13:43:12.995269 230266 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f ( ( -name *bridge* -or -name *podman* ) -and -not -name *.mk_disabled ) -printf "%!p(MISSING), " -exec sh -c "sudo mv {} {}.mk_disabled" ; I0131 13:43:13.008483 230266 cni.go:262] disabled [/etc/cni/net.d/87-podman-bridge.conflist] bridge cni config(s) I0131 13:43:13.008509 230266 start.go:472] detecting cgroup driver to use... I0131 13:43:13.008652 230266 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %!s(MISSING) "runtime-endpoint: unix:///run/containerd/containerd.sock " | sudo tee /etc/crictl.yaml" I0131 13:43:13.024322 230266 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)sandbox_image = .*$|\1sandbox_image = "registry.k8s.io/pause:3.9"|' /etc/containerd/config.toml" I0131 13:43:13.034124 230266 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)restrict_oom_score_adj = .*$|\1restrict_oom_score_adj = false|' /etc/containerd/config.toml" I0131 13:43:13.044312 230266 containerd.go:145] configuring containerd to use "cgroupfs" as cgroup driver... I0131 13:43:13.044416 230266 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)SystemdCgroup = .*$|\1SystemdCgroup = false|g' /etc/containerd/config.toml" I0131 13:43:13.053762 230266 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runtime.v1.linux"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml" I0131 13:43:13.063421 230266 ssh_runner.go:195] Run: sh -c "sudo sed -i '/systemd_cgroup/d' /etc/containerd/config.toml" I0131 13:43:13.072467 230266 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runc.v1"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml" I0131 13:43:13.081515 230266 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk" I0131 13:43:13.091381 230266 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)conf_dir = .*$|\1conf_dir = "/etc/cni/net.d"|g' /etc/containerd/config.toml" I0131 13:43:13.098617 230266 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables I0131 13:43:13.104079 230266 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward" I0131 13:43:13.109543 230266 ssh_runner.go:195] Run: sudo systemctl daemon-reload I0131 13:43:13.213564 230266 ssh_runner.go:195] Run: sudo systemctl restart containerd I0131 13:43:13.224089 230266 start.go:472] detecting cgroup driver to use... I0131 13:43:13.224190 230266 ssh_runner.go:195] Run: sudo systemctl cat docker.service I0131 13:43:13.237043 230266 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service containerd I0131 13:43:13.245141 230266 ssh_runner.go:195] Run: sudo systemctl stop -f containerd I0131 13:43:13.257817 230266 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service containerd I0131 13:43:13.265735 230266 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio I0131 13:43:13.273100 230266 ssh_runner.go:195] Run: sudo systemctl stop -f crio I0131 13:43:13.325796 230266 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio I0131 13:43:13.338522 230266 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %!s(MISSING) "runtime-endpoint: unix:///var/run/cri-dockerd.sock " | sudo tee /etc/crictl.yaml" I0131 13:43:13.358187 230266 ssh_runner.go:195] Run: which cri-dockerd I0131 13:43:13.362554 230266 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/cri-docker.service.d I0131 13:43:13.370637 230266 ssh_runner.go:362] scp memory --> /etc/systemd/system/cri-docker.service.d/10-cni.conf (189 bytes) I0131 13:43:13.385822 230266 ssh_runner.go:195] Run: sudo systemctl unmask docker.service I0131 13:43:13.501784 230266 ssh_runner.go:195] Run: sudo systemctl enable docker.socket I0131 13:43:13.612931 230266 docker.go:560] configuring docker to use "cgroupfs" as cgroup driver... I0131 13:43:13.612971 230266 ssh_runner.go:362] scp memory --> /etc/docker/daemon.json (130 bytes) I0131 13:43:13.625058 230266 ssh_runner.go:195] Run: sudo systemctl daemon-reload I0131 13:43:13.707057 230266 ssh_runner.go:195] Run: sudo systemctl restart docker I0131 13:43:15.057890 230266 ssh_runner.go:235] Completed: sudo systemctl restart docker: (1.350800045s) I0131 13:43:15.057977 230266 ssh_runner.go:195] Run: sudo systemctl enable cri-docker.socket I0131 13:43:15.176891 230266 ssh_runner.go:195] Run: sudo systemctl unmask cri-docker.socket I0131 13:43:15.276928 230266 ssh_runner.go:195] Run: sudo systemctl enable cri-docker.socket I0131 13:43:15.381365 230266 ssh_runner.go:195] Run: sudo systemctl daemon-reload I0131 13:43:15.483650 230266 ssh_runner.go:195] Run: sudo systemctl restart cri-docker.socket I0131 13:43:15.498145 230266 ssh_runner.go:195] Run: sudo systemctl daemon-reload I0131 13:43:15.585468 230266 ssh_runner.go:195] Run: sudo systemctl restart cri-docker I0131 13:43:15.628653 230266 start.go:519] Will wait 60s for socket path /var/run/cri-dockerd.sock I0131 13:43:15.628763 230266 ssh_runner.go:195] Run: stat /var/run/cri-dockerd.sock I0131 13:43:15.632123 230266 start.go:540] Will wait 60s for crictl version I0131 13:43:15.632218 230266 ssh_runner.go:195] Run: which crictl I0131 13:43:15.634024 230266 ssh_runner.go:195] Run: sudo /usr/bin/crictl version I0131 13:43:15.667751 230266 start.go:556] Version: 0.1.0 RuntimeName: docker RuntimeVersion: 24.0.7 RuntimeApiVersion: v1 I0131 13:43:15.667847 230266 ssh_runner.go:195] Run: docker version --format {{.Server.Version}} I0131 13:43:15.683418 230266 ssh_runner.go:195] Run: docker version --format {{.Server.Version}} I0131 13:43:15.708160 230266 out.go:204] ๐Ÿณ Preparing Kubernetes v1.28.3 on Docker 24.0.7 ... I0131 13:43:15.711301 230266 out.go:177] โ–ช env NO_PROXY=127.0.0.1 I0131 13:43:15.714485 230266 ssh_runner.go:195] Run: grep 10.0.2.2 host.minikube.internal$ /etc/hosts I0131 13:43:15.717080 230266 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "10.0.2.2 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts"" I0131 13:43:15.724345 230266 certs.go:56] Setting up /home/caerulescens/.minikube/profiles/minikube for IP: 10.0.2.15 I0131 13:43:15.724379 230266 certs.go:190] acquiring lock for shared ca certs: {Name:mk64b385ec688d7401f4709104c32dac3b04a23d Clock:{} Delay:500ms Timeout:1m0s Cancel:} I0131 13:43:15.724503 230266 certs.go:199] skipping minikubeCA CA generation: /home/caerulescens/.minikube/ca.key I0131 13:43:15.724534 230266 certs.go:199] skipping proxyClientCA CA generation: /home/caerulescens/.minikube/proxy-client-ca.key I0131 13:43:15.724599 230266 certs.go:437] found cert: /home/caerulescens/.minikube/certs/home/caerulescens/.minikube/certs/ca-key.pem (1675 bytes) I0131 13:43:15.724622 230266 certs.go:437] found cert: /home/caerulescens/.minikube/certs/home/caerulescens/.minikube/certs/ca.pem (1094 bytes) I0131 13:43:15.724641 230266 certs.go:437] found cert: /home/caerulescens/.minikube/certs/home/caerulescens/.minikube/certs/cert.pem (1135 bytes) I0131 13:43:15.724662 230266 certs.go:437] found cert: /home/caerulescens/.minikube/certs/home/caerulescens/.minikube/certs/key.pem (1675 bytes) I0131 13:43:15.724944 230266 ssh_runner.go:362] scp /home/caerulescens/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes) I0131 13:43:15.737473 230266 ssh_runner.go:362] scp /home/caerulescens/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1675 bytes) I0131 13:43:15.750977 230266 ssh_runner.go:362] scp /home/caerulescens/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes) I0131 13:43:15.765840 230266 ssh_runner.go:362] scp /home/caerulescens/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1675 bytes) I0131 13:43:15.778183 230266 ssh_runner.go:362] scp /home/caerulescens/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes) I0131 13:43:15.790421 230266 ssh_runner.go:195] Run: openssl version I0131 13:43:15.793654 230266 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem" I0131 13:43:15.799438 230266 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem I0131 13:43:15.801881 230266 certs.go:480] hashing: -rw-r--r-- 1 root root 1111 Jan 31 18:42 /usr/share/ca-certificates/minikubeCA.pem I0131 13:43:15.801949 230266 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem I0131 13:43:15.804755 230266 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0" I0131 13:43:15.809873 230266 ssh_runner.go:195] Run: ls /var/lib/minikube/certs/etcd I0131 13:43:15.811828 230266 certs.go:353] certs directory doesn't exist, likely first start: ls /var/lib/minikube/certs/etcd: Process exited with status 2 stdout: stderr: ls: cannot access '/var/lib/minikube/certs/etcd': No such file or directory I0131 13:43:15.811957 230266 ssh_runner.go:195] Run: docker info --format {{.CgroupDriver}} I0131 13:43:15.827191 230266 cni.go:84] Creating CNI manager for "" I0131 13:43:15.827219 230266 cni.go:136] 2 nodes found, recommending kindnet I0131 13:43:15.827237 230266 kubeadm.go:87] Using pod CIDR: 10.244.0.0/16 I0131 13:43:15.827270 230266 kubeadm.go:176] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:10.0.2.15 APIServerPort:8443 KubernetesVersion:v1.28.3 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:minikube NodeName:minikube-m02 DNSDomain:cluster.local CRISocket:/var/run/cri-dockerd.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "10.0.2.15"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:10.0.2.15 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt StaticPodPath:/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[hairpinMode:hairpin-veth runtimeRequestTimeout:15m] PrependCriSocketUnix:true} I0131 13:43:15.827427 230266 kubeadm.go:181] kubeadm config: apiVersion: kubeadm.k8s.io/v1beta3 kind: InitConfiguration localAPIEndpoint: advertiseAddress: 10.0.2.15 bindPort: 8443 bootstrapTokens: - groups: - system:bootstrappers:kubeadm:default-node-token ttl: 24h0m0s usages: - signing - authentication nodeRegistration: criSocket: unix:///var/run/cri-dockerd.sock name: "minikube-m02" kubeletExtraArgs: node-ip: 10.0.2.15 taints: [] --- apiVersion: kubeadm.k8s.io/v1beta3 kind: ClusterConfiguration apiServer: certSANs: ["127.0.0.1", "localhost", "10.0.2.15"] extraArgs: enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota" controllerManager: extraArgs: allocate-node-cidrs: "true" leader-elect: "false" scheduler: extraArgs: leader-elect: "false" certificatesDir: /var/lib/minikube/certs clusterName: mk controlPlaneEndpoint: control-plane.minikube.internal:8443 etcd: local: dataDir: /var/lib/minikube/etcd extraArgs: proxy-refresh-interval: "70000" kubernetesVersion: v1.28.3 networking: dnsDomain: cluster.local podSubnet: "10.244.0.0/16" serviceSubnet: 10.96.0.0/12 --- apiVersion: kubelet.config.k8s.io/v1beta1 kind: KubeletConfiguration authentication: x509: clientCAFile: /var/lib/minikube/certs/ca.crt cgroupDriver: cgroupfs hairpinMode: hairpin-veth runtimeRequestTimeout: 15m clusterDomain: "cluster.local" # disable disk resource management by default imageGCHighThresholdPercent: 100 evictionHard: nodefs.available: "0%!"(MISSING) nodefs.inodesFree: "0%!"(MISSING) imagefs.available: "0%!"(MISSING) failSwapOn: false staticPodPath: /etc/kubernetes/manifests --- apiVersion: kubeproxy.config.k8s.io/v1alpha1 kind: KubeProxyConfiguration clusterCIDR: "10.244.0.0/16" metricsBindAddress: 0.0.0.0:10249 conntrack: maxPerCore: 0 # Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established" tcpEstablishedTimeout: 0s # Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close" tcpCloseWaitTimeout: 0s I0131 13:43:15.828047 230266 kubeadm.go:976] kubelet [Unit] Wants=docker.socket [Service] ExecStart= ExecStart=/var/lib/minikube/binaries/v1.28.3/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --container-runtime-endpoint=unix:///var/run/cri-dockerd.sock --hostname-override=minikube-m02 --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=10.0.2.15 [Install] config: {KubernetesVersion:v1.28.3 ClusterName:minikube Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} I0131 13:43:15.828157 230266 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.28.3 I0131 13:43:15.833142 230266 binaries.go:47] Didn't find k8s binaries: sudo ls /var/lib/minikube/binaries/v1.28.3: Process exited with status 2 stdout: stderr: ls: cannot access '/var/lib/minikube/binaries/v1.28.3': No such file or directory Initiating transfer... I0131 13:43:15.833218 230266 ssh_runner.go:195] Run: sudo mkdir -p /var/lib/minikube/binaries/v1.28.3 I0131 13:43:15.837874 230266 download.go:107] Downloading: https://dl.k8s.io/release/v1.28.3/bin/linux/amd64/kubectl?checksum=file:https://dl.k8s.io/release/v1.28.3/bin/linux/amd64/kubectl.sha256 -> /home/caerulescens/.minikube/cache/linux/amd64/v1.28.3/kubectl I0131 13:43:15.837877 230266 download.go:107] Downloading: https://dl.k8s.io/release/v1.28.3/bin/linux/amd64/kubelet?checksum=file:https://dl.k8s.io/release/v1.28.3/bin/linux/amd64/kubelet.sha256 -> /home/caerulescens/.minikube/cache/linux/amd64/v1.28.3/kubelet I0131 13:43:15.837878 230266 download.go:107] Downloading: https://dl.k8s.io/release/v1.28.3/bin/linux/amd64/kubeadm?checksum=file:https://dl.k8s.io/release/v1.28.3/bin/linux/amd64/kubeadm.sha256 -> /home/caerulescens/.minikube/cache/linux/amd64/v1.28.3/kubeadm I0131 13:43:16.811907 230266 ssh_runner.go:195] Run: stat -c "%!s(MISSING) %!y(MISSING)" /var/lib/minikube/binaries/v1.28.3/kubectl I0131 13:43:16.816787 230266 ssh_runner.go:352] existence check for /var/lib/minikube/binaries/v1.28.3/kubectl: stat -c "%!s(MISSING) %!y(MISSING)" /var/lib/minikube/binaries/v1.28.3/kubectl: Process exited with status 1 stdout: stderr: stat: cannot statx '/var/lib/minikube/binaries/v1.28.3/kubectl': No such file or directory I0131 13:43:16.816852 230266 ssh_runner.go:362] scp /home/caerulescens/.minikube/cache/linux/amd64/v1.28.3/kubectl --> /var/lib/minikube/binaries/v1.28.3/kubectl (49872896 bytes) I0131 13:43:19.397243 230266 ssh_runner.go:195] Run: stat -c "%!s(MISSING) %!y(MISSING)" /var/lib/minikube/binaries/v1.28.3/kubeadm I0131 13:43:19.401747 230266 ssh_runner.go:352] existence check for /var/lib/minikube/binaries/v1.28.3/kubeadm: stat -c "%!s(MISSING) %!y(MISSING)" /var/lib/minikube/binaries/v1.28.3/kubeadm: Process exited with status 1 stdout: stderr: stat: cannot statx '/var/lib/minikube/binaries/v1.28.3/kubeadm': No such file or directory I0131 13:43:19.401782 230266 ssh_runner.go:362] scp /home/caerulescens/.minikube/cache/linux/amd64/v1.28.3/kubeadm --> /var/lib/minikube/binaries/v1.28.3/kubeadm (49045504 bytes) I0131 13:43:21.973447 230266 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service kubelet I0131 13:43:21.987835 230266 ssh_runner.go:195] Run: stat -c "%!s(MISSING) %!y(MISSING)" /var/lib/minikube/binaries/v1.28.3/kubelet I0131 13:43:21.991742 230266 ssh_runner.go:352] existence check for /var/lib/minikube/binaries/v1.28.3/kubelet: stat -c "%!s(MISSING) %!y(MISSING)" /var/lib/minikube/binaries/v1.28.3/kubelet: Process exited with status 1 stdout: stderr: stat: cannot statx '/var/lib/minikube/binaries/v1.28.3/kubelet': No such file or directory I0131 13:43:21.991779 230266 ssh_runner.go:362] scp /home/caerulescens/.minikube/cache/linux/amd64/v1.28.3/kubelet --> /var/lib/minikube/binaries/v1.28.3/kubelet (110780416 bytes) I0131 13:43:22.703039 230266 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system I0131 13:43:22.711528 230266 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (370 bytes) I0131 13:43:22.726081 230266 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes) I0131 13:43:22.740133 230266 ssh_runner.go:195] Run: grep 10.0.2.15 control-plane.minikube.internal$ /etc/hosts I0131 13:43:22.743421 230266 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "10.0.2.15 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts"" I0131 13:43:22.752720 230266 host.go:66] Checking if "minikube" exists ... I0131 13:43:22.753053 230266 config.go:182] Loaded profile config "minikube": Driver=qemu2, ContainerRuntime=docker, KubernetesVersion=v1.28.3 I0131 13:43:22.753060 230266 start.go:304] JoinCluster: &{Name:minikube KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube/iso/minikube-v1.32.1-amd64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.42@sha256:d35ac07dfda971cabee05e0deca8aeac772f885a5348e1a0c0b0a36db20fcfc0 Memory:5333 CPUs:2 DiskSize:20000 VMDriver: Driver:qemu2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:35673 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.28.3 ClusterName:minikube Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP:10.0.2.15 Port:8443 KubernetesVersion:v1.28.3 ContainerRuntime:docker ControlPlane:true Worker:true} {Name:m02 IP:10.0.2.15 Port:0 KubernetesVersion:v1.28.3 ContainerRuntime:docker ControlPlane:false Worker:true}] Addons:map[default-storageclass:true storage-provisioner:true] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop: ExposedPorts:[] ListenAddress: Network:builtin Subnet: MultiNodeRequested:true ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/caerulescens:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 AutoPauseInterval:1m0s GPUs:} I0131 13:43:22.753189 230266 ssh_runner.go:195] Run: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.28.3:$PATH" kubeadm token create --print-join-command --ttl=0" I0131 13:43:22.753212 230266 sshutil.go:53] new ssh client: &{IP:localhost Port:39193 SSHKeyPath:/home/caerulescens/.minikube/machines/minikube/id_rsa Username:docker} I0131 13:43:22.896977 230266 start.go:325] trying to join worker node "m02" to cluster: &{Name:m02 IP:10.0.2.15 Port:0 KubernetesVersion:v1.28.3 ContainerRuntime:docker ControlPlane:false Worker:true} I0131 13:43:22.897032 230266 ssh_runner.go:195] Run: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.28.3:$PATH" kubeadm join control-plane.minikube.internal:8443 --token 7setjc.jwz5cbq7sux87glt --discovery-token-ca-cert-hash sha256:099eb32cca9e38030398eb0f9f7ebe0e851b9cef71131d1fb16a522f99b41bce --ignore-preflight-errors=all --cri-socket /var/run/cri-dockerd.sock --node-name=minikube-m02" *