KubeSolo Networking: Node IP, MTU, LoadBalancer & CNI

How networking works

A single node has no peers to route to, so KubeSolo's networking is deliberately simple. Pods attach to a Linux bridge (cni0) through the standard bridge, host-local, portmap and loopback CNI plugins, which ship inside the binary. kube-proxy implements Services, CoreDNS resolves names, and a built-in webhook gives LoadBalancer Services an external IP. ClusterIP, NodePort and LoadBalancer all work without a cloud provider.

Default address ranges

Range	Value
Pod CIDR	10.42.0.0/16
Service CIDR	10.43.0.0/16
`kubernetes` Service	10.43.0.1
CoreDNS	10.43.0.10
Cluster domain	cluster.local

These are fixed in v1.2.1. Make sure they do not overlap networks the node needs to reach, such as a site LAN on 10.42.x.x.

Node IP

KubeSolo picks one IPv4 address as the node IP and uses it for the API server's advertise address, the server URL in the admin kubeconfig, the kubelet's node address and the LoadBalancer EXTERNAL-IP.

Auto-detection takes the first private (RFC 1918) IPv4 address it finds, and falls back to the first non-loopback IPv4 address. On a host with several private addresses the first one found wins, and that is not guaranteed to be the same after a reboot. Pin it on multi-NIC hosts:

bash
1$ sudo kubesoloctl config set network.nodeIP 192.168.10.5
2$ sudo systemctl restart kubesolo

The value must be an IPv4 address; anything else is ignored with a warning and auto-detection is used. An address not bound to a local interface is accepted with a warning, which allows a VIP. When the node IP changes, KubeSolo re-signs its certificates and regenerates the admin kubeconfig, so copy it off the node again.

Reaching the API server by another name

The API server certificate covers the node IP. To use kubectl through a DNS name, a NAT address or a second interface, add them as Subject Alternative Names:

yaml
1kubernetes:
2  apiServer:
3    extraSANs: [kubesolo.local, 203.0.113.10]

At install time use --apiserver-extra-sans=kubesolo.local,203.0.113.10. Then edit the server: line of your copied kubeconfig to the name you use.

MTU

The bridge CNI takes its MTU from the same interface the node IP comes from, and applies it to cni0 and every pod interface. This keeps pod traffic from fragmenting or blackholing on VPN, tunnel and PPPoE links. If detection fails, KubeSolo uses 1500.

Override it with network.mtu. Values outside 68 to 65535 are ignored with a warning, and values below 1280, the IPv6 minimum, warn unless IPv6 is disabled. The install script has no --mtu option, so set it in the file, or pass KUBESOLO_MTU with sudo -E at install time:

bash
1$ sudo kubesoloctl config set network.mtu 1400
2$ sudo systemctl restart kubesolo
3$ ip link show cni0

In kubesoloctl container mode, --mtu also sizes the Docker network the KubeSolo container runs on.

IPv6

The pod and Service networks are IPv4. By default CoreDNS also serves ip6.arpa reverse zones. network.disableIPv6: true (or KUBESOLO_DISABLE_IPV6=true at install) does three things:

CoreDNS stops serving ip6.arpa;
the kubelet registers the node with an explicit IPv4 address;
KubeSolo writes 1 to net.ipv6.conf.{all,default,lo}.disable_ipv6, turning IPv6 off for the whole host.

Host-wide effect: disabling IPv6 changes kernel settings for every interface on the machine, not just for KubeSolo. Check that nothing else on the host needs IPv6 first.

kube-proxy and nftables-only hosts

kube-proxy normally runs in iptables mode. If the kernel has no iptables support (/proc/net/ip_tables_names is missing), KubeSolo switches it to nftables mode, which is stable since Kubernetes 1.31. On Alpine kube-proxy uses nftables, so the nft binary must be installed.

The install script still checks for the iptables command and its xt_comment module before installing, so install the iptables package even on nftables-based systems.

DNS

CoreDNS 1.14.4 runs in kube-system at 10.43.0.10. It answers for cluster.local and forwards everything else to the resolvers in the host's /etc/resolv.conf. In container mode it forwards to 1.1.1.1 and 8.8.8.8 instead, because the node's own resolver configuration is not usable inside the container.

LoadBalancer services

KubeSolo ships a built-in LoadBalancer implementation. When a Service of type LoadBalancer is created, or an existing Service is changed to that type, a KubeSolo webhook sets its EXTERNAL-IP to the node IP. kube-proxy then forwards traffic arriving on that IP and port to the pods. Unlike NodePort, which uses 30000–32767, a LoadBalancer Service can use any port, including ports below 1024, so 80, 443 and 53 work without NAT or port-mapping workarounds.

example-lb-service.yaml
1apiVersion: v1
2kind: Service
3metadata:
4  name: my-web
5spec:
6  type: LoadBalancer
7  selector:
8    app: my-web
9  ports:
10    - port: 80          # exposed on the node IP
11      targetPort: 8080  # container port

After applying it, kubectl get svc my-web shows the node IP in the EXTERNAL-IP column.

Settings

Setting	Default	Effect
network.loadBalancer.enabled	true	Turn the built-in LoadBalancer off. Required while d2k is enabled.
network.loadBalancer.ip	"" (node IP)	Publish a different IPv4 address as EXTERNAL-IP, for example a second NIC or a VIP. An invalid value falls back to the node IP.

Service types

Type	Port range	External IP	Use when
ClusterIP	Any (internal only)	None	Service-to-service traffic inside the node.
NodePort	30000–32767	Node IP and a high port	Simple external access where the port does not matter.
LoadBalancer	Any, including <1024	Node IP (or `loadBalancer.ip`) on the declared port	Exposing HTTP/S, DNS or any well-known port directly.

Firewall note: LoadBalancer and NodePort traffic arrives on the host's interfaces. If the node is reachable from untrusted networks, allow only the ports you intend to expose in the host firewall.

Using another CNI (Cilium)

The built-in bridge CNI covers a single node. For NetworkPolicy enforcement or an eBPF dataplane you can install another CNI on top. Two KubeSolo specifics apply to any CNI:

Plugin directory. KubeSolo's containerd loads CNI binaries from /var/lib/kubesolo/containerd/cni/plugins, not /opt/cni/bin. CNI configuration stays in the standard /etc/cni/net.d.
No scheduler. KubeSolo has no kube-scheduler. NodeSetter, a lightweight mutating admission webhook built into KubeSolo, sets spec.nodeName on every new pod. Scheduler predicates such as anti-affinity and host-port checks therefore never run, so extra replicas that want the same host port are rejected by the kubelet with status NodePorts. Run single replicas of such components.

For Cilium (tested with 1.19.5 via the Cilium CLI):

bash
1$ cilium install --version 1.19.5 \
2  --set cni.binPath=/var/lib/kubesolo/containerd/cni/plugins \
3  --set operator.replicas=1
4$ cilium status --wait

If Cilium is already installed, apply the same values with cilium upgrade --reuse-values and restart the agent with kubectl -n kube-system rollout restart ds/cilium. Cilium is far heavier than the built-in CNI; size the hardware for it.

Ports

Port	Component	Listens on	Open in a firewall?
6443/tcp	Kubernetes API server	all interfaces	Only if kubectl or Portainer connects from another machine.
10250/tcp	kubelet	all interfaces	No.
10443/tcp	NodeSetter and LoadBalancer webhook	all interfaces	No. Only the local API server calls it.
2379/tcp	Kine (etcd API over SQLite)	127.0.0.1	No.
9105/tcp	Metrics endpoint, when enabled	127.0.0.1 by default	Only if you change `metrics.bindAddress` for remote scraping.
2376/tcp	d2k Docker API, when enabled	LoadBalancer IP	For Docker clients on other machines.
6060/tcp	pprof, when enabled	all interfaces	No. Enable only while profiling.
30000–32767	NodePort Services	all interfaces	Ports you expose.

The installer and kubesoloctl check fail if 2379, 6443 or 10443 is already taken by another process.