Nodes

The K3s cluster consists of three nodes: one control-plane/master node (megatron) and two worker nodes (thing1, thing2).

Node Summary

NodeRoleK3s VersionAgeStatus
megatroncontrol-plane, masterv1.33.6+k3s188 daysReady
thing1workerv1.33.6+k3s111 hoursReady
thing2workerv1.33.6+k3s111 hoursReady

Note: thing1 and thing2 were recently joined or re-joined to the cluster (11 hours old at audit time). This is significantly newer than megatron's 88-day age, suggesting the workers were recently provisioned, re-imaged, or reconnected.

Node Details

megatron (Control Plane)

PropertyValue
Rolecontrol-plane, master
OSUbuntu 20.04.6 LTS
Kernel5.15.0-139-generic
CPUAMD Threadripper 3990X (64C/128T)
RAM251 GiB
GPUs2x NVIDIA RTX 3090 (24GB each)
Root Disk3.1T NVMe
K3s Versionv1.33.6+k3s1
Age88 days
Tailscale IP100.70.25.47

This node runs:

  • K3s server (API server, scheduler, controller-manager, embedded etcd)
  • Traefik ingress controller
  • cert-manager
  • Most application pods (as the primary node)
  • GPU-accelerated workloads
  • Docker daemon (for non-K3s containers)

thing1 (Worker)

PropertyValue
Roleworker
K3s Versionv1.33.6+k3s1
Age11 hours

thing2 (Worker)

PropertyValue
Roleworker
K3s Versionv1.33.6+k3s1
Age11 hours

Hardware details for thing1 and thing2 are not captured in the audit. Run kubectl describe node thing1 and kubectl describe node thing2 for full specs.

Node Architecture

┌──────────────────────────────────────────────────┐
│  megatron (control-plane)                        │
│  AMD Threadripper 3990X | 251GB RAM | 2x 3090    │
│  ┌──────────────────────────────────────────────┐│
│  │ K3s Server: API, scheduler, etcd, controller ││
│  │ Traefik, cert-manager, Longhorn mgr          ││
│  │ Application pods (stiq, glamhub, core, ...)  ││
│  │ GPU workloads, Docker containers             ││
│  └──────────────────────────────────────────────┘│
└──────────────────────────────────────────────────┘
         │                    │
         v                    v
┌────────────────┐   ┌────────────────┐
│ thing1 (worker)│   │ thing2 (worker)│
│ K3s Agent      │   │ K3s Agent      │
│ App pods       │   │ App pods       │
│ Longhorn       │   │ Longhorn       │
│ replicas       │   │ replicas       │
└────────────────┘   └────────────────┘

Node Management

# List nodes with details
kubectl get nodes -o wide

# Describe a specific node
kubectl describe node megatron

# Check node resource usage
kubectl top nodes

# Drain a node for maintenance
kubectl drain thing1 --ignore-daemonsets --delete-emptydir-data

# Uncordon after maintenance
kubectl uncordon thing1

# Check K3s agent status on worker
ssh thing1 'systemctl status k3s-agent'

# Join a new worker node
# On the new node:
curl -sfL https://get.k3s.io | K3S_URL=https://megatron:6443 \
  K3S_TOKEN=<token> sh -

Labels and Taints

To schedule GPU workloads only on megatron:

# Label GPU node
kubectl label node megatron gpu=true

# Taint non-GPU nodes
kubectl taint nodes thing1 gpu=false:NoSchedule
kubectl taint nodes thing2 gpu=false:NoSchedule

Pods requesting nvidia.com/gpu resources will automatically be scheduled on nodes with available GPUs through the NVIDIA device plugin.