Services currently in a degraded, failed, or problematic state on MEGATRON.
Service Unit State Impact Resolution
MOTD News motd-news.service Failed None sudo systemctl reset-failed motd-news.serviceWake-on-LAN wol-enable.service Failed None sudo systemctl mask wol-enable.service
These services cause the system to report State: degraded in systemctl status, but have no functional impact.
# Check system state
systemctl status
# List failed units
systemctl --failed
# Reset all failed units
sudo systemctl reset-failed
Certificate Namespace State Duration Root Cause
harbor-tls registry NOT READY 20 days HTTP-01 solver broken behind CF tunnel devtron-tls devtroncd NOT READY 20 days HTTP-01 solver broken behind CF tunnel wildcard-sslip-tls monitoring NOT READY Unknown HTTP-01 solver broken behind CF tunnel
Resolution: Migrate cert-manager to DNS-01 solver. See Certificate Renewal .
# Check all certificate states
kubectl get certificates -A
# Check pending challenges
kubectl get challenges -A
# View cert-manager logs for errors
kubectl logs -n cert-manager deployment/cert-manager --tail=200
Property Value
Namespace monitoring Issue 203% CPU consumption CPU Hours 24,000+ accumulated Likely Cause Memory leak or alert routing loop
# Check AlertManager status
kubectl get pods -n monitoring -l app.kubernetes.io/name=alertmanager
# Check resource consumption
kubectl top pod -n monitoring
# Restart
kubectl rollout restart statefulset/alertmanager -n monitoring
Property Value
Namespace linkerd Status Running Age 11 hours Concern Very recently deployed; monitor for stability
Linkerd was deployed/redeployed very recently. Watch for:
Proxy injection issues
mTLS handshake failures
Increased latency
# Check Linkerd health
kubectl get pods -n linkerd
linkerd check # if CLI installed
Property Value
Namespace vault Uptime 3h51m at audit time Concern Recently restarted; may need unsealing
# Check Vault status
kubectl get pods -n vault
kubectl exec -n vault vault-0 -- vault status
Property Value
Container bibliotheca-categorizer Image custom Status RESTARTING (continuous) Impact Book categorization broken
# Check restart count
docker inspect bibliotheca-categorizer --format='{{.RestartCount}}'
# Check last exit code
docker inspect bibliotheca-categorizer --format='{{.State.ExitCode}}'
# View recent logs
docker logs bibliotheca-categorizer --tail=50
# Stop the restart loop temporarily
docker update --restart=no bibliotheca-categorizer
Property Value
Service VNC server Status Restart loop (617K+ attempts) Auth NONE (security risk) Impact Resource waste, security vulnerability
This is the most urgent degraded service due to the security implications. See Current Issues for remediation steps.
# Find and stop VNC processes
ps aux | grep -i vnc
sudo systemctl stop 'vncserver@*'
sudo systemctl disable 'vncserver@*'
# Verify no VNC ports are listening
ss -tlnp | grep -E '590[0-9]'
Service Status Action Needed
─────────────────────────────────────────────────────
systemd overall DEGRADED Reset failed units
motd-news.service FAILED Reset or disable
wol-enable.service FAILED Mask service
AlertManager 203% CPU Restart pod
harbor-tls NOT READY DNS-01 migration
devtron-tls NOT READY DNS-01 migration
monitoring certs NOT READY DNS-01 migration
Vault LOW UPTIME Monitor stability
Linkerd NEW DEPLOY Monitor stability
bibliotheca-categorizer RESTARTING Debug and fix
VNC server RESTART LOOP Kill immediately
NFS mounts HOST DOWN Lazy unmount