Ollama
Ollama provides local LLM inference on the MEGATRON cluster, using GPU acceleration for fast generation.
Overview
| Property | Value |
|---|---|
| Namespaces | ai-ml, glamhub |
| Port | 11434 |
| GPU | RTX 3090 (GPU 0, ~2.7 GB VRAM at idle) |
Deployments
Ollama runs in two namespaces:
| Namespace | Purpose | Notes |
|---|---|---|
| ai-ml | General LLM inference for research | Cluster-wide access |
| glamhub | Content generation for GlamHub | N8N workflow integration |
Usage
# From within the cluster
curl http://ollama.ai-ml.svc.cluster.local:11434/api/generate \
-d '{"model": "llama3", "prompt": "Hello"}'
# List loaded models
curl http://ollama.ai-ml.svc.cluster.local:11434/api/tags
kubectl get pods -n ai-ml -l app=ollama
TODO: expand with model inventory, VRAM requirements per model, and performance benchmarks.