Skip to main content
MK
Mohan Kumar BN Platform Engineer · Kubestronaut
SRE & AI Ops
· 8 min read

AI-Powered Kubernetes Troubleshooting with K8sGPT and Local LLMs

Deploy the K8sGPT CLI and in-cluster Kubernetes Operator backed by local LLMs to continuously scan clusters, diagnose CrashLoopBackOff and Ingress errors, and export SRE metrics to Prometheus.

Mohan Kumar BN

Mohan Kumar BN

Senior Platform Engineer & CNCF Certified Kubestronaut

GitHub Code ↗ Full Guide on MKBN-Tech ↗

Blueprint Topology

TOPOLOGY // DECLARATIVE BLUEPRINT
Mermaid Topology Specification (Architecture-as-Code) ▼ 
flowchart LR
  K8s["Kubernetes API Server\n(Pods, PVCs, Services, Ingress)"] --> Operator["K8sGPT In-Cluster Operator\n(Continuous Analyzers)"]
  Operator <-->|"Diagnostic Prompt"| LLM["Local Ollama Backend\n(Private AI Inference)"]
  Operator --> CRD["Result CRDs\n(kubectl get results)"]
  Operator --> Prom["Prometheus ServiceMonitor\n(k8sgpt_number_of_results)"]
▶ Companion Video Walkthrough (@mkbn-tech) Open on YouTube ↗
Watch video tutorial for AI-Powered Kubernetes Troubleshooting with K8sGPT and Local LLMs

Overview

Kubernetes clusters generate vast streams of events, conditions, and status transitions that take valuable time to correlate during an incident. K8sGPT codifies SRE domain knowledge into built-in analyzers (covering Pods, Deployments, StatefulSets, PVCs, Services, Ingresses, NetworkPolicies, and CronJobs) and enriches those findings with clear natural-language remediation steps.

Installing and Configuring K8sGPT CLI

1. Configure Local AI Backend Authentication

# Add local Ollama backend to K8sGPT
k8sgpt auth add --backend localai --model llama3 --baseurl http://localhost:11434/v1
k8sgpt auth default --provider localai

2. Run Cluster-Wide SRE Diagnostics

# Scan cluster for anomalies and explain root causes
k8sgpt analyze --explain --with-doc

# Filter analysis specifically to failing Pods and Services in JSON format
k8sgpt analyze --filter=Pod,Service --explain --output=json

Deploying the In-Cluster K8sGPT Operator

For continuous 24/7 cluster monitoring, deploy the K8sGPT Operator via Helm and wire it to Prometheus:

helm repo add k8sgpt https://charts.k8sgpt.ai/
helm repo update

helm install release k8sgpt/k8sgpt-operator \
  --namespace k8sgpt-operator-system \
  --create-namespace \
  --set serviceMonitor.enabled=true

Declarative K8sGPT Custom Resource

apiVersion: core.k8sgpt.ai/v1alpha1
kind: K8sGPT
metadata:
  name: cluster-sre-analyzer
  namespace: k8sgpt-operator-system
spec:
  ai:
    enabled: true
    model: llama3
    backend: localai
    baseUrl: http://ollama.ai-stack.svc.cluster.local:11434/v1
  noCache: false
  version: v0.3.41
  filters:
    - Pod
    - Deployment
    - PersistentVolumeClaim
    - Service
    - Ingress

Once reconciled, the operator emits Result custom resources (kubectl get results -n k8sgpt-operator-system -o yaml) and exposes k8sgpt_number_of_results metrics directly to Prometheus and Grafana.

Related Engineering Blueprints