Name: k8s-incident
Author: xuyun-io

Imported from xuyun-io/k8s-mcp (skills/kubernetes/k8s-incident/SKILL.md). Install upstream with npx skills add xuyun-io/k8s-mcp --skill k8s-incident. Copyright stays with the author (Apache-2.0).

Kubernetes Incident Response

Runbooks and diagnostic workflows for common Kubernetes incidents.

When to Apply

Use this skill when:

User mentions: "incident", "outage", "emergency", "down", "not working"
Operations: emergency response, production issues, service degradation
Keywords: "urgent", "broken", "fix", "restore", "recover"

Priority Rules

Priority	Rule	Impact	Tools
1	Check control plane first	CRITICAL	`get_pods(namespace="kube-system")`
2	Assess node health	CRITICAL	`get_nodes`
3	Gather events before changes	HIGH	`get_events`
4	Document timeline	HIGH	Manual notes
5	Rollback if safe	MEDIUM	`rollback_deployment`

Quick Reference

Incident	First Tool	Next Steps
Pod failure	`get_pod_logs(previous=True)`	`describe_pod`, `get_events`
Node down	`describe_node`	Check kubelet logs
Service unreachable	`get_endpoints`	`get_network_policies`
Control plane	`get_pods(namespace="kube-system")`	Check API server logs

Incident Triage

Quick Health Check

get_nodes()
get_pods(namespace="kube-system")
get_events(namespace)

Severity Assessment

Indicator	Severity	Action
Multiple nodes NotReady	Critical	Escalate immediately
kube-system pods failing	Critical	Control plane issue
Single pod CrashLoop	Medium	Debug pod
High latency	Medium	Check resources

Runbook: Pod Failures

CrashLoopBackOff

get_pod_logs(name, namespace, previous=True)
describe_pod(name, namespace)
get_events(namespace, field_selector="involvedObject.name=<pod>")
get_pod_metrics(name, namespace)

Common Causes:

OOMKilled → Increase memory limits
Exit code 1 → Application error in logs
Exit code 137 → Killed by OOM or SIGKILL
Exit code 143 → Graceful SIGTERM

ImagePullBackOff

describe_pod(name, namespace)
get_secrets(namespace)

Pending Pod

describe_pod(name, namespace)
get_nodes()
get_events(namespace)

Runbook: Node Issues

Node NotReady

describe_node(name)
get_events(namespace="", field_selector="involvedObject.name=<node>")
node_logs_tool(name, "kubelet")

Node DiskPressure

describe_node(name)
get_pods(field_selector="spec.nodeName=<node>")

Runbook: Network Issues

Service Not Accessible

get_services(namespace)
get_endpoints(namespace)
get_pods(namespace, label_selector="<service-selector>")
get_network_policies(namespace)

DNS Resolution Failures

get_pods(namespace="kube-system", label_selector="k8s-app=kube-dns")
get_pod_logs("coredns-xxx", "kube-system")

With Cilium

cilium_status_tool()
cilium_endpoints_list_tool(namespace)
hubble_flows_query_tool(namespace)

With Istio

istio_analyze_tool(namespace)
istio_proxy_status_tool()

Runbook: Storage Issues

PVC Pending

describe_pvc(name, namespace)
get_storage_classes()
get_events(namespace)

Pod Stuck in ContainerCreating

describe_pod(name, namespace)
get_pvc(namespace)
get_events(namespace)

Runbook: Control Plane Issues

API Server Unavailable

get_pods(namespace="kube-system", label_selector="component=kube-apiserver")
get_events(namespace="kube-system")

etcd Issues

get_pods(namespace="kube-system", label_selector="component=etcd")
get_pod_logs("etcd-xxx", "kube-system")

Emergency Actions

Force Delete Pod

delete_pod(name, namespace, grace_period=0, force=True)

Rollback Deployment

rollback_deployment(name, namespace, revision=0)

Helm Rollback

rollback_helm_release(name, namespace, revision=1)

Diagnostic Collection Script

For comprehensive incident diagnostics, see scripts/collect-diagnostics.py.

Multi-Cluster Incident Response

Check all clusters:

for context in ["prod-1", "prod-2", "staging"]:
    get_nodes(context=context)
    get_pods(namespace="kube-system", context=context)
    get_events(namespace="kube-system", context=context)

Post-Incident

Document Timeline

When did the incident start?
What was the impact?
What was the root cause?
What fixed it?

Prevent Recurrence

Add monitoring/alerting
Improve resource limits
Add readiness probes
Document runbook

Related Skills

k8s-troubleshoot - Detailed debugging
k8s-security - Security incidents

{ "ocm": "1", "id": "xuyun-io-k8s-mcp-k8s-incident", "kind": "skill", "name": "k8s-incident", "description": "Respond to Kubernetes incidents with runbooks and diagnostics. Use for outages, pod failures, node issues, network problems, and emergency response.", "publisher": "xuyun-io", "version": "1.0.0", "capabilities": { "domains": [ "coding", "observability" ], "tags": [ "skill-md", "github" ], "languages": [ "en" ] }, "quality_prior": 0.6, "examples": [ "Respond to Kubernetes incidents with runbooks and diagnostics. Use for outages, pod failures, node issues, network problems, and emergency response." ], "primary": false, "metadata": { "source": { "provider": "github", "repository": "https://github.com/xuyun-io/k8s-mcp", "path": "skills/kubernetes/k8s-incident/SKILL.md", "ref": "3ce7b99c8084e0db13b01a0c6af7169ba9227638", "url": "https://github.com/xuyun-io/k8s-mcp/blob/3ce7b99c8084e0db13b01a0c6af7169ba9227638/skills/kubernetes/k8s-incident/SKILL.md", "key": "xuyun-io/k8s-mcp/skills/kubernetes/k8s-incident/SKILL.md" }, "license": "Apache-2.0" }, "instructions": "# Kubernetes Incident Response\n\nRunbooks and diagnostic workflows for common Kubernetes incidents.\n\n## When to Apply\n\nUse this skill when:\n- User mentions: \"incident\", \"outage\", \"emergency\", \"down\", \"not working\"\n- Operations: emergency response, production issues, service degradation\n- Keywords: \"urgent\", \"broken\", \"fix\", \"restore\", \"recover\"\n\n## Priority Rules\n\n| Priority | Rule | Impact | Tools |\n|----------|------|--------|-------|\n| 1 | Check control plane first | CRITICAL | `get_pods(namespace=\"kube-system\")` |\n| 2 | Assess node health | CRITICAL | `get_nodes` |\n| 3 | Gather events befor", "cost": { "context_tokens": 1179 } }

k8s-incident

About