Kubernetes#

Collects cluster, node, workload and pod metrics, Kubernetes events, container logs and cost allocation. Installed with Helm.

Requirements#

Kubernetes 1.24 or later, Helm 3, and outbound HTTPS from the cluster. Cluster-admin rights for the initial install, because the chart creates a ClusterRole to read cluster state.

Install#

helm repo add observeiq https://charts.observeiq.io
helm repo update

helm upgrade --install observeiq-agent observeiq/agent \
  --namespace observeiq --create-namespace \
  --set apiKey="obs_your_key_here" \
  --set endpoint="https://app.observeiq.io" \
  --set cluster.name="prod-eu-west-1"

Warning

cluster.name labels every metric from this cluster and cannot be changed later without orphaning historical data. Use a name that identifies the cluster unambiguously across your whole estate, not just production.

What gets deployed#

Component	Runs as	Purpose
Node agent	DaemonSet	Node and container metrics, pod logs
Cluster collector	Deployment	Cluster state, Kubernetes events, workload inventory
Kube state metrics	Deployment	Deployment, StatefulSet, Job and pod condition metrics

Confirm it is running:

kubectl get pods -n observeiq
kubectl logs -n observeiq -l app.kubernetes.io/name=observeiq-agent --tail=50

The cluster appears under Infrastructure → Kubernetes within about two minutes.

What is collected#

Cluster: node count and condition, capacity and allocatable, control plane reachability.
Nodes: CPU, memory, disk and network per node, and pressure conditions.
Workloads: desired against ready replicas for Deployments, StatefulSets and DaemonSets; Job and CronJob outcomes.
Pods: CPU and memory against requests and limits, restart counts, waiting and terminated reasons such as CrashLoopBackOff and OOMKilled.
Events: the Kubernetes event stream, correlated with the workload it refers to.
Logs: container logs, labelled with namespace, workload, pod and container.

Limiting what is collected#

By namespace:

helm upgrade --install observeiq-agent observeiq/agent \
  --namespace observeiq \
  --set 'collection.namespaces={default,production,payments}'

Or exclude noisy ones:

  --set 'collection.excludeNamespaces={kube-system,cert-manager}'

Logs can be turned off entirely if you already ship them elsewhere:

  --set logs.enabled=false

Cost allocation#

Enable cost allocation to break spend down by namespace, workload and label:

  --set cost.enabled=true \
  --set cost.nodePricing.source="cloud"

With cloud pricing the agent reads instance types and looks up on-demand rates from the connected cloud account, so connect the cloud account first. Otherwise set a flat hourly rate per node with cost.nodePricing.hourlyRate.

Results appear under Cost Explorer.

Adding more clusters#

Repeat the install in each cluster with a different cluster.name. They all report into the same organisation and are filterable by cluster on every page. There is no per-cluster setup on the ObserveIQ side.

Scraping your own Prometheus targets#

If you already annotate pods for Prometheus scraping, the agent can honour those annotations:

  --set prometheus.scrapeAnnotations=true

Pods annotated with prometheus.io/scrape: "true" are then scraped and their metrics forwarded.

If instead you run a full Prometheus and would rather not duplicate the data, leave this off and attach that Prometheus as a data source. See Connecting a cloud account for the general pattern, or the data sources page in the product.

Upgrading and uninstalling#

helm repo update && helm upgrade observeiq-agent observeiq/agent -n observeiq
helm uninstall observeiq-agent -n observeiq && kubectl delete namespace observeiq