Changelog
What’s new in Napwake.
Each release in plain words first, then every change, from the project’s CHANGELOG.md. Newest first.
0.2.0
Release candidate rc.7 · 24 October 2026
- rc.118 Oct
- rc.218 Oct
- rc.319 Oct
- rc.421 Oct
- rc.522 Oct
- rc.623 Oct
- rc.724 Oct
- 0.2.0when it ships
What’s new
0.2.0 is the relaunch: a fresh start, so install it new rather than upgrading from 0.1.x.
Napwake pauses the workloads you opt in with a label after idleAfter with no sign of use. A request to a paused workload’s Service wakes it through the doorman, with a waking page for browsers. The forecast learns each workload’s week. With autoResume, it wakes them ahead of demand.
kubectl napwake scan shows what’s idle with nothing installed, and Napwake reports what pausing freed apart from what came off the bill.
- Opt in with a labelOn a Deployment, a StatefulSet or a whole namespace, with settings as annotations.
- Wake on requestThe doorman holds a request to a paused workload while it starts, at its own Service.
- A waking pageBrowsers see the address they opened and how long it has been starting.
- Scan before you installA read-only report on a cluster’s Deployments and StatefulSets, with nothing installed.
- Freed, and savedWhat a pause freed, apart from what it measurably took off the bill.
- GitOps undoing a pause, reportedArgo CD, Flux, Fleet, kapp and Kluctl undoing a pause is reported with the fix.
Renamed
Each old name, and the new one.
Hybernate is now Napwake. Every name it leaves in a cluster or on your machine changes. None is kept for compatibility:
| Thing | Before | After |
|---|---|---|
| Product | Hybernate | Napwake |
| Repository | okedeji/hybernate | napwake/napwake |
| Go module | github.com/okedeji/hybernate | github.com/napwake/napwake |
| API group, labels, annotations | hybernate.io/…, hybernate.io/v1alpha1 | napwake.dev/…, napwake.dev/v1alpha1 |
| CRD | managedworkloads.hybernate.io | managedworkloads.napwake.dev |
| Plugin | kubectl-hybernate, kubectl hybernate | kubectl-napwake, kubectl napwake |
| Image | ghcr.io/okedeji/hybernate | ghcr.io/napwake/napwake |
| Chart | charts/hybernate, oci://ghcr.io/okedeji/charts/hybernate | charts/napwake, oci://ghcr.io/napwake/charts/napwake |
| Namespace | hybernate-system | napwake-system |
| Release objects | hybernate, hybernate-doorman, hybernate-metrics | napwake, napwake-doorman, napwake-metrics, and the wake-key Secret with them |
| Metrics | hybernate_* | napwake_* |
| Alerts | Hybernate* | Napwake* |
| Homebrew | okedeji/tap/kubectl-hybernate | napwake/tap/kubectl-napwake |
| Krew | krew-hybernate.yaml, plugin hybernate | krew-napwake.yaml, plugin napwake |
| Release archives | kubectl-hybernate-<os>-<arch>.tar.gz | kubectl-napwake-<os>-<arch>.tar.gz |
| Docs | https://okedeji.io/hybernate/<path> | https://docs.napwake.dev/<path>, the same paths |
| The asleep page's form field | hybernate-wake | napwake-wake |
| Go identifiers, environment variables | Hybernate*, hybernate*, HYBERNATE_* | Napwake*, napwake*, NAPWAKE_* |
There's no upgrade path from Hybernate 0.1.x or a 0.2.0 release candidate. Uninstall Hybernate, then install Napwake and opt your workloads in again with the new labels and annotations. Moving from Hybernate has the one thing to watch for.
Security 13
What closes a way in, or narrows what something may do.
The asleep page's Wake it button posts a token the doorman signed, an HMAC-SHA256 of the workload's namespace, name and UID, the pause it was shown…
The asleep page's Wake it button posts a token the doorman signed, an HMAC-SHA256 of the workload's namespace, name and UID, the pause it was shown during, and an expiry 10 minutes on, checked in constant time. Only a post of a fresh token from the page wakes the workload, from a browser's
User-Agentthat says the click came from the page: over HTTPS, where browsers send fetch metadata, withSec-Fetch-Site: same-origin, and from Chrome, Edge or Firefox withSec-Fetch-User: ?1; over plain HTTP, where they don't, with anOriginthat's the page's address, its host and port, which the page'sReferrer-Policy: same-originlets browsers send. A token for another workload or pause, or an expired one, wakes nothing, nor does a click a browser says came from another site, a script's, or anything's but a browser's; a browser from before about 2020 that says neither gets the asleep page again, asking it to wake the workload another way. A robot that forges those headers on purpose, or drives a real browser, can still click, at most once a minute. The old key stops passing 11 minutes after a doorman pod switches to a new one, and a token passes only in the spelling it was signed in. The page runs no script, itsContent-Security-PolicyandX-Frame-Optionskeep other sites from framing it, and itsReferrer-Policy: same-originsends them nothing. The doorman's replicas share the key through a Secret the chart and the Kustomize manifests create empty,<fullname>-doorman-wake-key, which the first doorman pod fills; a Role lets the doormangetandupdatethat one Secret by name, and nothing else of Secrets. The doorman's admission policy also lets it setnapwake.dev/turned-awayA dependency is only learned from a request that woke a workload when the doorman wrote its napwake.dev/last-request-from, by the field manager the…
A dependency is only learned from a request that woke a workload when the doorman wrote its
napwake.dev/last-request-from, by the field manager the API server records, and only for a sender in the woken workload's namespace or one it allows withnapwake.dev/dependents-from, the only namespaces searched for it. A search by IPv6 address reads at most 2000 pods. Before, anyone who could annotate their own ManagedWorkload could have the operator write a learned dependency and an event into the ManagedWorkload of whatever pod had the address they named, in any namespace, and page through up to 10000 pods per wakeA dependency in another namespace, declared in dependsOn or learned, is only held awake, woken and waited for when its namespace allows dependents…
A dependency in another namespace, declared in
dependsOnor learned, is only held awake, woken and waited for when its namespace allows dependents from the dependent's, with the newnapwake.dev/dependents-fromnamespace annotation (namespace names, or"*"). Otherwise the dependent reportsDependencyNotAllowed=True, with the same message whether the dependency exists or not. Before, anyone who could write a ManagedWorkload in one namespace could keep any other namespace's workloads awake, and tell fromDependencyNotFoundwhat was in itThe doorman's exposure is documented, with an example NetworkPolicy: anyone who can reach a doorman pod can reach every routed, paused workload on its Service ports, and the workload's own NetworkPolicy sees the doorman as the source. See Security
The doorman only dials Pod-backed IP endpoints of EndpointSlices the EndpointSlice controller manages, never loopback, link-local, or cloud metadata…
The doorman only dials Pod-backed IP endpoints of EndpointSlices the EndpointSlice controller manages, never loopback, link-local, or cloud metadata addresses (AWS's
fd00:ec2::254, Alibaba Cloud's100.100.100.200), and only an address the pod an endpoint names has, in the Service's namespace (it maygetpods for this), so a forged EndpointSlice can't send it past a NetworkPolicy into another namespace; it caps held connections (2048 per pod, 1024 per source IP, 256 per workload, and 128 per source IP and workload, so one workload that never becomes Ready can't take every place) and connections it passes through (1024 per pod and 512 per source IP, holding a caller over the cap until a place frees), closes a passed-through connection idle for 10 minutes or open 5 minutes after its route started draining, and rate-limits wakes per source and overall, holding a caller over the wake limit, or showing it the waking-up page, while the wake is retried. The Helm valuedoorman.networkPolicy.ingressFromrestricts who can reach paused workloads through the doorman, such as only the ingress controller's namespace, anddoorman.networkPolicy.egress(off by default) adds an egress NetworkPolicy allowing the doorman only the pod CIDRs and the API serverOne workload may hold at most 32 doorman ports and one namespace 256 (
doorman.portsPerWorkload,doorman.portsPerNamespace;--doorman-ports-per-workload,--doorman-ports-per-namespace), so one tenant can't use up the 10,000 the cluster shares; a workload over either reportsWakeOnRequest=False, reasonTooManyPortsThe operator can only
getConfigMaps, one at a time, and leader election only needs Leases and Events: the chart no longer grants ConfigMap writes or unused leader-election verbsThe scan's HTML report is written readable only by you, and a reported address is only its scheme, host and port: credentials, paths and query strings are never shown
Released Helm charts are signed with cosign, like the image, and every release publishes a signed checksums.txt (checksums.txt.sigstore.json) covering…
Released Helm charts are signed with cosign, like the image, and every release publishes a signed
checksums.txt(checksums.txt.sigstore.json) covering the plugin binaries,install.yaml, the chart, and an SBOM for each of the image's platforms, and GitHub build provenance for the image and those files (gh attestation verify). See Security for the commandsEvery pull request and release scans the Go code with govulncheck and the image with Trivy, failing on reachable Go vulnerabilities and on critical…
Every pull request and release scans the Go code with govulncheck and the image with Trivy, failing on reachable Go vulnerabilities and on critical and high image vulnerabilities with a fix; each release attaches Trivy's report for each platform (
vulnerabilities-linux-amd64.txt,vulnerabilities-linux-arm64.txt). A release is scanned before anything is published, andmainand the released image are scanned again every dayBuilds use Go 1.26.10 and golang.org/x/net 0.61.0, fixing every reachable vulnerability govulncheck reports, among them those in net/http, crypto/tls…
Builds use Go 1.26.10 and
golang.org/x/net0.61.0, fixing every reachable vulnerability govulncheck reports, among them those innet/http,crypto/tlsandhtml/templatethat the doorman and metrics servers reach; CI built with Go 1.25.3. The base images are pinned by digest, every GitHub Action by commit, and Dependabot updates them weekly. Workflows get read-only permissions unless a job needs moreA ValidatingAdmissionPolicy (doorman.admissionPolicy.enabled, on by default; in the Kustomize manifests too, following the namespace and name prefix…
A ValidatingAdmissionPolicy (
doorman.admissionPolicy.enabled, on by default; in the Kustomize manifests too, following the namespace and name prefix of any overlay on them) lets the doorman's ServiceAccount change only thenapwake.dev/last-request,last-request-fromandturned-awayannotations of ManagedWorkloads, though its role can patch them, and a second,<fullname>-request-stamps, lets no one else set them, not even under Napwake's field manager, which the operator also checksKubernetes' own namespaces (
kube-*) and Napwake's own are always protected unless labellednapwake.dev/allow-protected: "true", and Napwake never manages its own operator or doorman, whatever its namespace allows (Protected=True, reasonNapwakeComponent): labelling every namespace can't pause CoreDNS, metrics-server, or Napwake itself
Added 76
New in this release.
Asking before waking, for environments on a public URL that crawlers and scanners would keep awake, such as a preview environment for each pull…
Asking before waking, for environments on a public URL that crawlers and scanners would keep awake, such as a preview environment for each pull request: with
spec.wake.mode: Ask(napwake.dev/wake: askwith the label, or the Helm valuedefaults.wake: ask), the doorman routes the paused workload as before, but only a person wakes it. A browser opening a page, or posting a small form that isn't the button's, such as the app's login form, gets an asleep page, a200withCache-Control: no-store,X-Robots-Tag: noindexandReferrer-Policy: same-origin, with a Wake it button; a click wakes the workload as a held request does, and the waking-up page follows. Everything else, crawlers,curl,HEAD, API clients and scanners, gets a503withRetry-After: 300and a short text or JSON body saying how to wake it, and a connection that isn't HTTP is closed; both are counted as turned away, and nothing wakes. Probes, health checks and hidden files are answered as before, and once it's waking, callers are answered as forRequest. A click wakes a workload at most once a minute, and a workload that asks never pauses early, since someone who came early would find it asleep.WakeOnRequestsays the doorman asks;kubectl napwake statussayswakes on a click, a wake request or activity. See Asking before wakingWhat a workload that asks turns awaystatus.turnedAway counts the requests the doorman turned away during its current or last pause, a count only, brought up to date about once a minute…
status.turnedAwaycounts the requests the doorman turned away during its current or last pause, a count only, brought up to date about once a minute from the doorman'snapwake.dev/turned-awayannotation rather than written once a request;kubectl napwake statuslists the workloads that turned requests away, andnapwake_doorman_wakes_totalgains the resultsaskedandturned_away. Many mean something automated calls the workloadPublic preview environments, a guide for teams with an environment for each pull request on a public URLopting in each preview's namespace with napwake.dev/wake: ask from a Namespace, an Argo CD Application's managedNamespaceMetadata or a kustomization,…
opting in each preview's namespace with
napwake.dev/wake: askfrom a Namespace, an Argo CD Application'smanagedNamespaceMetadataor a kustomization, or every workload withdefaults.wake; having what only other workloads call, such as an API or a database, woken with what people open, as its dependency, or on any request; what people and robots see; waking a preview from a GitHub Actions workflow on every push, withkubectl napwake wake, or with a portal'sPATCH, and the access each takes,listincluded, whichkubectl waitneeds; what the forecast andautoResumelearn from people's wakes, and how long that takes; and what's turned away. See Public Preview EnvironmentsForecast setting in plain wordsnapwake.dev/forecast (eager, balanced or cautious) on an opted-in workload or its namespace, the Helm value defaults.forecast (--default-forecast),…
napwake.dev/forecast(eager,balancedorcautious) on an opted-in workload or its namespace, the Helm valuedefaults.forecast(--default-forecast), andspec.prediction.modeon a ManagedWorkload (balancedby default) say how soon the forecast acts, at a confidence of 60, 75 or 85: eager acts sooner, and on less regular patterns, and wakes a workload for nothing more often; cautious acts later, for workloads costly to keep awake.spec.prediction.confidenceoverrides the setting's threshold, no longer defaults to 75, and is kept when a label-created ManagedWorkload's annotations are applied again, except a stored 75, the old default, so the setting applies to workloads opted in before it. The thresholds were chosen by simulation, and the forecasting page says what each trades. See Forecast settingA new workload's forecast starts from its historywith prometheus.url (--prometheus-url), the operator reads up to four weeks of a new ManagedWorkload's CPU in one range query, measured as live (the…
with
prometheus.url(--prometheus-url), the operator reads up to four weeks of a new ManagedWorkload's CPU in one range query, measured as live (the busiest moment of each hour, its own containers against their requests), and learns it in one go before carrying on live, so a workload with a steady pattern acts from the first day. A controller of its own reads one workload at a time, within 30 seconds, tried three times two minutes apart, so a slow Prometheus never holds up a reconcile.status.prediction.historysays what came of it (Learnedwithhours,fromandto,NoHistory,NoCPURequests,QueryFailed,NoPrometheusURL), with aForecastLearnedHistory,ForecastLearningLiveorForecastHistoryUnavailableevent. See Learning from historystatus.predictionsays whether the forecast is acting and when it willacting, threshold, reason (Acting, Observing, EarningConfidence, Relearning, NotLearning), message, and startsActing, projected by running a copy of…acting,threshold,reason(Acting,Observing,EarningConfidence,Relearning,NotLearning),message, andstartsActing, projected by running a copy of the forecast on through the last week it saw.kubectl get managedworkloads -o wideshows the reason in aForecastcolumn. See When it actskubectl napwake forecast [NAME] replays the forecast over the cluster's own Prometheus history, read as scan reads it: for each workload, whether it…
kubectl napwake forecast [NAME]replays the forecast over the cluster's own Prometheus history, read asscanreads it: for each workload, whether it would act opted in today, and over the last half of the history, from a forecast that learned the first, withautoResume, the mornings it would have had the workload ready, those it would have found it paused, the wakes for nothing and when it would have woken it.--forecast,--timezone,--window(up to 34 days), and scan's Prometheus flags; NAME shows one workload day by day; table, JSON or YAML. See Replay the ForecastA forecast simulator, internal/forecast/sim, runs the operator's own forecast engine and decision rules a minute at a time through months of synthetic…
A forecast simulator,
internal/forecast/sim, runs the operator's own forecast engine and decision rules a minute at a time through months of synthetic or recorded traffic in milliseconds, and its tests run in CI: office hours, every day, always busy and idle, loosely kept and irregular hours, a holiday, a Saturday at work, a permanent change of hours, a workload nobody uses any more, daylight saving in New York and London, daily operator restarts, pauses without the doorman, light use, a warm start, and the projection of when a forecast acts. A test runs six weeks of office hours through the reconciler and the simulator and finds the same wakes and pauses at the same minutes, and the forecasting page's timings are asserted against it.go run ./hack/forecastsimprints what it measures for each setting, or replays a workload's CPU recorded from Prometheus. See Proving itWake on request behind GKE container-native load balancing (NEGs), an optional mode for teams reached through a GKE Ingress or Gateway, off by…
Wake on request behind GKE container-native load balancing (NEGs), an optional mode for teams reached through a GKE Ingress or Gateway, off by default: with
doorman.negForwarder.enabled(--neg-forwarder), while a workload with a Service behind NEGs is paused, the operator runs a forwarder Deployment beside it,napwake-forwarder-<workload>, whose pods GKE's NEG controller accepts as the Service's endpoints, since they're in its namespace, and which pass every connection, untouched, to the doorman (/manager --forwarder). It runs two pods of 10m CPU and 32Mi (doorman.negForwarder.replicas,doorman.negForwarder.resources) spread across nodes, listens on the workload's target ports, which GKE's firewall rules already open to Google's load balancers, has no API access, meets the restricted Pod Security level, keeps mesh sidecars out, is Ready only while it reaches the doorman, and is deleted once the workload is awake and its route has drained, or with its ManagedWorkload. NewWakeOnRequestreasonForwarderNotReady, saying why;ForwarderStartedandForwarderStoppedevents;status.doorman[].forwarded. The chart adds a headless Service<fullname>-doorman-pods, a<fullname>-forwarderClusterRole, or Role per watched namespace, grantingcreateanddeleteon Deployments only in this mode, and admits forwarders to the doorman's NetworkPolicy. A request through a GKE Ingress whose Service is in NEGs, or through a GKE Gateway, wakes a paused workload only in this mode. See GKE container-native load balancingPausing early, off unless you turn it onwith napwake.dev/pause-early: "true" on an opted-in workload or its namespace, the Helm value defaults.pauseEarly (--default-pause-early), or…
with
napwake.dev/pause-early: "true"on an opted-in workload or its namespace, the Helm valuedefaults.pauseEarly(--default-pause-early), orspec.idlePolicy.pauseEarlyon a ManagedWorkload, a workload idle for 20 minutes is paused beforeidleAfterruns out when a confident forecast expects every hour until then, and at least the hour under way and the next, to be quiet, by the veto's own rules, and a request would wake it: the doorman would route every Service that selects its pods, on every port but thosenapwake.dev/doorman-ignore-portslists, within the doorman ports it and its namespace may hold, so a headless Service or one with a UDP or SCTP port keeps it from pausing early, and the operator logs why. Anactive-untilhold, awake dependents and the wait after a GitOps conflict still hold it, and in dry-run it's a would-be pause (napwake_dryrun_actions_total{action="early_pause"}). APausedEarlyevent names the hours the forecast expects quiet,status.activity.forecastQuietUntilwhen they end, andkubectl napwake statussayspaused early: the forecast expected quiet until Fri 20:00 UTCin NEXT. In the simulator, on office hours at the defaultidleAfter, it saves half an hour a weekday (48.7 hours awake a week to 46.3, or 53.7 to 46.3 atidleAfter: 2h), with no extra cold starts on any learned pattern, holidays and changes of hours included; use the forecast can't foresee, such as someone coming back after everyone left one weekday in three, costs 0.6 cold starts a week.go run ./hack/forecastsim -pause-earlymeasures it. See Pausing earlyIdleVetoedcondition, reasonForecastExpectsDemand, while a confident forecast holds off a pause; its message names the hour the forecast expects demand, and anIdleVetoedevent marks the start of each vetostatus.costis per calendar month and priced on requests (see Changed)tracked, lastAccumulatedAt, awakeCPUHours, awakeMemoryHours, storageHours, pausedCPUHours, pausedMemoryHours, savedCPUHours, savedMemoryHours,…tracked,lastAccumulatedAt,awakeCPUHours,awakeMemoryHours,storageHours,pausedCPUHours,pausedMemoryHours,savedCPUHours,savedMemoryHours,costThisMonth,freedThisMonth,savedThisMonth,savingsUnknown,costWithoutNapwakeThisMonth,projectedMonthlyCost,projectedMonthlyFreed,projectedMonthlySavings,running,pricedAt,pricedTemplateHash.kubectl get managedworkloadsgainsSavedandFreedcolumns.status.dryRungainsfreedCPUHoursandfreedMemoryHours, priced asestimatedFreedFreed and savedNapwake reports what a pause freed apart from what it saved, and measures saved rather than assuming it. Freed (freedThisMonth, projectedMonthlyFreed,…
Napwake reports what a pause freed apart from what it saved, and measures saved rather than assuming it. Freed (
freedThisMonth,projectedMonthlyFreed, theFreedcolumn) is what the paused replicas requested, priced at their nodes' list price; saved (savedThisMonth,projectedMonthlySavings,savedCPUHours,savedMemoryHours, theSavedcolumn) is a lower bound on what came off the bill: every 5 minutes the leader sets what Napwake holds paused in each node pool (karpenter.sh/nodepool,cloud.google.com/gke-nodepool,eks.amazonaws.com/nodegroup,kubernetes.azure.com/agentpool) against the spare capacity of the pool's Ready nodes, and credits what wouldn't fit to the paused workloads there. With no autoscaler, or scale-down blocked, saved stays at zero while freed grows. A pause records its pools instatus.pause.resources.nodePools. An operator limited bywatchNamespacescan't see every pod, so it reports saved asunknown, with why instatus.cost.savingsUnknown. New metrics by pool:napwake_capacity_paused_cpu_cores,napwake_capacity_paused_memory_bytes,napwake_capacity_spare_cpu_cores,napwake_capacity_spare_memory_bytes,napwake_savings_realised_dollars_per_hour. The operator caches pods for this, keeping only those bound to a node and not finished, as their node, phase, and requests (about 2.6 KB a pod), and nodes without their image lists; its role gainswatchon pods.kubectl napwake statusshows the two side by side (FREED THIS MONTHandSAVEDcolumns, totals in the summary, JSONfreedThisMonthandsavedThisMonth), says why saved can't be measured where it can't, and, when less than half of what was freed was saved, says under Needs attention that nodes aren't scaling down (JSONnodesNotScalingDown), pointing atkubectl napwake nodes.kubectl napwake scanshows them for live workloads, per workload and intotals, and its estimates are of what pausing could free, never a saving it can't know before Napwake runs:COULD FREE/MO, "could be freed a month" in the report, and JSONfreeableBasis. Dry-run measures only what pausing would free (status.dryRun.estimatedFreed). See Cost Tracking, and Node Pools for Real Savings for turning freed into savedNode autoscaler detectionthe operator looks every 10 minutes for what scales the cluster's nodes, Karpenter, Cluster Autoscaler, EKS Auto Mode or GKE Autopilot, logs what it…
the operator looks every 10 minutes for what scales the cluster's nodes, Karpenter, Cluster Autoscaler, EKS Auto Mode or GKE Autopilot, logs what it finds when that changes, warning when nothing does, and exports
napwake_node_autoscaler{kind},nonewhen it looked everywhere and found nothing andunknownwhen it wasn't allowed to look.kubectl napwake status,scanandnodesname it too (JSONnodeAutoscaler), andscansays what pausing frees comes off the bill only as nodes scale down. The operator's ClusterRole gainsliston Karpenter NodePools. See Cluster Autoscalerstatus.prediction.state: the forecast's learned state, kept in the ManagedWorkload. State that can't be read is discarded with aForecastResetwarning eventOperator flags
--max-concurrent-reconciles(8) and--timezone(UTC; the forecast's hours and weekdays follow it through daylight saving)--prometheus-selector, and the Helm valueprometheus.selectorlabel matchers, such as cluster="staging", added to the history query the operator makes, for Thanos, Mimir or a Prometheus shared by several…label matchers, such as
cluster="staging", added to the history query the operator makes, for Thanos, Mimir or a Prometheus shared by several clusters, so a same-named workload of another cluster isn't learned as this one's. It takes matchers askubectl napwake scan --prometheus-selectordoes; activity queries run as written. One that isn't label matchers, or one without--prometheus-url, stops the operator at startup, and the chart refuses the second. See Learning from historyHelm values crds.install, maxConcurrentReconciles, timezone, logEncoder, metrics.readerSubjects (bound to a new <fullname>-metrics-reader ClusterRole,…
Helm values
crds.install,maxConcurrentReconciles,timezone,logEncoder,metrics.readerSubjects(bound to a new<fullname>-metrics-readerClusterRole, so Prometheus can scrape secure metrics),doorman.podDisruptionBudget,doorman.nodeSelector,doorman.tolerations,doorman.affinity,doorman.topologySpreadConstraints,nameOverrideandfullnameOverride; and avalues.schema.jsonthat rejects invalid values, and values the chart doesn't have, such as a misspeltwatchNamespace, at installA doorman ServiceMonitor, a
NapwakeDoormanDownalert, andconfig/prometheusfor installs from the Kustomize manifestsnapwake_build_info{version}, from the operator and the doorman, which also log their version at startupnapwake_doorman_proxied_connectionsandnapwake_doorman_port_conflicts;napwake_doorman_wakes_totalgains the resultscanceled,ignored,limitedandrefusedWake on requesta client that stays connected without sending anything for 3 seconds wakes the workload, for protocols where the server speaks first, such as MySQL; a…
a client that stays connected without sending anything for 3 seconds wakes the workload, for protocols where the server speaks first, such as MySQL; a connection closed without sending anything doesn't. HTTP
GETandHEADrequests from probes, scrapers and uptime monitors (kube-probe/,Prometheus/, the blackbox exporter,vm_promscrape,GrafanaAgent/,Alloy/,OpenTelemetry Collector,otelcol, Datadog's agent and Synthetics,ELB-HealthChecker/, Route 53, Google Cloud uptime checks,Envoy/HC, UptimeRobot, Pingdom, New Relic, Uptime Kuma and Gatus, also when they follow(compatible;in a browser-styleUser-Agent) get a503and don't wake it, until a pod is Ready, when they're passed through; other methods from them, such as Prometheus remote write, wake it. Google Cloud's load balancer health check (GoogleHC/) gets a200instead, without waking it: Google's load balancers send nothing to a backend whose check fails, so told the workload is down they'd drop the request that should wake it. A connection to a workload that already has a Ready pod is passed straight through without waiting for the caller to speak, and a held caller that sends a body and leaves is let go at once. HTTPS health checks on the traffic port can't be recognised and wake the workload; see the compatibility page.doorman.healthCheckUserAgents(--doorman-health-check-user-agents) adds your own checkers and scrapers. A request for a hidden file, such as/.envor/.git/config, gets a404and doesn't wake the workload, so scanners hunting for leaked secrets don't keep it awake.napwake.dev/doorman-ignore-portson a Service keeps ports off the doorman. IPv6 and dual-stack Services are routed, one EndpointSlice per family. NewWakeOnRequestreasons:ServedByOtherPods,UnsupportedIPFamily,NodeLocalTraffic,RoutingFailed,DoormanDisabledkubectl napwakekubectl's connection flags on every command (--kubeconfig, --context, --cluster, --user, --as, --token, --server, --request-timeout, and the rest);…kubectl's connection flags on every command (
--kubeconfig,--context,--cluster,--user,--as,--token,--server,--request-timeout, and the rest);-A/--all-namespacesonstatusandscan; aversioncommand;wakeanddepstake the workload's name,kind/name, or the ManagedWorkload's name, and list the candidates when a name is ambiguous;--timeoutonstatus,depsandenablekubectl napwake scanreaches any Prometheus:--prometheus-selectorreads one cluster's series from a store that holds many,--prometheus-header(such asX-Scope-OrgIDfor Mimir),--prometheus-bearer-token-file,--prometheus-ca-fileand--prometheus-insecure-skip-verify. A--prometheus-urlthat doesn't answer like a Prometheus API fails the scankubectl napwake scan exits 1, after writing its report, when it couldn't read something your access allows; JSON and YAML gain namespaces and…
kubectl napwake scanexits 1, after writing its report, when it couldn't read something your access allows; JSON and YAML gainnamespacesandincomplete, and an empty cluster gives"workloads": []. A scan that runs out of--timeoutreports the namespaces it scanned and lists the rest inincomplete.unscanned. Scanning every namespace, it lists each kind of object across the cluster at once, a page at a time, so a cluster of thousands of namespaces takes a few dozen requestskubectl napwake scansays Napwake won't pause a StatefulSet whose PVCs are deleted on scale-down (JSONwontPause), and leaves it out of what pausing could freeKrew installs the plugin on Linux and macOS (amd64 and arm64) and Windows (amd64)
A
NOTICEfile credits the instance price data the operator and the plugin embed, from ec2instances.info (MIT); it ships in the image, under/licenseswith theLICENSE, and in every plugin archiveOpting in with Kustomizethe opt-in guide shows how to add the label without commonLabels, which also writes it into each workload's immutable selector so applying it to a…
the opt-in guide shows how to add the label without
commonLabels, which also writes it into each workload's immutable selector so applying it to a running workload fails, andmake testchecks the settings in the docs' kustomization patches and that no kustomization in the docs puts the label in a selector or a pod template. See With Kustomizemake test validates every ManagedWorkload in the docs and config/samples against the CRD, along with the partial spec snippets, the Helm values blocks…
make testvalidates every ManagedWorkload in the docs andconfig/samplesagainst the CRD, along with the partialspecsnippets, the Helm values blocks against the chart's schema, and every settings annotation through the operator's parser. It runs with the race detectorhelm uninstall is refused while ManagedWorkloads are left, by a pre-delete hook (guard.uninstall), since paused workloads would stay at zero with…
helm uninstallis refused while ManagedWorkloads are left, by a pre-delete hook (guard.uninstall), since paused workloads would stay at zero with their Services pointing at the deleted doorman; after waiting two and a half minutes for any being deleted, the hook lists those left and how to stop managing them, and those stuck being deleted without an operator, leading with one sentence that names them, what to do and the docs, which Helm 4 prints above its error as one log message. Argo CD 3.3 and later runs it as a PreDelete hook, blocking the Application's deletion the same way.helm upgradeandhelm rollbackto awatchNamespacesthat leaves out namespaces where the operator manages workloads are refused the same way, by a pre-upgrade hook (guard.upgrade). The hooks run the operator's image (--guard), withguard.*scheduling, pod annotations and labels, and keep Istio's and Linkerd's sidecars out of their pods. The installation guide gives the recovery by hand. The chart's CRD also carriesargocd.argoproj.io/sync-options: Delete=false, so deleting an Argo CD app keeps itManagedWorkloads are added to Kubernetes' built-in
view,editandadminroles, by aggregated ClusterRoles in the chart (<fullname>-view,<fullname>-edit) and the Kustomize manifests, so namespace users can see them, and editors pause, wake and configure them with the pluginHelm values
podAnnotations,podLabels,priorityClassNameandextraEnv, for the operator and, underdoorman.*, the doormanResumeIncompletecondition and warning eventa resume stops waiting 2 minutes after its first replica is Ready (status.pause.firstReadyAt), or, with none Ready, once the Deployment's…a resume stops waiting 2 minutes after its first replica is Ready (
status.pause.firstReadyAt), or, with none Ready, once the Deployment'sprogressDeadlineSecondshas passed, and the workload runs with those that are Ready. It's cleared, with anAllReplicasReadyevent, once the rest are Ready. AwaitForReadydependent starts once such a dependency serves with some of its replicas.kubectl napwake statuslists it under Needs attentionWake on request warns when a routed workload's pods have no readiness probeKubernetes counts such a pod Ready as soon as its containers start, and the doorman passes held requests to it then, which may be before the app can…
Kubernetes counts such a pod Ready as soon as its containers start, and the doorman passes held requests to it then, which may be before the app can answer them. The
WakeOnRequestcondition says so, with aNoReadinessProbewarning event once each pause, and the docs say that wake on request relies on a readiness probe that passes only once the app can serve, with an example. A startup probe or a readiness gate countskubectl napwake pause --clear-holdends anactive-untilhold, such as onewake --forset, on the ManagedWorkload or the workload, and pauses; on a terminal,pauseasks insteadkubectl napwake enablefinds the GitOps tool from who owns the setting's field, so it also finds Argo CD tracking by its default label, and warns thathelm upgradeputs back a value Helm setProtected namespaceslabel a namespace napwake.dev/protected: "true", or match the Helm value protectedNamespaces (--protected-namespaces, patterns such as prod-*), and…
label a namespace
napwake.dev/protected: "true", or match the Helm valueprotectedNamespaces(--protected-namespaces, patterns such asprod-*), and Napwake never manages anything in it: nothing is opted in, a ManagedWorkload there reportsProtected=Trueand is never paused, and a workload it had paused is woken.napwake.dev/allow-protected: "true"on the namespace allows it. The scan shows such workloads as(protected)and leaves them out of what pausing could freewatchNamespaces(--watch-namespaces)Napwake works only in those namespaces, with a Role and RoleBinding in each for the operator and the doorman instead of ClusterRoles, so Kubernetes…Napwake works only in those namespaces, with a Role and RoleBinding in each for the operator and the doorman instead of ClusterRoles, so Kubernetes refuses it anything elsewhere. The operator keeps one ClusterRole, to read Namespaces and Nodes and list Karpenter NodePools
More than one install per cluster, each in its own namespace with its own release name and
watchNamespaces, sharing one CRD.The operator stamps each ManagedWorkload it takes on with napwake.dev/operator, its namespace; another install leaves it alone, with a…The operator stamps each ManagedWorkload it takes on with
napwake.dev/operator, its namespace; another install leaves it alone, with aManagedByAnotherInstallwarning event, and its uninstall and upgrade guards don't count it. See More than one installData and Access: everything the operator, doorman, and plugin read, write, and send
Learned dependenciesNapwake reads the environment of every workload it manages, as the scan does, and holds and wakes the workloads its addresses name like a dependsOn,…
Napwake reads the environment of every workload it manages, as the scan does, and holds and wakes the workloads its addresses name like a
dependsOn, headless addresses included, with no YAML to write. They're kept instatus.learnedDependencies, with aDependenciesLearnedevent when they change, and learned again when the pod template changes and hourly. They're also learned from wakes: the doorman records where the request that woke a workload came from (napwake.dev/last-request-from), and when the pod that sent it belongs to a workload Napwake manages, that workload depends on the one it woke (source: wake).napwake.dev/ignore-dependenciesdrops one. The scan shows them as connected by Napwake, andkubectl napwake deps NAMEshows a workload's dependencies and dependents, in any namespace, with where each link came from. See Learned dependenciesHPA and KEDAthe Autoscaled condition says what scales a workload and its range. A workload a KEDA ScaledObject scales is paused through KEDA, with…
the
Autoscaledcondition says what scales a workload and its range. A workload a KEDA ScaledObject scales is paused through KEDA, withautoscaling.keda.sh/paused-replicas: "0"on the ScaledObject (status.pause.scaledObject), so KEDA doesn't scale it back up; a wake holds it at the restored replicas until they're Ready, then gives back the ScaledObject's ownpaused-replicasvalue, if it had one. A workload resumes within its HPA's or ScaledObject's range, and to at least one replica. The operator's ClusterRole gains read on HPAs, and read and patch on ScaledObjects. See HPA and KEDAGitOps tools (Argo CD, Flux, Rancher Fleet, kapp and Kluctl)when one sets a paused workload's replicas from Git, undoing the pause, the workload reports GitOpsConflict=True and a warning event with that tool's…
when one sets a paused workload's replicas from Git, undoing the pause, the workload reports
GitOpsConflict=Trueand a warning event with that tool's one-time setting that leaves replicas to Napwake, and Napwake waits an hour before pausing again instead of fighting Git; the condition is removed, with aGitOpsConflictResolvedevent, once a pause holds for an hour.kubectl napwake scanlists workloads whose replicas are set from Git before they're opted in (JSONreplicasFromGit). See GitOps toolskubectl napwake scana read-only report on one cluster's Deployments and StatefulSets, run with your own kubeconfig and nothing installed in the cluster. --context picks…a read-only report on one cluster's Deployments and StatefulSets, run with your own kubeconfig and nothing installed in the cluster.
--contextpicks the cluster (EKS and GKE context names are shortened to cluster and region),-nthe namespaces- State: each workload is idle, active, or paused right now, with the evidence (CPU against its request, when it was last deployed, activity annotations), and says how Napwake is involved: unmanaged, dry-run, or live. Managed workloads are judged by Napwake's activity clock and their own settings, others by
--cpu-threshold(10) and--idle-after(1h). A workload someone scaled to zero shows as paused (unmanaged), counted apart from what Napwake has paused - Money, by source: what each workload costs while running; what Napwake has freed this month pausing live ones, and what of that was saved, from what it records; and what pausing could free a month, measured by Napwake for dry-run workloads and estimated for unmanaged ones by replaying the activity clock over Prometheus history, with how long they'd have slept and how many times they'd have been woken. Prometheus is found by the Services the Prometheus Operator, kube-prometheus-stack, and the community chart create, and read through the API server's service proxy, or from
--prometheus-url;--windowsets how much history (7d; 0 for CPU right now only). Denied the service proxy, the scan prints the Role and RoleBinding an admin can create for that one Service.--cpu-priceand--memory-priceset your prices, and the report says which were assumed - Dependencies: what each workload connects to, from its environment variables, literal and from ConfigMaps, matched to the cluster's Services in any form cluster DNS gives them and followed to the workload behind them, with whether each is declared. Secrets are never read and passwords never shown
- Report page: run in a terminal on a machine with a desktop, the scan also opens the report as a self-contained web page: what Napwake freed and saved and what pausing could free, four facts, workloads by namespace, every workload (sortable, filterable), the dependencies (without their addresses), and what the numbers mean. It prints cleanly to PDF.
--html FILEsaves it, also with-o json;--open=falsekeeps it closed; piped output, CI, and SSH sessions get none unless asked - Output: a terminal table, or
-o json|yamlwith the cluster's report at the top level. The notes say what the scan couldn't see
- State: each workload is idle, active, or paused right now, with the evidence (CPU against its request, when it was last deployed, activity annotations), and says how Napwake is involved: unmanaged, dry-run, or live. Managed workloads are judged by Napwake's activity clock and their own settings, others by
Label opt-innapwake.dev/managed: "true" on a Deployment or StatefulSet makes Napwake manage it, and on a Namespace, every Deployment and StatefulSet in it;…
napwake.dev/managed: "true"on a Deployment or StatefulSet makes Napwake manage it, and on a Namespace, every Deployment and StatefulSet in it;napwake.dev/ignore: "true"leaves one out. Napwake creates a ManagedWorkload named after the workload, owned by it so it's deleted with it, markednapwake.dev/from-label: "true", and without the workload's labels, so a GitOps tool's tracking label can't make the tool claim it. Settings are annotations on the workload or its namespace:napwake.dev/dry-run,idle-after,cpu-threshold,auto-resume,depends-on,wake,wake-max-wait, andwake-page. The workload's own wins, then its namespace's, then the cluster-wide defaults. Booleans are exactly"true"or"false". A value that can't be read is reported in anInvalidSettingevent and skipped, so the setting comes from the namespace, then the cluster default, then the built-in default; an unreadabledry-runturns dry-run on; a label value other than"true"is pointed out in aLabelIgnoredevent. Removing the label deletes the ManagedWorkload, and a ManagedWorkload someone wrote for the same workload always wins. The operator's ClusterRole gains read access to NamespacesCluster-wide defaults for opted-in workloads: Helm values
defaults.idleAfter(1h),defaults.cpuThreshold(10), anddefaults.dryRun(false), passed as--default-idle-after,--default-cpu-threshold, and--default-dry-runDry-run summarya ManagedWorkload in dry-run keeps status.dryRun with when measuring started, how many times it would have been paused, how long it would have slept,…
a ManagedWorkload in dry-run keeps
status.dryRunwith when measuring started, how many times it would have been paused, how long it would have slept, and what that would have freed at its cost rates. A would-be pause runs from the clock running out until the next activity, when a paused workload would have been woken; its end is announced in theActivityResumedevent with the running total. The summary is dropped when dry-run ends.kubectl napwake scanshows it for each workload in dry-run, priced at the scan's prices, with a total in the headline, and its next steps give thekubectl napwake enablecommandkubectl napwake enable [KIND/]NAMEends dry-run for an opted-in workload by setting its own napwake.dev/dry-run annotation to "false", which wins over its namespace's and the cluster…ends dry-run for an opted-in workload by setting its own
napwake.dev/dry-runannotation to"false", which wins over its namespace's and the cluster default, orspec.dryRun: falseon a ManagedWorkload written by hand.--alldoes that for every workload in the namespace: when the namespace's annotation or the cluster default keeps any of them in dry-run, it sets the namespace's annotation to"false"and drops the workloads' own, or sets each workload's own when your access doesn't allow changing the namespace. When a GitOps tool (Argo CD, Flux, Fleet, kapp or Kluctl) applies the setting it changes nothing, names the tool, prints the change to make in Git, and exits non-zero;--forcechanges the cluster anywayWaking-up pagea browser opening a paused workload is answered at once with a page that shows the address it opened and how long the workload has been starting, and…
a browser opening a paused workload is answered at once with a page that shows the address it opened and how long the workload has been starting, and nothing about the workload's names, and reloads every 3 seconds; the first reload after any of the workload's pods is Ready reaches the app. Only page loads get it (
GETorHEADwithSec-Fetch-Mode: navigate, or asking for HTML from older browsers); scripts, API clients, WebSockets, and other protocols are held as before, with the bytes read to tell them apart passed on unchanged. It's a503withCache-Control: no-store. Opt out per workload withwake.page: false, ornapwake.dev/wake-page: "false"on a workload opted in with the label.napwake_doorman_wakes_totalgains the resultpage. The doorman also keeps a port open for 30 seconds after the workload is awake, passing connections straight through, so a reload that reaches it while proxies catch up lands on the app instead of an errorkubectl napwake wake NAMEasks Napwake to wake a workload now, whatever its wake setting, with a fresh napwake.dev/wake-requested token on its ManagedWorkload (below). A paused…asks Napwake to wake a workload now, whatever its wake setting, with a fresh
napwake.dev/wake-requestedtoken on its ManagedWorkload (below). A paused workload wakes, with the workloads it depends on; a running one has its idle clock restarted. It waits for Napwake's answer, then until the workload is Running, and exits 1 with Napwake's reason when it won't wake it, or, when the wait runs out, with whether Napwake had taken the request up.--for 2halso keeps it awake for that long;--wait=falsereturns straight away. It refuses with the reason, before asking, when Napwake isn't managing the workload: a missing or ignored target, or another ManagedWorkload managing itnapwake.dev/wake-requesteda token any tool sets on a ManagedWorkload to wake its workload, as napwake.dev/pause-requested pauses it, such as a pull request bot, a developer…a token any tool sets on a ManagedWorkload to wake its workload, as
napwake.dev/pause-requestedpauses it, such as a pull request bot, a developer portal or a chat command waking a preview environment for someone. It wakes a paused workload whatever itswakesetting,askincluded, with aWakeRequestedevent, and restarts an awake one's idle clock, which in dry-run ends a would-be pause. One pausing is woken once the pause is done. Napwake acts on each token once, recording it instatus.lastWakeRequest, and theWakeRequestcondition says what came of it, in the past tense with the time, until the next request:Woken(woken as requested at 2026-10-23 09:05 UTC),AlreadyAwake, orFalsewith why:ScaledToZero,Protected,TargetNotFound,TargetIgnored,DuplicateTarget, orBeingDeleted, answered before the ManagedWorkload goes. A token set before Napwake has taken the ManagedWorkload on, as by a tool that creates it and asks for its wake at once, is answeredAlreadyAwake: it's taken onRunning. One already there as Napwake takes up a pause whose record its status lost is answeredPredatesStatusand not acted on, so a backup restored without status doesn't wake the workload again on a request answered long ago. Apause-requestedtoken already on a ManagedWorkload as Napwake takes it on, or takes up such a pause, is answeredPredatesStatustoo, and never acted on.kubectl napwake statuslists the wake. See Wake NowThe forecast learns from people's wakes with
wake: ask.Only people wake such a workload while it's paused, with a click on the doorman's asleep page, a napwake.dev/wake-requested request or an activity…Only people wake such a workload while it's paused, with a click on the doorman's asleep page, a
napwake.dev/wake-requestedrequest or an activity annotation, so every hour it's paused behind the doorman is learned: busy if one of those woke it, or an activity query found activity, and quiet otherwise. A check while the doorman doesn't route its Services, or while its activity queries can't be evaluated, sees demand only if someone woke it, so an outage of the doorman or of Prometheus can't teach it quiet hours. Paused every night and woken by someone at 09:05 on weekdays, it acts after 8 days, as behind the doorman, and from thenautoResumehas it up at 08:45. Withwake: requestnothing changes.status.prediction's message endswhile it's paused, it learns from people waking it (a click on its asleep page, a wake request or an activity annotation), andkubectl napwake forecastreplays it so where the doorman would route it,peoplein WAKES ON andforecast.wakes.people. The simulator has the source asWakeOnAsk, andhack/forecastsimas-wake-on-ask. See What an Hour Recordskubectl napwake pause NAMEpauses a workload now, rather than when its idle clock runs out, such as a preview environment at the end of the day. It's an ordinary pause: the…pauses a workload now, rather than when its idle clock runs out, such as a preview environment at the end of the day. It's an ordinary pause: the workload wakes on a request through the doorman, on activity, by
autoResume, or withkubectl napwake wake. It waits until the workload is Paused, or Napwake says why it won't be (--wait=falsereturns straight away), and warns when a GitOps tool sets the replicas from Git and will likely undo the pause, with the one-time fix (see the GitOps check below). It takes NAME aswakedoes. Underneath, it sets thenapwake.dev/pause-requestedannotation to a fresh token, which any tool can set: an annotation, so Argo CD and Flux leave it alone on a ManagedWorkload kept in Git. Napwake acts on each token once, recording it instatus.lastPauseRequest, and pauses by the rules of an idle pause: not while awake workloads depend on it, anactive-untilhold applies, its namespace is protected, or it's already at zero; under dry-run it counts a would-be pause. Asking overrides the hour Napwake waits after a GitOps conflict. When a confident forecast expects demand within the hour, Napwake declines the pause and says when, andpauseasks whether to pause it anyway;--yes, or thenapwake.dev/pause-overrides-forecast: "true"annotation with the request, pauses it without asking. ThePauseRequestcondition and an event say what came of each request. See Pause nowkubectl napwake nodeswhy nodes aren't scaling down, so what pausing frees isn't saved. It names the node autoscaler, Karpenter (with each NodePool's consolidation…why nodes aren't scaling down, so what pausing frees isn't saved. It names the node autoscaler, Karpenter (with each NodePool's consolidation settings), Cluster Autoscaler (self-managed, GKE Standard or AKS), EKS Auto Mode or GKE Autopilot, or says none was found, or that it couldn't tell with your access; then each node's pool, instance type, requests against what it can allocate and utilisation, and what keeps the autoscaler from removing it:
do-not-disruptandsafe-to-evict=falsepods, PodDisruptionBudgets that allow no disruption, local storage, pods without a controller,kube-systempods without a PodDisruptionBudget,scale-down-disablednodes, and NodePools that consolidate only empty nodes or never; with how to fix each. It also shows what Napwake's paused workloads would request if awakeWake on requesta request to a paused workload's Service wakes it. While a workload is paused, the operator adds an EndpointSlice to each Service that selects it,…
a request to a paused workload's Service wakes it. While a workload is paused, the operator adds an EndpointSlice to each Service that selects it, pointing at the doorman, a new Deployment (two replicas) that runs the operator image with
--doorman. The doorman holds each TCP connection, wakes the workload through thenapwake.dev/last-requestannotation, and passes the connection to a Ready pod, or closes it afterwake.maxWait(default 2m) with aRequestNotServedwarning event. A workload woken this way recordslastActivitySource: request. On by default for workloads Napwake pauses itself; keep the doorman off a port withnapwake.dev/doorman-ignore-portson the Service, or disable it cluster-wide with the Helm valuedoorman.enabled: false. Services in GKE container-native load balancing (NEGs) aren't routed unless the optional forwarder is on, with anUnsupportedLoadBalancerwarning; see the compatibility page for the ingress controllers, Gateway API implementations, and load balancers checked. TheWakeOnRequestcondition reports whether a workload is routed;napwake_doorman_wakes_total,napwake_doorman_wait_seconds,napwake_doorman_held_connections, and theNapwakeDoormanWakesFailingalert cover it. The operator's ClusterRole gains read access to Services and full access to EndpointSlicesWorkload dependenciesspec.dependsOn lists the workloads a ManagedWorkload needs, in its own or another namespace. A dependency isn't paused while anything that depends on…
spec.dependsOnlists the workloads a ManagedWorkload needs, in its own or another namespace. A dependency isn't paused while anything that depends on it is awake (HeldByDependents), waking a workload wakes its dependencies, andwaitForReady: trueholds a resume until a dependency's pods are Ready (WaitingForDependencies). Cycles stop both workloads from pausing (DependencyCycle); a dependency that doesn't exist is reported (DependencyNotFound) but doesn't blockPrometheus activity sourcesidlePolicy.activity.prometheus queries count as activity when they return a value above zero, such as an ingress request rate. Configure the endpoint…
idlePolicy.activity.prometheusqueries count as activity when they return a value above zero, such as an ingress request rate. Configure the endpoint with--prometheus-url(Helm valueprometheus.url); path-prefixed endpoints (Thanos, Mimir, reverse proxies) work as-is. If a query can't be evaluated, thePrometheusAvailablecondition reportsEndpointNotConfiguredorQueryFailedand the workload isn't pausedPrometheusQueryEmptycondition, reasonNoSeries, and a warning eventa Prometheus activity query has matched no series since it was first evaluated, as a misspelt metric or label gives, so it can never keep the workload…a Prometheus activity query has matched no series since it was first evaluated, as a misspelt metric or label gives, so it can never keep the workload awake. It's informational: the other sources still decide.
kubectl napwake statuslists it under Needs attentionMetricsAvailablereasonNoCPURequests: idle detection doesn't act on a workload whose CPU utilization can't be measurednapwake_idle_seconds{namespace, workload}: time since the workload's last activitykubectl napwake scan names the Prometheus its history came from, and how much it replayed, in a History: line and the report's subtitle, and lists any…
kubectl napwake scannames the Prometheus its history came from, and how much it replayed, in aHistory:line and the report's subtitle, and lists any others it found (JSONhistory.others). With several in the cluster it reads the one with the most of the window's CPU history, and--prometheus-service namespace/name[:port]reads the one you name through the API server, such as a Thanos Query or GKE's managed Prometheus frontend it wouldn't recognise. The docs say how to read Google Cloud Managed Service for Prometheus with an access tokenVersioned docsdocs.napwake.dev keeps the docs of each minor release, such as /0.2/, built from its tag so their install commands name that release, with a version…
docs.napwake.dev keeps the docs of each minor release, such as
/0.2/, built from its tag so their install commands name that release, with a version selector.latest, where the site's root goes, is the newest release's, andnextthe newest release candidate's, with a banner saying so. A link without a version lands on the same page inlatest. The chart's NOTES, and each GitHub release's notes, link to the docs of their versionSetting the time zone is part of installing: the install commands set
timezone, the installation guide says why, and the chart's NOTES name the zone the forecast counts hours and weekdays in, asking for the users' while it's UTCThe chart's NOTES tell teams whose workloads Argo CD or Flux applies to have it leave replicas to Napwake before
enable, with a link to the GitOps guide, since otherwise it scales each workload back up after every pauseInstalling with Kustomizethe installation guide uses the Kustomize manifests as a remote base pinned to the release,…
the installation guide uses the Kustomize manifests as a remote base pinned to the release,
https://github.com/napwake/napwake//config/default?ref=v0.2.0, setting the operator's time zone in your kustomization, withconfig/prometheusandconfig/network-policyfor what the chart's values turn on. The docs callconfig/default, andinstall.yaml, which is built from it, the Kustomize manifests throughout, and give each base as a pinned link rather than a path in a clone of the repositoryInstalling with Argo CD or Flux: the installation guide has an Argo CD
Applicationfor the OCI chart (repoURL: ghcr.io/napwake/charts,chart: napwake, nooci://), and a FluxOCIRepositoryandHelmReleasethat keep the release namednapwakeinnapwake-systemMoving a Helm install to Argo CD or Fluxthe installation guide hands an install made with helm install to either where it runs, with the same values, without restarting the operator or…
the installation guide hands an install made with
helm installto either where it runs, with the same values, without restarting the operator or waking paused workloads, rather than uninstalling, which the uninstall guard refuses while ManagedWorkloads are left. Argo CD shows the ApplicationSyncedwithout applying anything, so it's synced once by hand, which stamps its tracking annotation, and Helm's release Secrets are deleted; a FluxHelmReleasewith the release'sreleaseNameandstorageNamespaceupgrades it in placeOpting in a whole namespace from Gitthe opt-in guide's "A whole namespace" has an Argo CD tab, with the label in the Application's managedNamespaceMetadata, for a namespace it creates…
the opt-in guide's "A whole namespace" has an Argo CD tab, with the label in the Application's
managedNamespaceMetadata, for a namespace it creates withCreateNamespace=true, and what it takes for a namespace that exists already, and a Flux tab, with the Namespace in the kustomization. The GitOps guide links to itbrew install napwake/tap/kubectl-napwakeinstalls the plugin on macOS and Linux, with no krew needed.The release workflow writes the formula from each release's published archives, and each release candidate's until the first stable release, and…The release workflow writes the formula from each release's published archives, and each release candidate's until the first stable release, and pushes it to the napwake/homebrew-tap repository. The README and the plugin's page install it with Homebrew first, then with krew, linking krew's own setup, then by a download for any platform
The plugin's usage and help name it as it's run,
kubectl napwake, as krew asks of plugins, rather than by its binary's nameThe krew manifest is ready for krew-index: it describes every command,
nodesincluded, and its caveats say which need the operator. The contributing guide gives the submission, after 0.2.0, and the update each later release needskubectl napwake status --htmlwrites the status as a web page to share, in the scan report's stylethe summary, what pausing freed and saved this month and what dry-run would have freed, what needs attention, every workload's state and what comes…the summary, what pausing freed and saved this month and what dry-run would have freed, what needs attention, every workload's state and what comes next in a table to sort and filter, and recent pauses and wakes. Given a file it writes there, given none to a new temporary file, readable only by you either way, and opens it in the browser from a terminal unless
--open=false. See Share it as a web pageDry-run's estimate where people lookkubectl get managedworkloads gains a Would Free column (status.dryRun.estimatedFreed, blank for a workload not in dry-run), and lists Freed before…
kubectl get managedworkloadsgains aWould Freecolumn (status.dryRun.estimatedFreed, blank for a workload not in dry-run), and listsFreedbeforeSaved.kubectl napwake statusleads with it: its summary says what pausing the dry-run workloads would have freed since they started measuring, and aWOULD FREEcolumn gives each one's (JSONwouldFreeanddryRunSince).estimatedFreedcounts the would-be pause under way, as of the last status write, so a workload idle all weekend shows what it would have freed before Monday's activity ends the pause;sleptand the freed hours still count finished onesstatus.prediction.timeZone: the time zone the forecast counts hours and weekdays in, the operator's--timezone.kubectl napwake statusshows it (JSONforecastTimeZones), and, when it's UTC, the default, how to set it to your users' zoneGitOps check in
kubectl napwake pauseandenablea GitOps tool that sets a workload's replicas from Git will undo its pauses, and they warn, with the one-time fix, only when it isn't told to leave…a GitOps tool that sets a workload's replicas from Git will undo its pauses, and they warn, with the one-time fix, only when it isn't told to leave them. Argo CD keeps owning replicas it's told to ignore, so for it they read the Application that applies the workload, by its tracking annotation or label, and
argocd-cm: replicas ignored in either, withRespectIgnoreDifferences=truein the Application's sync options, are left to Napwake, and the warning gives only the step that's missing. When they can't be read, such as without access to Argo CD's namespace, the warning says why and to make the fix "if you haven't already".enablewarns whatever it does, ends dry-run, finds it ended, or leaves it to Git, and--allwarns once for workloads a tool treats alike. See GitOps tools
Changed 45
Behaviour that’s different now.
spec.wake.mode: Request | Ask replaces spec.wake.onRequest, and napwake.dev/wake: request | ask replaces the wake-on-request annotation, with a…
spec.wake.mode: Request | Askreplacesspec.wake.onRequest, andnapwake.dev/wake: request | askreplaces thewake-on-requestannotation, with a cluster default,defaults.wake(--default-wake). A value ofnapwake.dev/wakethat can't be read givesask, with anInvalidSettingwarning, so a typo never lets crawlers wake a workload meant to wake only for people. Waking can no longer be turned off for one workload, and theWakeOnRequestreasonTurnedOffis gone: on a public URL, have the doorman ask; to keep the doorman off a port, usenapwake.dev/doorman-ignore-ports; for the whole cluster,doorman.enabled: false. A ManagedWorkload's storedwake.onRequestis dropped once the new CRD is installed, and the old annotation is ignored, but applying a manifest that still sets it is refused, bykubectl apply's default strict field validation (unknown field "spec.wake.onRequest") and by a server-side apply, as a GitOps sync's can be (field not declared in schema): take it out of your manifests in Git as you upgrade, and putmode: Askwhere it wasfalse; see Upgrading. The field ismode, noton, since YAML 1.1, which kubectl and Helm read manifests with, takes an unquotedonfortrueA Prometheus activity query now wakes a paused workload.This is new behaviour for every workload that has idlePolicy.activity.prometheus queries: they used to be run only while it was awake, to keep it…
This is new behaviour for every workload that has
idlePolicy.activity.prometheusqueries: they used to be run only while it was awake, to keep it awake. Now they're run every minute while it's paused too, and one that shows activity wakes it, with aWokeByActivityevent,a Prometheus activity query shows activity, waking, andlastActivitySource: prometheus, and wakes its dependencies as any wake does. So a workload no Service routes requests to, such as a queue worker with a query of its queue's length, wakes when messages wait, rather than staying paused, its queue filling, until someone wakes it by hand. A query already showing activity as a requested pause begins, such as a queue with messages still on it, doesn't undokubectl napwake pause: it wakes the workload only once it has been quiet and shows activity again, whichstatus.pause.queryActiveSincePauserecords. A query that can't be evaluated never wakes it, andPrometheusAvailable=Falsesays so, worded for a paused workload; behind the doorman, its paused hours are learned through requests meanwhile, as without queries. The forecast learns the hours a paused workload's queries watch, quiet while they find nothing, so a worker paused every night now learns whole days and acts after 8 days, as one behind the doorman does; before, it never acted. See While the workload is pausedA forecast that can't learn while its workload is paused says why, and how to fix itstatus.prediction.message, and kubectl napwake status FORECAST and forecast, give the cause from the WakeOnRequest condition's reason, such as not…
status.prediction.message, andkubectl napwake statusFORECAST andforecast, give the cause from theWakeOnRequestcondition's reason, such asnot learning while paused: no Service routes requests to it, then the fix and a link to the section of the wake-on-request page that explains it, in place ofnot learning: paused without wake on request. While a paused workload is learning, its message says from what: requests through the doorman, its Prometheus activity queries, or bothThe forecasting page opens with what the forecast doesit learns for every managed workload whatever autoResume says, and holds off pauses on its own; with autoResume it also wakes the workload 15 minutes…
it learns for every managed workload whatever
autoResumesays, and holds off pauses on its own; withautoResumeit also wakes the workload 15 minutes ahead of demand, and turning that on later loses nothingNapwake only pauses what's running, and only starts what it stoppeda workload scaled to zero by someone else (a person, a pipeline, or KEDA) is left at zero. It isn't paused, woken by a request or activity, or routed…
a workload scaled to zero by someone else (a person, a pipeline, or KEDA) is left at zero. It isn't paused, woken by a request or activity, or routed to the doorman, and reports the new
ScaledToZerocondition. A pause recorded at zero replicas is handed back at zero, and hand-back events say how many replicas the workload was left atstatus.cost covers the current calendar month in UTC, and everything is priced on what the workload's pods request, sidecars included, not on what…
status.costcovers the current calendar month in UTC, and everything is priced on what the workload's pods request, sidecars included, not on what they use. Rates come fromcostTracking.rates, then the list price of the workload's nodes, then the defaults, and are applied when the figures are worked out, so a rate change reprices the month. The hours are kept to a billionth of an hour; time is settled at every phase change, and a gap of more than 2 hours counts as 2. Projections arependinguntil a day of the month has been tracked. Freed becomes money only once nodes go, which saved measuresThe forecast is an additive double-seasonal Holt-Winters model, stable at zero demand, that learns each hour's peak CPU as a share of the app…
The forecast is an additive double-seasonal Holt-Winters model, stable at zero demand, that learns each hour's peak CPU as a share of the app containers' requests, so the veto,
autoResumeand what a wake teaches compare the same thing whatever sidecars are injected. Confidence is 1 − WAPE, daily over the last 24 scored hours and weekly over 168. A phase needs every hour of the day, or of the week, observed, and is lost 5 points below the threshold; a regime change demotes it one level below where it was before the anomalies began, never above where it is, and discards its evidence, and the forecast doesn't act again until a full day has been scored. A large spike that recurs in the same hour of the week is learned as a pattern rather than clipped as an outlier. Missed hours are skipped, not filled in.prediction.confidence, when set, must be at least 50spec.targetcan't be changed once set, since a paused target would be left at zeroidlePolicy.idleAfter and wake.maxWait take only durations Go can read, such as 90m or 2h; 1d is rejected, where it used to be stored and then break…
idlePolicy.idleAfterandwake.maxWaittake only durations Go can read, such as90mor2h;1dis rejected, where it used to be stored and then break every read of ManagedWorkloads in the cluster.wake.maxWaitis at most 1h.spec.predictioncan be left outCPU activity is measured against the requests of a running pod of the current revision, for the app containers, so requests a LimitRange or VPA sets…
CPU activity is measured against the requests of a running pod of the current revision, for the app containers, so requests a LimitRange or VPA sets count, a template that sets only limits works, and evicted or older pods don't skew it. A workload scaled to zero is idle, not unmeasurable. The forecast reports a phase that acts only while it acts, so status doesn't say
Activewhile a regime change has it holding back, and an hour in which the workload was seen in use counts as busy, soautoResumewakes a lightly used workload every morning, not every other oneidlePolicy.activity.cpuThresholdis 1 to 100; 0 is rejectedThe chart installs the ManagedWorkload CRD as a template, upgrades it with each
helm upgrade, and keeps it onhelm uninstall(crds.install: falseopts out)Logs are JSON by default (
logEncoder,--zap-encoder).image.tag defaults to the chart's appVersion. Memory defaults rise to 128Mi requested and 512Mi limit for the operator, and 64Mi and 256Mi for the…image.tagdefaults to the chart'sappVersion. Memory defaults rise to 128Mi requested and 512Mi limit for the operator, and 64Mi and 256Mi for the doorman, with Go's soft memory limit set to 90% of the container's (GOMEMLIMIT, if set, wins). The doorman caches only the EndpointSlices the EndpointSlice controller writes, cut down to what it reads, and its Service prefers dual-stack, so on a dual-stack cluster both families' Services are routedmetrics.secure: falseserves plain HTTP, and secure metrics can be scraped by subjects inmetrics.readerSubjects.NapwakeDownmatches only the operator's scrape job, andNapwakeDoormanWakesFailingcounts onlytimeoutanderrorresultsThe doorman's pods inherit the operator's nodeSelector, tolerations and affinity unless doorman.* sets its own, spread across nodes and zones, and…
The doorman's pods inherit the operator's
nodeSelector,tolerationsandaffinityunlessdoorman.*sets its own, spread across nodes and zones, and keep one replica up through drains. On shutdown a doorman pod reports not Ready at once, keeps accepting connections for 15 seconds so the operator can move paused workloads' Services to the new pods, then gives what's left of 25 seconds to the rest, within a 40-second termination grace period; a rollout waits 5 seconds after a new pod is Ready before stopping an old one. The operator gives up its leader lease as it stops, so a new one takes over at onceWake on request works with
watchNamespaces: the operator always reads the doorman's own EndpointSlices, through a Role in the release namespaceA Service is routed to the doorman only while none of its pods is Ready, so a Service shared with workloads that are running keeps serving from them.The operator reads the Service's own EndpointSlices from the API server, rather than caching every one in the cluster, and looks again every 30…
The operator reads the Service's own EndpointSlices from the API server, rather than caching every one in the cluster, and looks again every 30 seconds while other pods serve it
The label opt-in finds a workload's ManagedWorkload by its target, names a new one <name>-<kind> when the workload's name is taken, and never…
The label opt-in finds a workload's ManagedWorkload by its target, names a new one
<name>-<kind>when the workload's name is taken, and never retargets one. It keeps what no annotation sets:prediction.confidence,costTracking, the Prometheus activity queries, andwaitForReadyon a dependency the annotation still lists, sokubectl patchworks on a label-created ManagedWorkloadA paused workload wakes on any change to napwake.dev/last-activity, last-request or active-until, on the ManagedWorkload or its target, since the…
A paused workload wakes on any change to
napwake.dev/last-activity,last-requestoractive-until, on the ManagedWorkload or its target, since the pause began, whatever time the value states, so a clock behind the operator's can't lose a wake. Paused workloads are looked at at least every 5 minutes, and 15 minutes before each hourDry-run never reduces availability: a pause requested under dry-run is counted as a would-be pause instead, and a paused workload switched into dry-run is woken (
DryRunWake)Prometheus activity queries count any series above zero;
NaNand infinite values are ignored, so an all-NaNor empty result is no data and the other sources decide. Scalars are accepted, range vectors and responses over 4 MiB rejectedkubectl napwake scan leaves the cluster's own infrastructure out of what it says could be freed and of its next steps: kube-*, Napwake's namespace,…
kubectl napwake scanleaves the cluster's own infrastructure out of what it says could be freed and of its next steps:kube-*, Napwake's namespace, and well-known platform namespaces and charts such as monitoring, ingress controllers, cert-manager, Argo CD and Flux, listed in a note. Prometheus history gets part of--timeout, and a namespace whose history doesn't arrive is judged from CPU right now with a note, rather than the scan failing. A workload with no CPU metrics but with history is judged from its historykubectl napwake status and scan read every namespace you can by default, and fall back to the context's namespace with a note when you can't list them…
kubectl napwake statusandscanread every namespace you can by default, and fall back to the context's namespace with a note when you can't list them all.statussays in its NEXT column why a workload won't pause, shows a hand-written ManagedWorkload's name in brackets, and counts last month's freed and saved as noneThe scan's headline share of what could be freed is measured against what the same workloads cost over the same time, for replayed history
kubectl napwake scanreads each namespace in a few paged calls, and says what it couldn't read. Workloads already scaled to zero by hand count toward nothing it says could be freedKubernetes 1.30 and later is supported, and CI runs the end-to-end tests on 1.30 and 1.37. Kubernetes 1.26 to 1.29 are past their end of life upstream and with every cloud provider
Costs, and what pausing frees and saves, are priced at the list price of the nodes each workload runs on, from their instance type and region, for…
Costs, and what pausing frees and saves, are priced at the list price of the nodes each workload runs on, from their instance type and region, for every AWS, Google Cloud, and Azure instance type, instead of AWS us-east-1 rates for everything: on-demand, or a spot or preemptible node's spot price (Google Cloud's and Azure's published regional price, AWS's average across the region's zones), with on-demand for a type not sold as spot. The rates are kept in
status.cost.listRates, and which prices they are instatus.cost.listPrice(OnDemand,SpotorSpotAndOnDemand), so what a paused workload frees and saves stays priced where it ran.costTracking.ratesstill overrides them, and nodes without a list price use the previous defaults. The operator reads Nodes' labels for this, so its ClusterRole gainsget,list, andwatchon nodeskubectl napwake scan prices workloads the same way, and says which node types it priced at, on-demand or spot, which nodes had no list price, how many…
kubectl napwake scanprices workloads the same way, and says which node types it priced at, on-demand or spot, which nodes had no list price, how many workloads run on spot nodes, priced at spot, and which run on spot nodes of a type without a spot price, priced on-demand, so cost less than shown.--cpu-priceand--memory-priceoverride node prices; JSON addsnodePrices, with each node type'sspotandatSpot, and each workload'sonSpotandspotAtOnDemandautoResumewakes a workload 15 minutes before an hour the forecast expects to be busy, instead of once that hour has started, so it's Ready when people arriveThe forecast records each hour once it's over, at the peak CPU read in it, instead of one reading as the hour began, and keeps learning while a…
The forecast records each hour once it's over, at the peak CPU read in it, instead of one reading as the hour began, and keeps learning while a workload is paused behind the doorman: an hour spent paused is no demand, since a request would have woken it, and an hour a request woke it in is busy. It learns quiet hours, and reaches confidence sooner, from workloads that spend most of the day paused, and
autoResumelearns to wake a workload before the people who'd wake it arrive. An hour the operator didn't see from start to end, such as one it restarted in, is skipped. Awake workloads without an idle policy are checked every minute instead of hourlyThe forecast's veto holds back a pause for the hour under way as well as the next, so a pause isn't woken straight back by
autoResume; why the veto couldn't judge the forecast is loggedIdle detection is an activity clock.Each ManagedWorkload records its last activity in status.activity, and the idle action runs once there has been none for idlePolicy.idleAfter (default…
Each ManagedWorkload records its last activity in
status.activity, and the idle action runs once there has been none foridlePolicy.idleAfter(default 1h). CPU aboveidlePolicy.activity.cpuThreshold% of requests, a pod template change, and thenapwake.dev/last-activity/napwake.dev/active-untilannotations count as activity; any one keeps the workload awake. There is no learning period: the forecast no longer gates idle detection, and only defers a pause when it is confident demand is comingA paused workload scaled up outside Napwake wakeswhoever did it wants it running, so it goes back to Running and its idle clock starts again (lastActivitySource: scaled-up). status.lastScaledUp…
whoever did it wants it running, so it goes back to Running and its idle clock starts again (
lastActivitySource: scaled-up).status.lastScaledUprecords when, how many replicas, and the field manager that did it, orunknownfor a scale through the scale subresource, which records none. Napwake doesn't manage the replica count of a running workload, so changes by its team or an autoscaler are left alone.napwake_external_scale_ups_total{by}counts them, by the GitOps tool orotherEvents are emitted through the
events.k8s.io/v1API (events.EventRecorder) instead of the deprecatedrecord.EventRecorder. Each event now carries an action (e.g.Pause,EvaluateIdle,CheckReplicas). The operator's ClusterRole gainscreate/patchonevents.k8s.ioevents; the Helm chart and kustomize RBAC include itThe scan report's headline, before Napwake has paused anything, is what pausing could free a month and what the workloads cost, rather than "$0 freed…
The scan report's headline, before Napwake has paused anything, is what pausing could free a month and what the workloads cost, rather than "$0 freed by Napwake this month"; a zero that's down to under a day of history says so, "too little history yet", as the terminal does, and to run the scan again after a few days. What Napwake freed and saved leads once workloads are live
The README's and docs' install and download links name their release, such as releases/download/v0.2.0/install.yaml, instead of releases/latest, so…
The README's and docs' install and download links name their release, such as
releases/download/v0.2.0/install.yaml, instead ofreleases/latest, so each page installs the version it documents.make bumpkeeps them in step, andmake verify-version, which the release workflow runs first, refuses a release whose docs linkreleases/latestor another releasekubectl napwake statussays in NEXT when and why a held-off pause happenspauses at 04:13 UTC (after GitOps conflict) during the hour Napwake waits after a GitOps tool undid a pause, held awake while preview-7/worker uses…pauses at 04:13 UTC (after GitOps conflict)during the hour Napwake waits after a GitOps tool undid a pause,held awake while preview-7/worker uses it,held awake for 2h (active-until), andheld awake by the forecast: busy from 13:00 UTCWakeOnRequest keeps a problem the last pause found with the workload's Services (ServedByOtherPods, NodeLocalTraffic, UnsupportedIPFamily,…
WakeOnRequestkeeps a problem the last pause found with the workload's Services (ServedByOtherPods,NodeLocalTraffic,UnsupportedIPFamily,UnsupportedLoadBalancer,TooManyPorts) once it wakes, its message startingat its last pause:, so it can be fixed before the next pause;kubectl napwake statuslists it under Needs attention, and no longer lists an awake workload'sNotPaused, orDoormanDisabledEvents and condition messages from the operator and the doorman give durations as people say them, such as
90 seconds,5 minutes,1 hour 30 minutesor4 days, instead of Go's5m0sand96h12mThe
IdleDetectedevent says in words what the workload has been idle since, and in dry-run that it would have paused rather than that it pauses:[dry-run] api: idle for 1 hour, since its last rollout; would have pausedThe
DependenciesLearnedevent says of each dependency what the scan says"woken and held with it" only for one Napwake manages, in a namespace that allows the workload's; of one it doesn't manage, that it isn't managed by…"woken and held with it" only for one Napwake manages, in a namespace that allows the workload's; of one it doesn't manage, that it isn't managed by Napwake, so it isn't woken with the workload, and to opt it in too. It used to say every dependency was held and woken
The GitOpsConflict event and condition say in a line what to change and link to the tool's section of the GitOps guide, instead of carrying the YAML…
The
GitOpsConflictevent and condition say in a line what to change and link to the tool's section of the GitOps guide, instead of carrying the YAML setting, whichkubectl describeflattened onto one line:To stop it, have Argo CD ignore replicas, as for an HPA: https://docs.napwake.dev/guides/gitops/#argo-cdThe kubectl plugin page opens with a Which Command tablewhat scan, status, forecast, nodes, enable, pause, wake, deps and version are each for, and which need the operator, starting with how scan and status…
what
scan,status,forecast,nodes,enable,pause,wake,depsandversionare each for, and which need the operator, starting with howscanandstatusdiffer. The README points to itThe dry-run guide says up front what dry-run runsthe activity clock, dependency holds and the forecast run as they do live, and only the scale to zero is skipped, so what it can't show is what…
the activity clock, dependency holds and the forecast run as they do live, and only the scale to zero is skipped, so what it can't show is what happens once a workload is paused, how long a wake on request takes and whether a GitOps tool scales it back up. It also says that a request counts in dry-run only through the activity it shows, such as its CPU, and
autoResume's wakes not at all, so a workload's real pauses can be shorter than its would-be ones, and that a Prometheus activity query on its request rate lets dry-run see its requests. The Prometheus guide's ingress-nginx queries name the namespace as well as the Service, since previews often share Service namesInstalling and upgrading with Helm use one values file, napwake-values.yaml, in the README, the installation guide and the Helm values reference,…
Installing and upgrading with Helm use one values file,
napwake-values.yaml, in the README, the installation guide and the Helm values reference, since Helm reuses a release's values only on an upgrade given none: one given--set timezone=...alone, as the chart's NOTES suggested, dropped every value set before, such asprometheus.url. The upgrade guide's Keeping your settings says why, writes the file from a release installed with--set(helm get values), changes a setting such as the time zone at the version installed, and gives--reuse-valuesfor an upgrade without the file, and what it keeps of the installed version. The chart's NOTES, asking for the users' time zone, give the command that sets it at the installed version with--reuse-values, which every Helm the chart supports has, and keeps the other values
Fixed 106
What used to go wrong.
In dry-run, an operator restart no longer ends every would-be pause with
activity resumed (unobserved).A gap in watching doesn't wake a paused workload, so it no longer ends the would-be pause that stands for one either; activity seen after the gap…A gap in watching doesn't wake a paused workload, so it no longer ends the would-be pause that stands for one either; activity seen after the gap still does. Before, every upgrade, and on spot nodes every preemption, split each would-be pause in two and added an hour awake, so dry-run understated what pausing would free
An operator restart never brings a pause forward.It restarted an awake workload's idle clock only when what's written was over 2 minutes old, which nearly every restart found, and otherwise went on…
It restarted an awake workload's idle clock only when what's written was over 2 minutes old, which nearly every restart found, and otherwise went on from what's written, which can be missing up to a tenth of
idleAfter, at most 5 minutes, of activity the operator saw since, so the workload could pause that much early. A restart now always restarts an awake workload's clock, with anUnobservedevent: it can delay a pause by up toidleAfter, as before, but never bring one forward. A paused workload and a dry-run would-be pause go on through itA gap in watching is no longer called activity.When one restarts a clock, an Unobserved event says so and for how long: Napwake wasn't watching for 9 minutes, so its idle clock starts again, or…
When one restarts a clock, an
Unobservedevent says so and for how long:Napwake wasn't watching for 9 minutes, so its idle clock starts again, orcouldn't read its activitywhile a source was out, and after a restartthe operator restarted, or another took over, so its idle clock starts again; it wasn't watching for up to 4 minutes. It comes with the first check that can read every source, so a restart whose first check can't is still announced.kubectl napwake statusshowsunobserved until 3m agoin LAST ACTIVITY, the scannot idle yet: Napwake wasn't watching until 3m ago, andIdleDetectedsays the workload was idlesince Napwake could watch it again after a gap. Likewise the start of managing a workload, and its waking, which start the clock but aren't activity: LAST ACTIVITY showsnone since Napwake began managing it 3m agoornone since it woke 3m ago, notcreated, 3m ago, and the scannot idle yet: Napwake began managing it 3m agoornot idle yet: it woke 3m ago, notactive 3m ago (created)kubectl napwake status shows a would-be pause that has freed less than a cent as <$1, in WOULD FREE and the dry-run summary, not $0, which it keeps…
kubectl napwake statusshows a would-be pause that has freed less than a cent as<$1, in WOULD FREE and the dry-run summary, not$0, which it keeps for a workload still waiting for its first: the operator's estimate is to the centA dependency whose own activity restarts its clock no longer goes on saying it's kept awake for its dependents:
HeldByDependentsturnsFalse, reasonNotIdle, andstatusshows when it pauses rather than a hold from beforekubectl napwake forecast replays each workload with what would wake it once it's paused, rather than as if the doorman routed every workload: a…
kubectl napwake forecastreplays each workload with what would wake it once it's paused, rather than as if the doorman routed every workload: a request through the doorman when one of its Services can be routed, its Prometheus activity query when it has one, both, or neither, when people wake it by hand and the forecast doesn't learn the hours it's paused. Whether the doorman is on, and its NEG forwarder, are read from the flags of the operator that manages the workload, so withdoorman.enabled: falseno request wakes it, and with the forwarder on its Services behind GKE NEGs do. The table says which in a new WAKES ON column, a named workload's replay in a sentence, and the JSON inforecast.wakes, with why a request wouldn't wake itIn dry-run, a pause asked for while a Prometheus activity query shows activity holds the query back as a real pause would: the query ends the would-be…
In dry-run, a pause asked for while a Prometheus activity query shows activity holds the query back as a real pause would: the query ends the would-be pause only once a check has found it quiet and it shows activity again, rather than at the next check. Activity the still-running workload makes itself, such as the CPU of draining its queue, still ends the would-be pause, which a real pause, with nothing draining the queue, wouldn't.
status.dryRun.queryActiveSincePauserecords itThe forecast learns from live hours again.It had learned nothing since a workload's history was read, or since an operator restart: data points never rose, no PredictionFed event appeared, and…
It had learned nothing since a workload's history was read, or since an operator restart: data points never rose, no
PredictionFedevent appeared, and no forecast could ever act. The operator worked out the start of the hour from the system clock, and Go compares two such times by the clock reading it keeps for measuring durations, which differs by nanoseconds from one look to the next. So each look seemed to start a new hour, and no hour was ever seen whole. Tests and the simulator use a clock without that reading, which hid it; the forecast's tests now read the clock as the operator doesDry-run measures a whole app's would-be pausesa dependent in dry-run that would have paused no longer holds its dependencies awake when they're in dry-run too, so each of them goes on to its own…
a dependent in dry-run that would have paused no longer holds its dependencies awake when they're in dry-run too, so each of them goes on to its own would-be pause, as it would live. Before, a frontend idle for hours held every service behind it "awake", and dry-run counted only the frontend. A dependency that pauses for real stays held by a dry-run dependent, whose pods are still up. When the dependent's would-be pause ends, its dry-run dependencies are stamped with activity, as its wake would have woken them, so theirs end too and their clocks restart
On GKE, a paused workload's Services are routed to the doorman, so requests wake itGKE annotates every Service of a VPC-native cluster cloud.google.com/neg: '{"ingress": true}', and every one was refused as UnsupportedLoadBalancer.…
GKE annotates every Service of a VPC-native cluster
cloud.google.com/neg: '{"ingress": true}', and every one was refused asUnsupportedLoadBalancer. That value only asks for container-native load balancing should a GKE Ingress use the Service. A Service is now left off the doorman only when GKE's NEG controller reports NEGs of its pods (cloud.google.com/neg-status), it asks for standalone NEGs (exposed_ports), or an Ingress GKE serves (kubernetes.io/ingress.classgce,gce-internal,gce-regional-externalorgce-multi-cluster, or no class at all) uses it, and it's taken off or put back as soon as one of those changes. The operator's role gainsget,listandwatchon IngressesA wake or a rollout no longer warns NoPodMetrics, telling you to check metrics-server, while the new pods simply have no metrics yet: MetricsAvailable…
A wake or a rollout no longer warns
NoPodMetrics, telling you to check metrics-server, while the new pods simply have no metrics yet:MetricsAvailableisUnknown, with no event, while none of the target's pods is Ready (NoReadyPods) and for 3 minutes after one is (AwaitingPodMetrics). Only Ready pods that go on without metrics areNoPodMetrics, with the warning, which now gives thekubectl top podscommand to checkThe forecasting page's How Long It Takes table, now measured by the simulator and checked against it in CI, was wrong for a pattern repeated every…
The forecasting page's How Long It Takes table, now measured by the simulator and checked against it in CI, was wrong for a pattern repeated every day, weekends included: the forecast acts after about 8 days, not 3. The page also says now what leaving
--timezoneat UTC costs when the clocks changeA new forecast has the workload ready on Mondays once it actsit learned each hour of the week half at a time, so for weeks it predicted weekday demand on Saturdays, lost its confidence every weekend, and the…
it learned each hour of the week half at a time, so for weeks it predicted weekday demand on Saturdays, lost its confidence every weekend, and the first request woke an office-hours workload on the Mondays of its second, third and fourth weeks. The first week now sets what the forecast expects of each hour of the week, as the first season does in classical Holt-Winters, and isn't scored, since there was nothing to forecast it from; an hour of the week never seen isn't acted on. Every pattern the forecast can get right now acts after 8 days at every setting (
cautiouswaited 15 on office hours) and is fully active after 14, and no setting wakes a workload on its first weekends.status.prediction.reasonisObservingthrough the first week, and its message counts the hours of the week seen. See How Long It TakesA weekday holiday no longer costs a second cold morning a week laterits quiet hours were learned straight into that weekday, so the forecast lost its confidence on the same weekday the week after. A break from the…
its quiet hours were learned straight into that weekday, so the forecast lost its confidence on the same weekday the week after. A break from the learned pattern, an anomaly or any hour of the day after a regime change, is now learned into the week only once the same hour the next day breaks the same way, so a lasting change, new hours or a retired workload, is still learned within a week, and one Saturday at work no longer has the forecast wake the workload the Saturday after. The hour after one far quieter than forecast, the forecast neither holds off a pause nor wakes the workload, so a holiday costs an hour and a quarter awake instead of two to three, and loosely kept hours' absent days about half as long. Forecast state is version 5: state of an earlier version is reset with a
ForecastResetevent. See Anomaly detectionkubectl napwake scan counts CPU as activity above --cpu-threshold, as the operator's activity clock does, rather than at it, in its history replay and…
kubectl napwake scancounts CPU as activity above--cpu-threshold, as the operator's activity clock does, rather than at it, in its history replay and in its judgement of current CPU, which also compares the share before rounding it to a whole percent. One rule, shared by the operator, the forecast, the simulator and the scan, decides itA new workload's history is read again two minutes after a failed attempt, as documented, rather than on each of the updates a new workload gets in…
A new workload's history is read again two minutes after a failed attempt, as documented, rather than on each of the updates a new workload gets in its first minute, which used up all three attempts in seconds and recorded a briefly unreachable Prometheus as
QueryFailedfor good. A failure to read the target or its CPU requests from the API server is retried rather than recorded asNoCPURequests, with its warning, and isn't counted against Prometheus; the activity clock reports it asMetricsUnavailable, with the error, rather thanNoCPURequestsIn dry-run, an activity annotation stamped in the same second a workload went Idle ends the would-be pause, rather than being taken as older than it: annotations are stamped to the second
A StatefulSet whose persistentVolumeClaimRetentionPolicy.whenScaled is Delete is never paused, since scaling it to zero has its controller delete…
A StatefulSet whose
persistentVolumeClaimRetentionPolicy.whenScaledisDeleteis never paused, since scaling it to zero has its controller delete every claim: it reportsProtected=True, reasonPVCsDeletedOnScaleDown, with aProtectedwarning event, and one paused before its policy changed is handed back. SetwhenScaled: Retainto let Napwake pause itHanding back a paused workload a KEDA ScaledObject scales, on deletion, the ignore label, protection, or a scale-up outside Napwake, holds KEDA at the…
Handing back a paused workload a KEDA ScaledObject scales, on deletion, the ignore label, protection, or a scale-up outside Napwake, holds KEDA at the restored replicas while scaling it up, so a ScaledObject that may scale to zero can't take it straight back down before it's released
A paused workload whose ManagedWorkload loses its status, to a backup restored without status or the ManagedWorkload being created again, is no longer…
A paused workload whose ManagedWorkload loses its status, to a backup restored without status or the ManagedWorkload being created again, is no longer left at zero for good, with KEDA holding it there: before scaling a workload to zero, Napwake copies what the pause changed to its
napwake.dev/paused-fromannotation, removed when it's handed back, and takes the pause up again from it (PauseRecoveredevent). The operator's role gainspatchon Deployments and StatefulSets for it. A workload leftPausing,PausedorResumingwithout a pause record is no longer resumed from its live replicas and KEDA value, which a pause had already set to zero; nothing was scaled, so it'sRunningagain (PauseRecordLostwarning event)A workload is Paused once its pods have stopped, not as soon as it's scaled to zero, so it no longer shows Paused while a StatefulSet's pods are still…
A workload is
Pausedonce its pods have stopped, not as soon as it's scaled to zero, so it no longer showsPausedwhile a StatefulSet's pods are still running. Pods that don't stop are waited for as long as their termination grace period and a minute more, one after another for a StatefulSet; then it'sPausedwithPauseIncomplete=Trueand aPauseIncompletewarning event, until they've gone.status.pause.pausedAtis written when it's scaled to zeroSomeone scaling up a workload Napwake has scaled to zero while it's still Pausing wakes it, as for a Paused one, instead of the pause scaling it back…
Someone scaling up a workload Napwake has scaled to zero while it's still
Pausingwakes it, as for aPausedone, instead of the pause scaling it back down; and someone scaling aResumingworkload back to zero after the resume scaled it up (status.pause.resumedAt) leaves it off, withScaledToZero, instead of the resume scaling it up againGitOpsConflict is no longer cleared as resolved by a wake after a pause of a few minutes, which a GitOps tool syncing less often hadn't yet had the…
GitOpsConflictis no longer cleared as resolved by a wake after a pause of a few minutes, which a GitOps tool syncing less often hadn't yet had the chance to undo: like a pause that stays, one Napwake wakes must have held for the hourDeleting the ManagedWorkload of a paused workload hands it back oncewhile its doorman ports drain, it's no longer scaled up again on every reconcile, over someone who had scaled it to zero meanwhile, nor are its…
while its doorman ports drain, it's no longer scaled up again on every reconcile, over someone who had scaled it to zero meanwhile, nor are its ScaledObject and dependencies touched again, and the
Resumedevent is no longer lost when there are ports to drainA ManagedWorkload that becomes the
DuplicateTargetwhile it holds its workload paused hands it back, with its doorman routes, instead of leaving it at zero with nothing to wake itA ManagedWorkload deleted without its finalizer running, or deleted and created again under the same name, no longer leaves its forecast engine, activity clock, routing backoff and doorman port claims in the operator's memory
A condition message is cut to the 32KiB the API server allows, and an event's note to 1KiB, ending in
…, so one that lists many workloads can't fail every status write, or have its event droppedThe CRD rejects an idlePolicy.idleAfter under 1m, which paused a workload about as soon as each wake finished, negative costTracking.rates, more than…
The CRD rejects an
idlePolicy.idleAfterunder1m, which paused a workload about as soon as each wake finished, negativecostTracking.rates, more than 10idlePolicy.activity.prometheusqueries, and more than 20dependsOn. Thenapwake.dev/idle-afterannotation and--default-idle-after(defaults.idleAfter) take at least1mtoo, and anapwake.dev/depends-onlisting more than 20 uses the first 20, with anInvalidSettingeventA change to a Deployment, a StatefulSet or a ManagedWorkload no longer lists and copies every ManagedWorkload in the cluster to find the ones it…
A change to a Deployment, a StatefulSet or a ManagedWorkload no longer lists and copies every ManagedWorkload in the cluster to find the ones it concerns: they're found through indexes on their target and dependencies, a ManagedWorkload's own status writes, such as cost and activity flushes, no longer re-queue its related workloads unless its phase, spec or dependencies changed, and one dropped from
dependsOnre-checks the hold it took part inReconciling a workload no longer lists and copies every ManagedWorkload in the cluster, several times over, to decide whether it's held awake, which…
Reconciling a workload no longer lists and copies every ManagedWorkload in the cluster, several times over, to decide whether it's held awake, which dependencies to wake and wait for, and whether another ManagedWorkload manages its target: the operator keeps the dependency graph up to date from its watch, and finds a duplicate through the target index. A deleted workload's doorman slices are listed from the API server when they're removed, not again on each reconcile while its ports drain. With 5000 ManagedWorkloads, deciding a hold went from about 5ms and 17MB allocated to under 2µs and about 1KB
A wake is no longer queued behind the periodic checks of every awake workloadreconciles are queued by priority, with a request the doorman holds, an activity or pause annotation, a scale-up from zero and a resume under way…
reconciles are queued by priority, with a request the doorman holds, an activity or pause annotation, a scale-up from zero and a resume under way first, and periodic checks last.
--max-concurrent-reconciles(maxConcurrentReconciles) defaults to 8, since most of a reconcile is spent waiting on metrics and PrometheusA resume of a workload that wasn't wholly Ready before its pause, such as one with a pod crash-looping, finishes instead of staying in Resuming with…
A resume of a workload that wasn't wholly Ready before its pause, such as one with a pod crash-looping, finishes instead of staying in
Resumingwith KEDA held; readiness is judged only from a status the workload's controller wrote for the scale-upA dependency learned from a single request no longer ties two workloads together for good: one learned from a wake is forgotten after 14 days with no request confirming it (
lastSeen)A namespace-level
napwake.dev/depends-onno longer makes the dependency depend on itself, which held it awake as a cycle;napwake.dev/managed: "false"on a workload opts it out without a warningA doorman port stays draining across an operator restart or leader change (status.doormanDraining), so it's never given to another workload while…
A doorman port stays draining across an operator restart or leader change (
status.doormanDraining), so it's never given to another workload while clients may still send the first one's traffic to it; a deleted ManagedWorkload stays until its ports have drained, at most a minuteWake on requestremoving napwake.dev/last-request from a paused ManagedWorkload no longer stops requests waking it; a browser sending more than 16KiB of headers, such…
removing
napwake.dev/last-requestfrom a paused ManagedWorkload no longer stops requests waking it; a browser sending more than 16KiB of headers, such as SSO cookies, gets the waking-up page; a request whose lines end in a bare LF is read as HTTP, so the probe and hidden-file rules apply to it; a Service withinternalTrafficPolicy: Localis left off the doorman (NodeLocalTraffic), since most clients couldn't reach it; and an orphaning delete right after routing no longer leaves a slice behindThe doorman drops a deleted workload's wake series, and the operator a ManagedWorkload's series when its finalizer was removed by hand.
napwake_idle_secondskeeps counting while a workload is pausedDry-run counts a would-be pause that ends because the workload was scaled to zero, ignored, or protected, and no longer panics for a workload whose activity was never measured
A pause's resource snapshot falls back to the last one when pods can't be read, instead of recording that it freed nothing; the forecast keeps…
A pause's resource snapshot falls back to the last one when pods can't be read, instead of recording that it freed nothing; the forecast keeps learning after an hour recorded by a clock that ran fast; and its state is encoded only when it changes
The ServiceMonitors keep the metrics' own
namespacelabel (honorLabels), so alerts and per-namespace queries see the workload's namespace, not Napwake'skubectl napwake enablerefuses, and says why, for a workload Napwake won't managein a protected namespace, opted out, not yet given a ManagedWorkload, or with the operator not installed; asks for the kind when a Deployment and a…in a protected namespace, opted out, not yet given a ManagedWorkload, or with the operator not installed; asks for the kind when a Deployment and a StatefulSet share a name; and
pause's forecast question no longer counts against--timeout.wake --forkeeps a longer hold already set;depsshows a dependency it can't read as unknown;statusfails for a namespace that doesn't exist and keeps this month's freed and saved for a workload back in dry-runkubectl napwake scan scans a namespace whose ReplicaSets it can't read, counts pods of Deployments' deleted ReplicaSets in their history, resolves…
kubectl napwake scanscans a namespace whose ReplicaSets it can't read, counts pods of Deployments' deleted ReplicaSets in their history, resolves dependencies in time on large clusters, leaves last month's freed and saved out until the operator rolls the month, tells a 403 from Prometheus behind the service proxy from the proxy's own, and rejects a--windowover a year and negative pricesmake release tags a release whose versions are already bumped, dates its changelog section, and refuses a VERSION that isn't a version; a release…
make releasetags a release whose versions are already bumped, dates its changelog section, and refuses aVERSIONthat isn't a version; a release publishes nothing users install until its GitHub release is drafted with every file, so a release that fails can be run again rather than left half published; and the docs are published after each release frommain, but not after a pre-release, which rehearses a release without movinglatestkubectl napwake enableleaves alone, and lists, what the operator reports it never pauses: a ManagedWorkload markedProtected, a target labellednapwake.dev/ignore, or a duplicatekubectl napwake scan keeps Napwake's own operator and doorman out of an allowed namespace's apps, counts infrastructure in namespaces opted in with…
kubectl napwake scankeeps Napwake's own operator and doorman out of an allowed namespace's apps, counts infrastructure in namespaces opted in with the label and keeps its dependencies, matches a pod-template-hash by the characters Kubernetes writes it in, and no longer blames Prometheus when reading the API server used up history's timeLearning a dependency from a wake can't hold up the wake: it gives up after 2 seconds
A pause is recorded before the workload is scaled, so a pause interrupted at any point, by a crash or a conflicting write, is finished or undone from the record, and a resume goes back to the exact replica count
The
napwake.dev/ignorelabel, a protected namespace, and deleting the ManagedWorkload all hand back a paused workload, scaled up and released from KEDA, and remove its doorman routing. A workload whose target is deleted stops routing to the doormanKEDA: a ScaledObject's own
paused-replicasvalue is put back after a pause, and uninstalling KEDA, or deleting the ScaledObject, while a workload is paused no longer blocks its wakeA forecast veto no longer holds back a pause already decided, and doorman routing problems never hold up a pause or a wake: they're retried with backoff from 5 seconds to 5 minutes
A dependency in a namespace outside watchNamespaces, or that can't be read, is reported (DependencyNotFound, reason DependencyNotVisible) and never…
A dependency in a namespace outside
watchNamespaces, or that can't be read, is reported (DependencyNotFound, reasonDependencyNotVisible) and never blocks;waitForReadydoesn't wait for a dependency that can't start (unmanaged at zero replicas); learned dependencies never form a cycle that holds both workloads awake; and a dependency learned from a wake gets a valid statusActivity is written to status promptly enough that a restart can't pause a workload early
Time an activity source couldn't be read no longer counts as idle timea source out for longer than a couple of checks restarts the clock (unobserved), while a shorter blind spot, such as the minute before a woken…
a source out for longer than a couple of checks restarts the clock (
unobserved), while a shorter blind spot, such as the minute before a woken workload's pods have metrics, leaves the last activity as the wake, so a workload whose use shows only in a Prometheus query isn't paused the moment Prometheus comes back from an outage, and the same holds for CPUThe gap in watching the activity clock allows before it restarts scales with idleAfter (half of it, between 2 and 7 minutes), and after a restart or a…
The gap in watching the activity clock allows before it restarts scales with
idleAfter(half of it, between 2 and 7 minutes), and after a restart or a change of leader any gap over 2 minutes restarts it, so an operator outage can't pause a workload with a shortidleAfterthe moment it's backCPU utilization is measured against the requests of the pods that report usage, not of every replica, so a workload with one pod serving and the others Pending or crash-looping, or just after a scale-up, no longer reads as idle
A slow or unreachable Prometheus can't hold up wakesthe operator keeps one client for it, gives each query 2 seconds and a workload's queries 5 together, and once Prometheus fails, or times out on three…
the operator keeps one client for it, gives each query 2 seconds and a workload's queries 5 together, and once Prometheus fails, or times out on three queries in a row, doesn't ask it again for a minute, failing every query at once instead, with the error that failed. The condition message is the same whether a check asked Prometheus or not, so an outage is one status write per workload, not one a minute, and it never shows another workload's query
Likewise for metrics-server: a read of the Metrics API has 2 seconds, and once it fails it isn't read again for a minute, with the same message throughout, which never shows another workload's namespace
An activity annotation that isn't an RFC 3339 time, such as epoch seconds, is reported in an InvalidActivityAnnotation warning event, once per value,…
An activity annotation that isn't an RFC 3339 time, such as epoch seconds, is reported in an
InvalidActivityAnnotationwarning event, once per value, instead of only logged, and no longer wakes a paused workload either, as it never kept an awake one awakeThe forecast doesn't learn the startup after a workload is scaled up outside Napwake, such as with
kubectl scale, as demand: the 15-minute warm-up after a wake counts from that scale-up, not from the pauseCost counts the time before a phase change in the phase it was spent in even when the replicas can't be read then, at what was last read, instead of…
Cost counts the time before a phase change in the phase it was spent in even when the replicas can't be read then, at what was last read, instead of leaving up to 5 minutes awake to be counted as paused and freed
A pause finished after an interruption, with the target already at zero, is priced at the replicas it paused, not as freeing nothing
Azure constrained vCPU sizes, such as
Standard_E4-2s_v5, are priced at their base size's list price spread over their vCPUs, instead of at the default ratesEvents are emitted once, when something changes: forecast vetoes, routing warnings, learned dependencies, and
HeldByDependents, once when awake dependents first keep an idle workload upEvery series of a deleted workload is dropped from the metrics, and the operator reads only the target's own pods and claims, with deadlines on every call
A GitOps tool that set the replicas in the same second as another field manager is reported as the one that woke a paused workload, so the
GitOpsConflictis never missednapwake_reconcile_errors_totalcounts the label opt-in's failed reconciles too, ascontroller="optin", so theNapwakeReconcileErrorsHighalert sees themThe plugin formats negative dollars and durations correctly
kubectl napwake scanno longer hangs when the API server stops answering: each request gives up after 30 seconds, and the scan after--timeout(5m), reporting the namespaces it scanned in time, or failing if it scanned noneWorkloads with an injected sidecar, such as an Istio or Linkerd proxy, can pause.CPU usage was counted for every container in the pod but measured against requests from the pod template, which doesn't include injected sidecars, so…
CPU usage was counted for every container in the pod but measured against requests from the pod template, which doesn't include injected sidecars, so the proxy's background work alone could keep a workload above the CPU threshold. Usage and requests now both cover the template's containers and native sidecars;
kubectl napwake scanmeasures workloads the same wayCost, and what pausing frees, include injected sidecars.What pausing freed was priced on the pod template's requests, which leave out a sidecar injected at pod creation, often as large as the app itself;…
What pausing freed was priced on the pod template's requests, which leave out a sidecar injected at pod creation, often as large as the app itself; they're now priced on a running pod's requests, sidecars included. The operator's ClusterRole gains
getandliston pods, which are read from the API server when a workload pauses and during discovery scansA resume completes when the workload's replicas are Ready, not when its pods exist. The check read the scale subresource, which counts pods that are still starting, such as a database replaying its log
Cost is tracked for running workloads. Cost accumulation ran at the end of a reconcile that the automation step always returned from first, so
status.costwas only ever updated for paused workloadsManagedWorkload status is no longer written on every check.Values that change continuously (cost totals and the activity clock's timestamps) are written at most every 5 minutes, and phase, condition, and…
Values that change continuously (cost totals and the activity clock's timestamps) are written at most every 5 minutes, and phase, condition, and lifecycle changes are written immediately; activity seen between writes is held in memory
Resuming a paused workload restores its previous replica count.Since 0.1.7 the controller created the pause status early to hold the resource snapshot, which made the pauser skip recording the replica count, so…
Since 0.1.7 the controller created the pause status early to hold the resource snapshot, which made the pauser skip recording the replica count, so every resume came back with a single replica
Two ManagedWorkloads targeting the same workload no longer block each other.The duplicate check OR'd the UID tie-breaker in unconditionally, so when the older CR had the larger UID both were marked DuplicateTarget and neither…
The duplicate check OR'd the UID tie-breaker in unconditionally, so when the older CR had the larger UID both were marked
DuplicateTargetand neither managed the target. The oldest CR now always wins, with UID breaking ties only for CRs created in the same secondA blocked duplicate now takes over when the owning ManagedWorkload is deleted. Previously it was never reconciled again, so it stayed blocked until something else touched it
The
DuplicateTargetwarning event fires once when the conflict is detected, not on every recheckWorkloads no longer get stuck in
PausingorResumingafter a transient failure.These intermediate phases were persisted before the action ran, and nothing acted on them afterwards, so one failed API call stranded the workload for…These intermediate phases were persisted before the action ran, and nothing acted on them afterwards, so one failed API call stranded the workload for good. Interrupted transitions are now retried until they finish
Discovery ranks workloads by what pausing them could free, as a number. It compared the formatted dollar strings, so
$9.42ranked above$11.68, and in namespaces over 500 workloads the cap kept the wrong onesThe discovery summary counts every scanned workload, matching the cost and could-free totals, instead of only the first 500
What paused workloads freed prices memory on the per-replica request, like CPU. The snapshot took CPU from requests but memory from live usage, which skewed it and disagreed with discovery estimates
Taking a resource snapshot no longer panics with an integer divide-by-zero when the target has zero replicas
Taking a resource snapshot no longer emits a
TargetNotFoundwarning or rewrites the target condition as a side effectWorkloads no longer sit in
Observingindefinitely with no explanation when CPU metrics can't be read (#10). A newMetricsAvailablecondition reportsNoPodMetricsorMetricsUnavailablewith the cause, and a warning event fires when it first failsA target scaled to zero replicas feeds the forecast an observation of zero demand instead of being treated as missing data
Status conditions now update their reason and message when the cause changes, even if the status stays the same
A change to a Deployment only re-queues ManagedWorkloads that target that Deployment, and likewise for StatefulSets. The target watch matched on name alone, so a Deployment and a StatefulSet with the same name re-queued each other's ManagedWorkloads
A namespace's napwake.dev/depends-on naming several workloads no longer makes them depend on each other, a cycle that kept every one of them from…
A namespace's
napwake.dev/depends-onnaming several workloads no longer makes them depend on each other, a cycle that kept every one of them from pausing. The workloads it names don't take it, and neither do those they depend on by their owndepends-onLearning a workload's dependencies from its environment runs after the wake or pause it's reconciled for, and a failure to, such as a ConfigMap read…
Learning a workload's dependencies from its environment runs after the wake or pause it's reconciled for, and a failure to, such as a ConfigMap read timing out, is logged and retried rather than failing the reconcile, so it never holds up a request-driven wake. A paused workload is only relearned when its pod template or ignore list changes
A dependency that can't be woken, such as one whose ManagedWorkload a policy engine won't let Napwake annotate, no longer keeps its dependent paused,…
A dependency that can't be woken, such as one whose ManagedWorkload a policy engine won't let Napwake annotate, no longer keeps its dependent paused, or keeps it from being handed back or deleted: the dependent goes on, with a
DependencyWakeFailedWarning event. A ManagedWorkload in a namespace being deleted lets go after 5 minutes of failed hand-backs, with aHandBackAbandonedWarning event, so it can't hold the namespace's deletion foreverA resume waiting on a waitForReady dependency that's up but never Ready, such as one crash-looping or whose image can't be pulled, no longer leaves…
A resume waiting on a
waitForReadydependency that's up but never Ready, such as one crash-looping or whose image can't be pulled, no longer leaves the dependent at zero forever. It waits as long as the dependency's DeploymentprogressDeadlineSeconds, or 10 minutes for a StatefulSet, then starts withWaitingForDependencies=False, reasonDependencyWontStart, and a Warning event naming the dependencyTwo workloads that
waitForReadyon each other, directly or through others, no longer wait for each other forever: neither waits for the other, withWaitingForDependencies=False, reasonDependencyCycleA dependent being deleted, a duplicate ManagedWorkload, or one whose target is gone no longer holds its dependencies awake forever
HeldByDependentsnames at most 10 dependents and counts the rest. A dependency with over a thousand awake dependents had a message past the condition's 32768-byte limit, which failed every status writeOpt-in annotation values are read without surrounding whitespace, so a YAML block scalar's trailing newline (
idle-after: |giving"2h\n") is no longer anInvalidSettingkubectl napwake scan names the history it replayed as people say it, the same in the terminal's headline and the report's subtitle, summary and…
kubectl napwake scannames the history it replayed as people say it, the same in the terminal's headline and the report's subtitle, summary and glossary: "the last 30 minutes" rather than "the last 1 hours", and its note says only why that's shorter than--windowkubectl napwake scan's first line counts the namespaces the workloads it lists are in, "14 workloads in 3 namespaces", rather than every namespace it read, infrastructure-only ones included; a scan that finds only infrastructure says sokubectl napwake scan's note on the cluster's infrastructure says apart what it leaves out of the totals and what Napwake would pause: never Kubernetes' own workloads or itself, and anything else, infrastructure included, only once you opt it inkubectl napwake scan says what Napwake does with each dependency it lists, knowing whether it manages the dependency too: "woken with it once both are…
kubectl napwake scansays what Napwake does with each dependency it lists, knowing whether it manages the dependency too: "woken with it once both are managed", "not managed, so never paused", "woken and held with it", rather than "connected once Napwake manages it". Before anything is managed, when every dependency would say the same, it's said once above the list, without the columnkubectl napwake scan's replay count, "0 of 13 unmanaged workloads would have slept", says which of the workloads it lists it doesn't count, and why,…
kubectl napwake scan's replay count, "0 of 13 unmanaged workloads would have slept", says which of the workloads it lists it doesn't count, and why, such as "1 that won't pause (PVCs deleted on scale-down)", in the terminal and the report alike, so it adds up to the tablekubectl napwake scanandstatusshow money alike<$1 for an amount above zero that rounds to nothing, and $0 only for none, in the terminal and their web pages. Scan said $0 for a workload that had…<$1for an amount above zero that rounds to nothing, and$0only for none, in the terminal and their web pages. Scan said$0for a workload that had freed three cents, whichstatuscalled<$1. A live workload that has freed nothing yet is$0instatustoo, as inscan, and-is for one only ever in dry-run. Scan'spausing N live workloadscounts only the workloads Napwake pauses, and says on a line of its own how many more are live but protected, so never paused (JSONtotals.liveProtected)The PauseRequest condition says what came of the last pause request and when, in the past tense, rather than Pausing: pausing as requested for days of…
The
PauseRequestcondition says what came of the last pause request and when, in the past tense, rather thanPausing: pausing as requestedfor days of wakes and idle pauses after:Pausedwithpaused as requested at 2026-10-21 03:32 UTC,AlreadyPausedwithalready paused at …, and a refusal's reason withnot paused at 2026-10-21 03:32 UTC:and why. It still answers the last request until the next, sokubectl napwake pauseand scripts can read their answerkubectl napwake forecast NAME shows a workload Napwake manages, rather than refusing it as having "no CPU history to replay" whenever Prometheus had…
kubectl napwake forecast NAMEshows a workload Napwake manages, rather than refusing it as having "no CPU history to replay" whenever Prometheus had under two days of its history, or none: first what Napwake's own forecast of it is doing, from its ManagedWorkload and the state the forecast saves there (whether it's acting and why, its phase, the hours of the day and week it has seen, the hours it has learned from history and live, its confidence, when it's expected to act, and the hours of the next day it expects busy), then the replay of its history when there's enough to replay. A workload that can't be replayed says why and what to do, such as when to run it again, and one that exists but is labellednapwake.dev/ignoresays so. JSON adds each workload'soperatorandnoReplay.forecastwith too little history says how to see a managed workload's own forecastkubectl napwake forecastno longer prints scan's note that "N infrastructure workloads are left out of the totals and next steps", which it has neither of, with or without NAME.scankeeps itkubectl napwakecarries the time zone database, as the operator does, soforecastworks on Windows, which has none of its own, for an operatortimezoneother than UTCkubectl napwake statussayspauses now, orwould pause nowin dry-run, in NEXT for anyIdleworkload, whose pause is due now.After a pause request it showed the clock the request overrode, such as would pause in 40m, and for a workload with no idlePolicy, never pauses: no…After a pause request it showed the clock the request overrode, such as
would pause in 40m, and for a workload with noidlePolicy,never pauses: no idlePolicy. The hour Napwake waits after a GitOps conflict still holds it first
No change in this release matches that.
Versions 0.1.0 (18 March 2026) to 0.1.7 (8 April 2026) were early experiments under the old name. Their notes are in CHANGELOG.md on GitHub.
Releases and dates as published on github.com/napwake/napwake. The entries are from the project’s CHANGELOG.md.