One new field in the KubeOne cluster spec: spec.auditLog turns on the apiserver audit log, in Go and bash (#257). No new command. A cluster spec without the field renders the same manifest as before. Only the file mode of the manifest changes (see Upgrade).

Added

• spec.auditLog (KubeOne driver). The apiserver writes an audit log: one line for each request, with the user, the verb, the object and the time. The log is off by default. Turn it on per cluster:

  # clusters/<domain>/cluster.lok8s.yaml
  spec:
    auditLog:
      policy: audit-policy.yaml   # required: relative to this file
      maxAge: 30                  # optional: days to keep old log files
      maxBackup: 10               # optional: old log files to keep
      maxSize: 100                # optional: MB before the log file rotates

  • lo provision adds KubeOne's features.staticAuditLog to clusters/<domain>/.kubeone/kubeone.yaml: enable: true, the absolute path of the policy file, and only the limits that you set. KubeOne applies its defaults (30, 3 and 100) to the other limits. The other features of the manifest, for example encryptionProviders, stay.
  • KubeOne then copies the policy file to each control-plane host and sets the apiserver audit flags. The log goes to /var/log/kubernetes/audit.log on each control-plane host.
  • lo recover runs the same provision step through the bash tree, so it renders the field too. A project with its own bash tree needs the v0.9.0 tree for that (see Upgrade).
  • The driver checks the field and the policy file before the provider creates or changes a server, and again when it writes the manifest. The error names the value or the file and the fix. The driver refuses:
    • a spec.auditLog that is not a mapping, or a field in it that is not in the list above (for example the typo maxBackups),
    • a policy that is not set or not text,
    • a limit that is not a whole number from 1 to 999999999,
    • a policy file that does not exist, that you cannot read, or that is not valid YAML,
    • a policy whose first YAML document is empty, is not apiVersion: audit.k8s.io/v1 and kind: Policy, or has no rules,
    • a policy file that is not plain YAML: a directive (%YAML 1.2), an anchor, an alias, a merge key (<<), a duplicate key, or a value on a document marker line (--- {…}).
  • Both implementations give the same verdict on 40 shared policy fixtures. The bash tree reads the policy with yq v4 (v4.30 or later). lo recover uses the bash tree, so it needs that yq also with the Go binary.
  • The KubeOne CRD schema (operator/crds/kubeone.yaml) lists the new field.

  The security guide has a new section, "Apiserver audit log", with an example policy and a way to verify the log. The spec reference lists the fields.

Changed

• The KubeOne manifest is written in one step. The driver renders kubeone.yaml into a temp file beside it (mode 0600) and renames the temp file only when every merge succeeded. A refused render keeps the last good manifest. This holds for every KubeOne cluster, with or without spec.auditLog, in both implementations.

Upgrade

• spec.auditLog is opt-in. Without it, the manifest content does not change. Apart from the write in one step (see Changed), lo provision and lo recover do what they did in v0.8.0.
• .kubeone/kubeone.yaml now has mode 0600 after every render. Before, both implementations created it with mode 0644 (with the usual umask). The manifest can hold registry credentials.
• Releases before v0.9.0 do not read spec.auditLog. They render no audit log and do not warn. Pin v0.9.0 before you add the field.
• A project with its own bash tree in .lok8s/ (lo assets eject bash) runs that tree for lo recover, and for a command that lok8s.yaml routes to bash. A tree from before v0.9.0 ignores spec.auditLog and does not warn. Run lo assets update bash before you add the field.
• To turn the log on for a running cluster, or to change its policy file, run lo provision to render the manifest. Then run kubeone apply --manifest kubeone.yaml --force-upgrade in clusters/<domain>/.kubeone/, with the credentials that lo provision uses. KubeOne changes a feature on running control planes only with --force-upgrade, and lo provision runs a plain kubeone apply.

Known limits

• lo has no flag for the forced apply yet. Run it by hand, as above.
• The driver does not check each rule of the policy. The apiserver reads the rules when it starts.
• A cluster with spec.kubehz.hosting: hosted has no KubeOne manifest, so the driver ignores spec.auditLog there.
• A temp file .kubeone/.kubeone.yaml.* can stay when lo ends before it removes the file. In the Go binary, a second INT or TERM signal or a HUP signal ends lo at once. In both implementations, a KILL signal or a crash does the same. Delete such a file, because it can hold registry credentials.

For contributors

• Every bats test of the unit and operator suites now gets its own XDG_STATE_HOME, LO_REGISTRY_STATE_DIR and XDG_CACHE_HOME under BATS_TEST_TMPDIR (#256). Before, a local run of tests/unit/kind_contract_test.bats replaced the registry configs in ~/.local/state/lok8s/registries/ with a stub. A shared registry container that mounts such a file then failed at its next restart. The lo binary was not affected.

Commits

• ec25221 feat(kubeone): spec.auditLog turns on the apiserver audit log (#257)
• 94ba9d5 fix(tests): isolate the registry state dir in every bats test (#256)