Skip to content

Running in production · Argo & Kargo

One line changed.
Twenty-eight manifests moved.
A 27B model drafted one.

Kargo opens more pull requests than anyone can read, and merges a good share of them on a version-shaped policy that cannot see what is in the diff. Bosun renders what the change deploys and blocks what breaks. It repairs the mechanical part without a model, and when only a model can draft the fix, that draft has to clear the target schema first. The rest it hands to a human, file and key named.

Classification
10/10
Full pass
10/10
Unsafe actions
0

Ten recorded incidents, scored against a 27B model on one workstation. Every case came off a live pull request.

pull request #279 · addons-gate

- defaultVersion: "0.10.3" + defaultVersion: 2.10.0

addons/environments/production/addons/addons.yaml

blocking Four CustomResourceDefinitions stopped serving a version

  • external-secrets.io drops v1alpha1 and v1beta1; only v1 survives
  • 28 manifests in this repository still declare a dropped version, and 60 ExternalSecrets are running on one
  • 11 CRDs added · 21 resources changed

repaired Twenty-seven needed only an apiVersion swap. Bosun made them with no model involved, comments and quoting intact.

reshaped The twenty-eighth needed more. dataFrom changed shape, so a swap alone would have applied cleanly and dropped three values.

- dataFrom: - - key: demo/legacy + dataFrom: + - extract: + key: demo/legacy

A 27B model drafted that document. Bosun checked it against the new schema before writing a line: same object, and no value in it that the original or the schema did not supply.

The text diff was one line. Blocked 03:04, repaired 03:05, green on the re-run.

The gap

A version bump can stop serving an API

A one-line bump renders perfectly and stops serving an API every manifest in your repository still declares. A values-layer edit adds a whole cluster to an addon’s scope without the selector changing, because the labels it matches did. Nothing in the text diff shows either. A merge policy that reads version numbers cannot see them, and a human reading their fortieth bump of the week will not either.

What a version-shaped policy sees

  • A version number moving in a values file.
  • One line changed: auto-merge it, or hand a human the same one line.

What the render shows

  • Eleven CRDs added, twenty-one resources changed.
  • Four CRDs stop serving v1alpha1 and v1beta1.
  • Twenty-eight manifests in the repository still declare them, and sixty objects are running on one.

The shape

The gate that demands a repair is the one that verifies it

A boatswain’s job is inspection and repair: daily rounds of the hull and rigging, fixing what they find on their own authority and reporting to the captain what they cannot. The inspection is the larger half, and it is what makes the repair authority safe to grant.

The bosun loop Kargo opens a pull request. The gate renders it at base and head and publishes a verdict. Bosun reads that verdict and does one of four things: repairs it deterministically, by swapping apiVersion values the gate's own report names, with no model involved; repairs it from a draft, where a model writes a whole migrated document or values file and the harness refuses it unless it passes every check; explains a green gate; or escalates it to a human. Either kind of repair goes back through the same gate, which re-runs and re-counts. Only a green gate reaches the merge, ArgoCD and the verification. either repair goes back through the same gate Kargo The gate Bosun Repair, computed Repair, drafted Explain Escalate Merge opens the pull request renders base and head reads the verdict apiVersion swaps, no model the model writes, code refuses what the bump changed needs-human, and it stops ArgoCD, then verification

What it does

Four jobs, with code drawing the line between them

Renders the truth, twice

Every bootstrap ApplicationSet expanded for every cluster, at the base revision and at the head. The diff of those two renders is what the pull request does. Charts whose version moved are pulled at both versions and diffed down to the field.

Blocks the four that break things

A cluster-targeting change, a moved source, project or namespace, an object’s apiVersion moving, and a CRD dropping a served version your manifests still declare. Everything else is reported rather than blocked, because that is what a version bump legitimately does.

Repairs, two ways

When the only blocking finding is dropped served versions, no model is called. The report names the kind, the dropped versions and the survivor; every declaring manifest is migrated and the gate re-counts them.

Where judgement is needed the model drafts and code refuses: a reshaped manifest when a swap alone would lose a field, a migrated values file when a chart version stops accepting settings you still make. Neither lands until it passes.

Escalates as a handoff

The comment names which file and key to open, what the choice is, and the one fact that stopped a mechanical fix. Then the needs-human label goes on and it stops. It never closes a pull request and its own status is never a failure.

Not only bumps

One entry added. Four Applications, on four clusters.

A pull request that adds an addon has nothing to diff against and nothing to block. The expansion is still the thing worth reading, so the report prints it: a New addons table, one row for every Application the change generates, listed for review.

What the pull request tells you

  • One entry added to a values file.
  • A description reading “add kargo-observability”.

What the table tells you

  • Every Application that entry generates, by name.
  • The cluster each one lands on.
  • The chart and pinned version, or the path it renders from.

The safety model

The model has no file-edit tool

It writes plenty. A scalar value, a whole reshaped manifest, a whole migrated values document: all three are content the model authored, and the first of them lands on the line exactly as it spelled it. What it cannot do is apply any of it, or decide which files are writable. It returns a structured proposal and the service does the writing, behind a path allowlist and a deny-list its own configuration cannot remove from.

Each path is checked by something the model does not control: a from value that must already be in the file, a target schema, and for values, helm itself, asked to render the chart with the proposal before a line is written. So “never edit the gate, never weaken a policy to go green” is an invariant the service enforces, not an instruction the model is asked to respect. A model that ignores the prompt entirely still cannot touch CI configuration.

A model holding file-edit tools can make a red gate green by deleting the check, and from outside that looks the same as a repair. Handing it a document to write is a much smaller grant, and the difference is that a document is judged against a schema it did not choose.

Read what is enforced, and where

Install

One chart, two Secrets, one config file

The chart consumes existing Secrets by name and creates none, so bring your own secret manager. Nothing here hardcodes a cluster, domain, namespace, CNI, git host or model provider.

Terminal window
helm install bosun oci://ghcr.io/integratnio/charts/bosun \
--namespace bosun --create-namespace \
-f my-values.yaml

Quickstart: 15 minutes   The full onboarding path

Where to go next

Where to read next

Licence

PolyForm Internal Use 1.0.0. Run it for your own business, commercially, in production, without asking anyone. You may not distribute it: not sold, not bundled, not offered as a hosted service. Installing the chart and image from the registry is use, not distribution. What that means in practice →