Harthelm Docs

Harthelm 1.4.0

What changed in Harthelm 1.4.0: browser hosts for RabbitMQ and S3 with single sign-on to RabbitMQ's management UI and public S3 stores, in-cluster TLS for S3, safer S3 backups, the restore runbooks as a release asset, three new doctor checks.

Last updated 2026-10-19

Released 2026-10-15. API version 0.12.0. A minor release with two database migrations, which only add. Nothing needs to be done before the upgrade, but read Before you upgrade: every RabbitMQ broker and every S3 store restarts once.

Terminal
helm upgrade harthelm oci://<your registry>/charts/harthelm --version 1.4.0 -n harthelm-system -f my-values.yaml

Before you upgrade

  • Every RabbitMQ broker restarts once after the upgrade, for about 10 to 30 seconds without AMQP: its configuration gains the single sign-on for the management UI. Clients reconnect; no service that uses the broker restarts. Until a broker has restarted, Open management UI ends at RabbitMQ's login form with "Not_Authorized"; the instance's user and password work there.
  • Every S3 store restarts once, for about 20 to 35 seconds without S3: it now reports ready only once reads and writes work (Changed). With in-cluster TLS on, a store also gets its certificate and serves HTTPS on 8443; its arrival can restart the store a second time.
  • Nothing else restarts. Workspaces, services, PostgreSQL, MariaDB and Valkey keep running.
  • hart 1.3 warns once a day that it speaks API 0.11. Update it from the release's archives (Install hart).
  • New values. The 1.3 chart's schema rejects marketplace.browserHosts and marketplace.storageWriteProbe. If you keep your values in Git for Argo CD, move the pin to the 1.4 chart before or together with a change of these values.
  • S3 bucket names that the provider's schema let through but S3 does not accept (64 characters, two dots in a row, an IP address, a reserved name) are refused from now on, with a 422 on the next create or update of such a store.

New

Browser hosts for managed services. Every RabbitMQ broker and every S3 store gets its own host, ms-<n>.<workspace domain>, where <n> is the instance's number, which is never reused. The workspace domain's wildcard DNS name and certificate cover it: nothing new is needed in DNS or at the edge. PostgreSQL, MariaDB and Valkey get no host. The details drawer's Overview shows the host under Browser host. See Managed services.

  • RabbitMQ's management UI with single sign-on. Open management UI opens the UI in a new tab, signed in as you for one hour, with the tags management and policymaker on the broker's virtual host. The host is always behind your team's sign-in, and it is never public. For a broker that belongs to a landscape, the single sign-on is for the editors and admins of its workspace; other team members reach the login form and sign in with the broker's user and password, which keep working for everyone.
  • S3 for browsers. Presign URLs with the store's own host, ${{ services.<name>.details.browserEndpoint }}, path-style. The router passes path and query unchanged, so every key works, also keys with spaces, +, % or non-ASCII characters. By default the host is behind your team's sign-in: Open the store's host signs your browser in for 12 hours, after which presigned URLs, images, video and downloads on your team's dev hosts load. Scripts send an API key in Harthelm-Authorization, added after the request is signed.
  • Public S3 stores. Team members can switch a store's host public (for a store that belongs to a landscape, the editors and admins of its workspace): Public host in the drawer, "public": true in PATCH /managed-services/{ms}, or hart ms create … --public. A public host has no sign-in: anonymous visitors read the objects of the store's publicBuckets, and every other request needs a signature with the store's key, so presigned URLs work for anyone who holds one until they expire. The switch takes effect within a second and restarts nothing. It is not part of harthelm.yml, and a Sync never changes it. An administrator can forbid public hosts for the whole installation.
harthelm.yml
  web:
    env:
      S3_ENDPOINT: ${{ services.files.dsn }}                                # in the cluster
      S3_PUBLIC_ENDPOINT: ${{ services.files.details.browserEndpoint }}     # for browsers: https://ms-42.<workspace domain>

In-cluster TLS for S3. Where in-cluster TLS is on and cert-manager is installed, S3 stores get a certificate of the internal CA, as Valkey, RabbitMQ and MariaDB did in 1.3, and serve HTTPS on 8443 next to HTTP on 8333. The connection URL stays http://<host>:8333; an app that wants HTTPS uses https://<host>:8443 and trusts $HART_INTERNAL_CA_FILE, for example with AWS_CA_BUNDLE. The detail tls says whether the store serves 8443. The nightly backups and the restore then use HTTPS.

The restore runbooks are a release asset. Each release attaches harthelm-restore-<version>.tar.gz with its checksum file harthelm-restore-<version>.tar.gz.sha256: the scripts that put a Valkey, MariaDB, RabbitMQ or S3 backup back into an instance, and a README.md. Take the archive of the release your installation runs. See Backups and restore.

Three new doctor checks. managed-storage-write shows whether a pod with the managed services' security context can write a fresh volume of each storage class they use; core probes each class with a short Job when it becomes the leader and then hourly. managed-s3-backup-space compares the backup volume's free space with what the S3 snapshots hold, and managed-s3-blob-gc reports whether the cleanup of unreferenced S3 backup data went through within 48 hours. See Troubleshooting.

hart --contexts. The global flag --config, the contexts file, has a second name, --contexts. Use it with hart ms create, whose own --config KEY=VALUE sets a setting of the new service. A --config there without = says so. See hart › Global flags.

Changed

  • An S3 store is ready only once reads and writes work. Its startup probe waits until the store's volume server has registered with the master and the master has finished its warm-up. A store that restarts is about 20 to 35 seconds without S3, and no longer reports ready while writes still fail.
  • S3 bucket names are checked by the API. A name of 3 to 63 characters, no two dots in a row, no IP address, no reserved name, none twice, every public bucket one of the buckets: a create or update that breaks one of these answers 422 with the field, and the console's form checks the same.
  • The S3 restore holds its snapshot. While the restore Job reads a snapshot, the nightly retention keeps it and its data. You no longer suspend the backup CronJob for a restore.
  • The Valkey restore goes over TLS to the target's port 6380 when the target has in-cluster TLS.
  • The RabbitMQ restore posts the topology first and the bindings after it, and posts queues the broker's import left out again, up to five times. A queue that is still missing ends the restore with an error that names it, before any binding is posted.
  • A RabbitMQ series move after a pause. A paused broker whose newest dump reported a warning cannot move to the next series until it has run again on its own series; the change is refused with update-constraint and says how to resume it first. A paused broker on a series that a later release drops starts on its old image once more when it resumes, so the dump before its move works.
  • Expected waits are logged at INFO. A background job that waits out an expected state (a certificate on its way, a stuck pod being replaced, a server that starts for the dump before a version move) logs job waits at INFO instead of a warning. The log lines of a written dump carry tls and port.
  • Release candidates carry no source bundle. Only a release has one; a candidate checks that every component has its source instead.

Fixed

  • S3 right after a restart. A store that had run a while could report ready while its writes still waited for up to 15 seconds, and any store could report ready a few seconds before it served objects. The new startup probe covers both.
  • A RabbitMQ restore could miss a queue. The broker's import drops a quorum queue that does not start within five seconds on a slow disk; the restore then reported success without it, or failed on the bindings to it.

For operators

Browser hosts. marketplace.browserHosts.enabled (on by default) gives RabbitMQ and S3 their hosts; false keeps 1.3's behaviour. Turning it on or off restarts every RabbitMQ broker once. marketplace.browserHosts.allowPublic (on by default) lets teams switch an S3 store's host public; with false the API answers 409 public-hosts-disabled, and stores switched public before are served behind the sign-in until it is allowed again. Every opening of a host through the console is an audit row managed_service.open. The router reports harthelm_router_managed_hosts and counts these requests with the kind managed. See Managed services.

The write probe. With marketplace.storageWriteProbe.enabled (on by default) core runs a short Job in the release namespace for each storage class that Valkey, S3, MariaDB and RabbitMQ put volumes on: a pod with their security context writes and reads a fresh 64 MiB volume, then the Job is deleted with it. Core uses its own image and its existing rights. A class that fails shows in managed-storage-write with the step that failed and the volume root's owner.

S3 backups. The backup role records the backup volume's size and free space at the end of every managed run. The two S3 checks of the doctor say skip until the first managed backup run after the upgrade (the nightly one, or a dump on demand) has recorded it, and warn if none did within 48 hours. Size the backup volume for about the S3 data times one plus five weeks of changes, plus 1.2 times the data free. See Backups and restore.

API 0.12.0. ManagedService.public (the switch) and ManagedService.browser (the host's URL, its kind, whether it is public and may be switched public, and whether you get the single sign-on); public on create and update, also for a service that belongs to a landscape; the 409 public-hosts-disabled; the three checks in GET /admin/doctor; the 422 for S3 bucket names. The migrations add the public flag, the S3 restore holds and the backup volume's record.

Changed values

ValueChange
marketplace.browserHosts.enabledNew, default true: RabbitMQ and S3 instances get their own host. false is 1.3's behaviour.
marketplace.browserHosts.allowPublicNew, default true: teams may switch an S3 store's host public.
marketplace.storageWriteProbe.enabledNew, default true: the doctor's write probe of the managed services' storage classes.

Known limits

  • A gated S3 host serves navigations and embeds, not scripts from other sites. fetch and XHR from a dev host carry no cookie and get 401; images, video and downloads in the console's Preview, on a custom domain, or under a workspace domain on the Public Suffix List do not load either. Switch the store public, or keep the forward route of 1.3 on your dev domain for script uploads during development.
  • CORS allows the origins https://*.<workspace domain> only: pages on a custom domain cannot use presigned requests against a store from the browser.
  • Path-style only. Bucket names as host names (<bucket>.ms-42.…) are not served.
  • The management UI's sign-in lasts one hour and is not renewed; the UI's Click here to log in leads back through the console.
  • Restores are runbooks. A button that restores a backup into a new instance is not in this release.

On this page