> Markdown version of https://tuna.am/en/blog/debian-12-kernel-freeze/. Documentation index: [llms.txt](https://tuna.am/en/llms.txt), full text in one file: [llms-full.txt](https://tuna.am/en/llms-full.txt). Any page is available as Markdown: append `.md` to its URL without the trailing slash.

[Blog](https://tuna.am/en/blog.md)|[Under the hood](https://tuna.am/en/blog/tags/engineering)

# Kernel 6.1.0-55 freezes Debian 12: how to check your servers and update

November 18 · 7 min

After the recent wave of leaks and breaches, we install all updates on our infrastructure servers and staging automatically and reboot them automatically, too. On November 16–17, that's how Debian 12 kernel `6.1.0-55` arrived, and all ten servers that booted into it froze: some after a couple of hours, others after 30. It didn't reach production: there, instead of an automatic reboot, we have an alert that a server needs one.

The cause is an ext4 bug, and a fix is already out. We're sharing what we ran into so you can be more careful. If you run Debian 12, check your kernel now.

> **In brief**
>
> * Kernel `6.1.0-55` (`linux 6.1.192-1`, released on November 15) has an ext4 bug. A few hours after boot, disk writes stall while the network keeps working.
> * The fix is kernel `6.1.0-56` (`linux 6.1.192-2`) in `bookworm-security`. The `linux-image-amd64` metapackage already installs it.
> * If a server is running `-55` right now, don't run `apt` on it: installing packages triggers the freeze by itself. Boot into `-54` first.
> * Roll out updates, kernels especially, in waves: staging and a couple of servers first, the rest a day later.

## What broke

On November 15, Debian LTS published [DLA-4906-1](https://lists.debian.org/debian-lts-announce/2026/11/msg00032.html), a `linux 6.1.192-1` kernel update for Debian 12 with security fixes. On the system, this is the `linux-image-6.1.0-55-amd64` package.

In this kernel, ext4 dereferences a null pointer in `ext4_mb_load_buddy_gfp` ([#1154101](https://bugs.debian.org/1154101), severity critical; a similar report is [#1154153](https://bugs.debian.org/1154153)). The crashing task holds a jbd2 journal handle. The journal commit waits for it forever, and every process that writes to the root filesystem hangs in the D state.

Meanwhile, the server looks alive. The network works, ping replies, and services that are already running keep serving requests as long as they don't need to write to disk. Everything that writes stalls: logs, databases, package installation.

The fix is `linux 6.1.192-2`, the `linux-image-6.1.0-56-amd64` package in `bookworm-security`. The bug was closed in the Debian tracker on November 16, and the `linux-image-amd64` metapackage already points to `-56`. The `-55` package hasn't been removed from the repository: if you install it explicitly by name, it will install.

## What happened on our servers

On our infrastructure servers and staging, unattended-upgrades installs updates with automatic reboots (`Automatic-Reboot`): on some, right after the update; on others, at night on a schedule. On November 16–17, it installed `6.1.0-55` and rebooted the servers.

All ten servers that booted into `-55` froze: infrastructure machines, part of staging, and CI runners. They included both virtual machines and bare metal, so both are affected. One runner froze three times on November 16, the rest 2–30 hours after boot. Servers that got the update later received `-56` right away and weren't affected.

## Check your servers

```shell
uname -r                                     # which kernel is running
dpkg -l 'linux-image-6.1.0-*' | grep '^ii'   # which kernels are installed

```

* **`6.1.0-55-amd64` is running.** The server can freeze at any moment. Don't install packages on it; go straight to [If the server is already on -55](#running-55).

* **`-55` is installed, but a different kernel is running.** `-55` will boot on the next reboot. Install the fix and remove `-55` before rebooting:

  ```shell
  apt update
  apt install linux-image-amd64
  apt purge linux-image-6.1.0-55-amd64

  ```

* **`-55` isn't installed.** A regular update will install `-56` directly.

## If the server is already on -55

Don't install packages on `-55`: `apt` rebuilds the initramfs, writes to disk, and triggers the freeze. That's how two of our servers froze a second time. This order worked for us:

1. If the server is frozen, hard-reset it through your provider's panel or API. A regular `reboot` can hang too.

2. Right after boot, while the server still responds, select the old kernel for one boot and reboot. Don't write anything else to disk:

   ```shell
   grub-reboot 'Advanced options for Debian GNU/Linux>Debian GNU/Linux, with Linux 6.1.0-54-amd64'
   grub-editenv list    # should show next_entry=Advanced options…
   systemctl reboot

   ```

   Menu entry names depend on `GRUB_DISTRIBUTOR`; you can list yours with `grep -E "menuentry |submenu " /boot/grub/grub.cfg`. You don't need to change `GRUB_DEFAULT`: booting into `-54` is a one-time thing.

3. On `-54`, install the fixed kernel, remove `-55`, and reboot. GRUB will boot the new kernel by default:

   ```shell
   apt update
   apt install linux-image-amd64
   apt purge linux-image-6.1.0-55-amd64
   systemctl reboot

   ```

4. Check `uname -r`: it should be `6.1.0-56-amd64`.

If the server freezes before you can log in over SSH, your provider's console (VNC) is the only option: in the GRUB menu, select "Advanced options for Debian GNU/Linux" and kernel `6.1.0-54`. We didn't try to save two such servers and reinstalled them with Debian 13.

## Kernel 6.12 or Debian 13

`bookworm-security` also has a kernel from the 6.12 branch, the same one as in Debian 13: `linux-image-6.12-amd64` (`6.12.117-1~deb12u1`). Follow the same steps, just with `linux-image-6.12-amd64` instead of `linux-image-amd64`:

```shell
apt install linux-image-6.12-amd64

```

We installed 6.12 on our Debian 12 servers. The ones we repaired have already booted into it, and the rest will switch at their next planned reboot. We haven't seen any problems so far, but our servers have been on 6.12 for less than two days, so it's too early to talk about stability.

Debian 12 is already in LTS. If your servers are due to move to Debian 13 anyway, this is a reason not to put it off.

## Roll out updates in waves

The kernel bug is only half the problem. The worse part is that the buggy kernel reached a dozen servers within a day and rebooted them by itself. `Automatic-Reboot` is convenient as long as updates are safe. When they aren't, a single bug becomes an outage of the whole fleet.

* **Staging and a couple of servers first, then the rest.** Wait at least a day between stages: the `-55` bug showed up 2–30 hours after boot, so a couple of hours wouldn't have caught it.
* **Don't reboot the whole fleet at once.** Use a different `Automatic-Reboot-Time` for different groups of servers, or reboot manually after checking on staging.
* **Track which kernel is running.** node_exporter exports the `node_uname_info` metric with a `release` label: it shows where a new kernel has booted, and you can alert on a specific version.

We weren't the only ones with update-related outages this week. On November 17, some servers at Timeweb Cloud, a Russian cloud provider, lost network connectivity for almost two hours (every location except Novosibirsk was affected): during an update of the hypervisors' network stack, the dynamic routing service [didn't fully restart](https://t.me/timewebcloud_alerts/423). The same day, iCloud, the App Store, and Apple Pay were down, and Apple [named an internal update](https://9to5mac.com/2026/11/17/icloud-app-store-and-other-apple-services-hit-by-ongoing-outage/) as the cause. The causes are different and unrelated to the Debian kernel, but all three outages started with an update. After its review, Timeweb promised to increase the intervals between update stages.

## Leave feedback

If you enjoy using Tuna, or on the contrary you are not happy with something, please [leave feedback](https://my.tuna.am/?show_feedback=1&lang=en).

![](/blog/img/2024-08-22/feedback.gif)

## News

We announce new features and posts on our [Telegram channel](https://t.me/tuna_am). Subscribe so you don't miss anything.

## Help

We value our users and carefully review every request. If you have any problems with tuna, please contact us in one of the following ways:

* [Email](mailto:info@tuna.am)
* [Telegram community](https://telegram.me/+Kgbtt0_gJwI5ZmEy)

Categories: [Under the hood](https://tuna.am/en/blog/tags/engineering)

[older →🧭 Сайт за туннелем не открывается? Проверьте DNS-резолвер](https://tuna.am/en/blog/dns-resolver-geodns.md)
