Memory Locking (mlock)

DT-007: Swap Deadlock Prevention

Memory locking is CRITICAL for daemons that serve as swap devices. Without it, your daemon can deadlock under memory pressure.

The Problem

When a daemon serves as a swap device (e.g., trueno-ublk), a deadly cycle can occur:

┌─────────────────────────────────────────────────────────────┐
│                    DEADLOCK SCENARIO                        │
├─────────────────────────────────────────────────────────────┤
│  1. Kernel needs to swap pages OUT to daemon's device       │
│                           ↓                                 │
│  2. Daemon needs memory to process I/O request              │
│                           ↓                                 │
│  3. Kernel tries to swap OUT daemon's pages to free memory  │
│                           ↓                                 │
│  4. Swap request goes to... the same daemon                 │
│                           ↓                                 │
│  5. Daemon waiting for itself → DEADLOCK                    │
└─────────────────────────────────────────────────────────────┘

Real-World Evidence

Kernel log from 2026-01-06 stress test:

INFO: task trueno-ublk:59497 blocked for more than 122 seconds.
task:trueno-ublk state:D (uninterruptible sleep)
__swap_writepage+0x111/0x1a0
swap_writepage+0x5f/0xe0

The Solution

Use mlockall(MCL_CURRENT | MCL_FUTURE) to pin all daemon memory, preventing it from ever being swapped out.

Configuration

Via ResourceConfig

#![allow(unused)]
fn main() {
use duende_core::ResourceConfig;
use duende_platform::apply_memory_config;

let mut config = ResourceConfig::default();
config.lock_memory = true;           // Enable mlock
config.lock_memory_required = true;  // Fail if mlock fails

// Apply during daemon initialization
apply_memory_config(&config)?;
}

Via TOML

[resources]
lock_memory = true
lock_memory_required = true

Direct API

#![allow(unused)]
fn main() {
use duende_platform::{lock_daemon_memory, MlockResult};

match lock_daemon_memory(true) {  // true = required
    Ok(MlockResult::Success) => println!("Memory locked"),
    Ok(MlockResult::Failed(errno)) => println!("Failed: {}", errno),
    Err(e) => panic!("Fatal: {}", e),
}
}

Running the Example

# Basic test
cargo run -p duende-platform --example mlock

# With mlock required (fails without privileges)
cargo run -p duende-platform --example mlock -- --required

# Check current status
cargo run -p duende-platform --example mlock -- --status

Container Configuration

Containers require special configuration for mlock.

Docker

# Minimum required
docker run --cap-add=IPC_LOCK your-image

# Recommended (unlimited memlock)
docker run --cap-add=IPC_LOCK --ulimit memlock=-1:-1 your-image

Docker Compose

version: '3.8'
services:
  trueno-ublk:
    image: your-image
    cap_add:
      - IPC_LOCK
    ulimits:
      memlock:
        soft: -1
        hard: -1

Kubernetes

apiVersion: v1
kind: Pod
spec:
  containers:
  - name: daemon
    securityContext:
      capabilities:
        add:
        - IPC_LOCK

Capability Requirements

mlock requires one of:

Checking Capabilities

# In container
cat /proc/self/status | grep Cap

# Check CAP_IPC_LOCK specifically (bit 14)
# CapEff: 00000000a80465fb  (bit 14 set = has IPC_LOCK)
# CapEff: 00000000a80425fb  (bit 14 not set)

API Reference

lock_daemon_memory(required: bool)

Locks all current and future memory allocations.

Arguments:

• required: If true, returns Err on failure. If false, returns Ok(MlockResult::Failed).

Returns:

• Ok(MlockResult::Success) - Memory locked successfully
• Ok(MlockResult::Failed(errno)) - Failed but continuing (when required=false)
• Ok(MlockResult::Disabled) - Platform doesn't support mlock
• Err(PlatformError) - Failed and required=true

is_memory_locked()

Checks if memory is currently locked by reading /proc/self/status.

#![allow(unused)]
fn main() {
if is_memory_locked() {
    println!("Memory is locked");
}
}

apply_memory_config(config: &ResourceConfig)

Convenience function that checks config.lock_memory and calls lock_daemon_memory if enabled.

#![allow(unused)]
fn main() {
let config = ResourceConfig {
    lock_memory: true,
    lock_memory_required: true,
    ..Default::default()
};

apply_memory_config(&config)?;
}

Testing

Docker Test Suite

cd duende
./docker/test-mlock.sh --build

This runs mlock tests across different privilege configurations:

Unit Tests

cargo test -p duende-platform memory

Troubleshooting

mlock fails with EPERM

Cause: Missing CAP_IPC_LOCK capability.

Solution:

# Docker
docker run --cap-add=IPC_LOCK ...

# Native Linux
sudo setcap cap_ipc_lock+ep ./your-daemon

mlock fails with ENOMEM

Cause: Memlock ulimit exhausted.

Solution:

# Docker
docker run --ulimit memlock=-1:-1 ...

# Native Linux
ulimit -l unlimited

Memory locked but daemon still deadlocks

Possible causes:

1. Container memory limit too restrictive
2. Swap enabled for container
3. cgroup memory controller limiting

Solution:

# Disable swap for container
docker run --memory=2g --memory-swap=2g ...

Security Considerations

• CAP_IPC_LOCK allows locking arbitrary amounts of memory
• This can impact host performance if abused
• In production, set reasonable memlock limits
• Monitor daemon memory usage for leaks