Kernel Detection Engineering: From Syscalls to Defensible Evidence

Do not conflate filter and tracer: seccomp classic BPF filters syscall number, architecture and raw arguments before the handler; it does not resolve a pointed-to pathname. eBPF tracing or LSM instrumentation asks a different question and cannot by itself prove that a policy denied an operation. Kernel seccomp semantics.

01Speak precisely about the boundary

A syscall alert without a subject, an object and a result is a lead—not a finding. A web worker opening a socket, a debugger reading memory and an implant doing the same can produce superficially similar events. Detection engineering reconstructs the operation, establishes whether it belongs to the workload, and records the limits of the evidence.

This companion to K8SEC’s kernel-boundary guide moves from hardening to investigation. All traces in the interactive exercise are synthetic. The page does not collect host data, execute malware or claim that a production detector has been validated.

Three paths, three meanings: /sys/devices/… describes kernel objects and attributes; /proc/sys/kernel/… exposes kernel sysctl settings; a function call path describes execution. A path string alone does not capture a mount namespace, underlying object or permission decision. Sources: Linux syscall reference, sysfs documentation, LSM architecture (conceptual background; that page warns that its API descriptions are outdated).

Architecture references: entry and exit handling, memory-management concepts. Conventional container assumptions do not automatically apply to VM-backed runtimes.

Infer the behavior from an evidence chain. Never infer malware merely from the name of an API.

02Choose sensors by the question they can answer

The following flow is conceptual, not a mandatory ordering for every subsystem. Several policy checks, asynchronous operations and object lifetimes may intervene.

1. User intent → ABI

   Wrapper, syscall number, arguments and calling task.

2. Kernel operation → policy

   Resolved object, relevant LSM checks and subsystem execution.

3. Result → evidence

   Return status, subsequent effect and collector health.

Lifecycle semantics: open/openat reference. The error values shown here are wrapper-level conventions; a kernel event may encode errno differently.

Operation references: VFS object and operation model, read results and page cache.

eBPF is an execution mechanism for programs attached to supported hooks. The attachment type and program logic determine whether it observes, transforms or participates in enforcement. A BPF LSM policy must respect its hook’s semantics and preserve an existing denial where the contract requires it. A tracing probe should not be advertised as an authorization control. For example, an inode-permission hook addresses filesystem permission checks; an execution-related hook examines a different stage. Do not use either as a substitute for a successful operation event. Read the BPF LSM contract, event tracing interfaces and kprobe limitations.

Visibility is a measurable dependency

Capture sensor version, kernel release/build, architecture, loaded attachments, enabled events, namespace visibility, clock source, sampling configuration and loss counters. A healthy agent process does not prove that its required hooks are active. Establish canary operations and check that their expected events arrive.

Account for asynchronous I/O and kernel-worker attribution when testing your sensor. A syscall-only model is not automatically a complete model of file or network effects. Conversely, do not claim that io_uring is universally invisible: coverage depends on the operation, kernel and collector. User-space work may also occur without a new syscall. State the coverage you tested instead of guessing.

03Read /proc/sys/kernel as a security control surface

The kernel sysctl reference describes controls, not a universal hardening profile. Verify availability and vendor behavior on the actual kernel. These concise semantics are the starting point:

• unprivileged_bpf_disabled: 0 allows unprivileged BPF calls; 2 disables them reversibly; 1 cannot be cleared in the running kernel.
• modules_disabled=1 prevents module loading and unloading; kexec_load_disabled=1 disables further kexec image loading. Both are one-way for the running kernel.
• kptr_restrict constrains pointer disclosure through the documented formatting interfaces; dmesg_restrict limits kernel-log access. Neither is a universal information-leak fix.
• core_pattern configures core handling; an initial pipe character selects a userspace handler.
• io_uring_disabled restricts new ring creation; existing rings remain usable. Mode 1 permits the documented privileged/group exceptions; mode 2 blocks creation for everyone.
• bpf_stats_enabled enables runtime statistics with overhead; it is not an incident recorder.

For perf restrictions, use the separate perf security model; capabilities and distribution-specific settings matter. For process inspection, Yama’s ptrace_scope distinguishes traditional checks, relationship-restricted attachment, admin-only attachment and a no-attach mode. This is additional policy, not a replacement for all process-access checks.

Turn configuration into an investigation contract

K8SEC’s proposed analytic is not “value differs from my laptop.” Maintain a versioned node baseline with approved exceptions. Join a change observation to the responsible process identity, authenticated operator or configuration-management job, expected value transition and maintenance window. A current value proves state at collection time; it does not identify who changed it. A failed write proves neither a changed value nor a later successful attempt.

Prioritize unexplained policy weakening or changes to execution-sensitive handlers. Review legitimate crash collectors, monitoring agents and driver maintenance before alerting. Do not paste a batch of irreversible settings into a worker. Establish required drivers and recovery procedures, test on a disposable node, then deploy through the supported node-management mechanism.

for key in kernel.unprivileged_bpf_disabled kernel.modules_disabled \
  kernel.kexec_load_disabled kernel.kptr_restrict kernel.dmesg_restrict \
  kernel.perf_event_paranoid kernel.yama.ptrace_scope kernel.io_uring_disabled
do
  if value=$(sysctl -n "$key" 2>/dev/null); then
    printf '%s=%s\n' "$key" "$value"
  else
    printf '%s=UNKNOWN (unavailable or not readable)\n' "$key"
  fi
done

This inventory makes no changes and does not establish collector coverage. A container’s view is not necessarily the host’s view. Record where it ran; keep inaccessible values UNKNOWN rather than treating them as disabled.

04Translate malware objectives into observable behavior

Reverse engineering starts with a hypothesis about a capability. Detection starts with what your telemetry can establish. An import, string or disassembled branch may reveal a possible operation without proving that execution reached it. Dynamic execution covers only the inputs, environment and branches exercised.

memfd_create supplies anonymous file-backed storage, not a malware verdict. execve replaces the current program image; a successful exec does not return to the old program. Model successful execution using the collector’s completion/event semantics, not a fabricated universal “exec returned zero” record. process_vm_readv returns transferred bytes and can complete partially; ptrace has multiple operations whose meanings differ. Sources: memfd_create, execve, process_vm_readv, ptrace.

Attacker lens, defender discipline

An attacker seeks a useful effect: inspect another identity, reach a protected object or alter a policy. Your rule should describe that effect and its prerequisites, not a particular utility name. Start with the shortest defensible claim. “An unexpected process read bytes from another process” is stronger evidence than “malware injection detected” when all you have is a read result.

Do not automatically attach a technique, campaign or exploit chain to a syscall. Record the mapping rationale and leave unsupported claims unmapped. A successful host operation also does not establish a container escape: reconstruct the expected boundary, actual namespace/mount exposure and effective authority first.

05From reverse engineering to a detection specification

1. Establish the specimen and scope. Record hashes, acquisition provenance, allowed environment and analyst ownership. Real malware belongs in a dedicated isolated analysis environment, not a production cluster or this website.
2. Extract capability hypotheses. Describe relevant data flow, call sites and prerequisites. Mark static-only observations separately from exercised behavior. Identify inputs and branches that remain untested.
3. Design controlled observations. Use disposable workloads, synthetic files and memory, bounded network simulation and independent outcome checks. Include an authorized positive baseline and a deliberately denied operation.
4. Normalize semantics. Specify sensor-specific success/failure conventions, partial results, task identity and object resolution. Preserve raw event references; avoid flattening everything into “syscall happened.”
5. Write the behavior contract. Declare the join keys, time window, required events, trust source for approval and missing-data behavior. Declare what the rule does not detect.
6. Challenge the rule. Replay benign lookalikes, missing events, duplicates, reordered timestamps, reused PIDs and mismatched objects. Have another analyst reject any conclusion the evidence cannot support.
7. Ship with observability. Version the rule, fixtures, sensor matrix and reviewer sign-off. Measure coverage and triage cost before enabling response automation.

A harmless trace to learn the recording format

In an authorized disposable Linux environment with strace, trace a fixed, benign shell command. It creates no payload and does not change kernel policy. The output file is created in your current directory. Loader and shell behavior will vary by system; that variation is part of the lesson.

strace -f -ttt -yy -e trace=execve,openat,read,write,close \
  -o benign-trace.log -- /bin/sh -c 'printf "K8SEC benign fixture\n"'

Match a few entries to returns, resolve the visible file descriptors and explain the loader’s activity before writing a rule. -f follows child processes; timestamps and descriptor decoding help reconstruction, but do not magically provide immutable object identity. Tracing perturbs execution and can expose argument or buffer contents. See the strace manual. This article’s validation checks shell syntax; it does not execute this capture.

06Interactive evidence graph: one API, four investigations

Select a path and follow the directional relationships. Observed means supplied in this synthetic fixture, not measured on a real host. Inferred marks the analyst interpretation. Disproven applies only to the stated claim for this call. Approval is independent fixture context, not something a syscall authenticates.

{
  "id": "unexpected",
  "label": "Unexpected memory read",
  "authorization": "unexpected",
  "coverage": "complete",
  "events": [
    {
      "phase": "enter",
      "at": 100,
      "call": "read-01",
      "subject": "nodeA/bootA/pid410/start90",
      "target": "nodeA/bootA/pid620/start70",
      "operation": "process_vm_readv"
    },
    {
      "phase": "exit",
      "at": 120,
      "call": "read-01",
      "subject": "nodeA/bootA/pid410/start90",
      "target": "nodeA/bootA/pid620/start70",
      "operation": "process_vm_readv",
      "result": 64
    }
  ]
}

{
  "id": "approved",
  "label": "Approved diagnostic",
  "authorization": "approved",
  "coverage": "complete",
  "events": [
    {
      "phase": "enter",
      "at": 100,
      "call": "read-02",
      "subject": "nodeA/bootA/pid411/start91",
      "target": "nodeA/bootA/pid620/start70",
      "operation": "process_vm_readv"
    },
    {
      "phase": "exit",
      "at": 120,
      "call": "read-02",
      "subject": "nodeA/bootA/pid411/start91",
      "target": "nodeA/bootA/pid620/start70",
      "operation": "process_vm_readv",
      "result": 64
    }
  ]
}

{
  "id": "denied",
  "label": "Denied attempt",
  "authorization": "unexpected",
  "coverage": "complete",
  "events": [
    {
      "phase": "enter",
      "at": 100,
      "call": "read-03",
      "subject": "nodeA/bootA/pid412/start92",
      "target": "nodeA/bootA/pid620/start70",
      "operation": "process_vm_readv"
    },
    {
      "phase": "exit",
      "at": 120,
      "call": "read-03",
      "subject": "nodeA/bootA/pid412/start92",
      "target": "nodeA/bootA/pid620/start70",
      "operation": "process_vm_readv",
      "result": -1,
      "error": "EPERM"
    }
  ]
}

{
  "id": "incomplete",
  "label": "Missing return event",
  "authorization": "unknown",
  "coverage": "loss",
  "events": [
    {
      "phase": "enter",
      "at": 100,
      "call": "read-04",
      "subject": "nodeA/bootA/pid413/start93",
      "target": "nodeA/bootA/pid620/start70",
      "operation": "process_vm_readv"
    }
  ]
}

The graph deliberately keeps the same API across scenarios. Changing the outcome, approval or telemetry completeness changes the conclusion. No scenario yields a “malware confirmed” result. A denial does not prove which security control caused it, and absence of a return does not prove that nothing happened.

07A reproducible, deliberately narrow correlation rule

The downloadable model evaluates exactly one normalized process_vm_readv entry/exit pair. It requires matching call, subject and target identities, complete coverage, a finite ordered time interval of no more than five seconds, and an integer result. That window is a teaching parameter—not a production tuning recommendation.

A positive byte count plus independently supplied “unexpected” context produces investigate-read. An approved diagnostic produces approved-read, which retains the observation rather than calling the machine clean. A normalized -1/EPERM produces denied-attempt. Missing, conflicting or unsupported evidence remains unknown. Zero bytes produces no-bytes-observed, not “safe.”

Normalization warning: the fixture uses libc-style -1 plus symbolic errno. A kernel exit sensor may expose a negative errno directly. A production adapter must normalize and test that convention. The compound identity strings are readable stand-ins for collector-issued identities; this model cannot verify their authenticity, host clock, coverage assertion or approval authority.

node --input-type=module -e '
import {readFileSync} from "node:fs";
import {assess} from "./kernel-detection-v1.mjs";
const fixture = JSON.parse(readFileSync("./kernel-detection-fixtures-v1.json", "utf8"));
for (const trace of fixture.cases) console.log(trace.id, assess(trace));
'

Download reference evaluator v1 · JavaScript module · Download synthetic evidence v1 · JSON

What a production implementation still needs

This is not a kernel collector, malware classifier or SIEM integration. Build and test the trusted ingestion adapter, immutable process/object identities, event deduplication, bounded buffering, late-arrival policy and retention controls. If you support out-of-order events, distinguish reorderable arrival time from contradictory event time. Never join different boots or process lifetimes merely because a PID matches.

Replace the fixture’s approval label with an independently verified entitlement or bounded maintenance record. A process-controlled name, command line, environment variable or annotation cannot grant itself an exception. Bind exceptions to workload identity, operation, target scope and expiry. Treat incomplete instrumentation as degraded detection coverage and surface it separately from suspicious activity.

Detection versus response

Start in observation mode. A false positive on a node security agent can remove the very telemetry needed to investigate. Escalate with the raw event references and context, not an automatic kill based on one syscall. For enforcement, specify the supported control, compatibility tests, rollback and a separate proof that the deployed policy rejects the intended operation.

08eBPF is a programmable kernel surface—not a security verdict

eBPF allows approved programs to run at supported kernel hooks, with program-type-specific access to context, helpers and data. Networking, observability and security tools use it for different purposes. A userspace loader prepares an object, creates maps, asks the kernel to load programs, then attaches them. Successful loading alone does not prove attachment or useful events. libbpf lifecycle.

1. 1 · Load requestThe loader’s credentials, kernel configuration and permissions constrain the requested program and attachment.
2. 2 · VerifyThe verifier analyses register types, pointer bounds and valid accesses before acceptance; rejection is a load failure, not a detection event.
3. 3 · Attach and runAccepted programs execute when their hook runs, through an interpreter or JIT-compiled code as supported. Hook type defines possible effects.
4. 4 · Export and correlateMaps or event buffers carry state to a collector. Loss, sampling, restarts and identity enrichment determine evidence quality.

The verifier enforces rules about program safety; it does not decide whether your organization authorized the loader or whether a telemetry policy covers the attack path. Verifier/JIT implementation defects also remain part of the kernel patching threat model. Verifier design.

Maps hold state shared between BPF programs and, subject to access, userspace. Configuration maps deserve change control; changing one can alter a sensor’s behavior without a new program load. Pinning can keep referenced objects alive beyond a loader’s lifetime, so process exit is not a complete uninstall audit. BPF maps and object lifetime.

Permission review: loader, program and attachment

CAP_BPF separates privileged BPF operations on supporting kernels; tracing may also require CAP_PERFMON, and networking operations may require CAP_NET_ADMIN. Legacy checks can involve CAP_SYS_ADMIN. These are not a universal three-capability recipe: program type, attach target, user namespace, LSM and distro configuration affect the decision. An application does not need these merely because the node runs an eBPF-based CNI. Linux capabilities.

Maintain an owner, image digest, program type, attachment, required authority and approved data fields for every sensor. Prefer hashes/build identities over program names as identity evidence. Treat raw arguments, paths and process data as potentially sensitive; do not export secrets in pursuit of a richer dashboard.

09What a container escape actually changes

A container escape is an action that crosses the workload’s intended isolation boundary into host resources or authority it should not possess. The phrase describes the result, not one particular exploit. Access to a host directory intentionally mounted into a pod may be a dangerous grant rather than a bypass; record the intended policy before calling it an escape.

There are at least three different mechanisms: an over-granted interface, a vulnerable userspace runtime, and a vulnerable kernel. They require different fixes. MITRE T1611, Escape to Host, is an investigation category—not proof that a weak configuration has already been exploited.

How does escape impact the kernel?

Root inside a container, visibility of host PIDs, an allowed syscall, and successful host access are four different observations. Report the narrowest supported claim. A service-account token used against the Kubernetes API can produce a serious cluster attack path without any kernel escape at all.

10Collect effective state before assigning severity

Start with API metadata. Preserve node and pod UIDs, container identity, release, timestamps and collection errors. The following triage includes regular, init and ephemeral containers; it deliberately does not fetch Secret objects or values.

bash# Authorized read-only API inventory; jq 1.6+.
oc get clusterversion -o jsonpath='{.items[0].status.desired.version}{"\n"}'
oc get mcp
oc get co
oc get pods -A -o json | jq -c '
  .items[] as $p
  | (($p.spec.containers // []) + ($p.spec.initContainers // [])
    + ($p.spec.ephemeralContainers // []))[] as $c
  | select(
      $c.securityContext.privileged == true
      or $p.spec.hostPID == true or $p.spec.hostIPC == true
      or $p.spec.hostNetwork == true
      or any($p.spec.volumes[]?; has("hostPath"))
      or any($c.securityContext.capabilities.add[]?;
        . == "SYS_ADMIN" or . == "SYS_PTRACE" or . == "SYS_MODULE"
        or . == "BPF" or . == "PERFMON" or . == "NET_ADMIN"))
  | {namespace:$p.metadata.namespace, pod:$p.metadata.name,
     podUID:$p.metadata.uid, node:$p.spec.nodeName,
     serviceAccount:($p.spec.serviceAccountName // "default"),
     scc:($p.metadata.annotations["openshift.io/scc"] // "UNKNOWN"),
     container:$c.name, securityContext:$c.securityContext}'

This flags candidate paths, not vulnerabilities. Inspect volume mounts, device exposure, runtime defaults and the actual process credentials next. For SCC entitlement, combine RBAC grants with the SCC’s own users/groups; a single oc auth can-i result is not a complete SCC grant inventory. Red Hat SCC reference.

Host inspection has a different authorization boundary

oc debug node is not a read-only Kubernetes operation: it creates a privileged debugging workload. chroot /host changes the filesystem view; it does not manufacture safe access. Use an approved administrator session and an explicitly selected canary node. Do not grant host-debug privilege to ordinary application accounts or to the production K8SEC Agent.

Within an already approved host inspection session, the following script reads metadata. Missing files and denied reads remain UNKNOWN. It does not prove every control is enforced or capture a trusted forensic image.

bash#!/bin/sh
# Run in an already authorized host inspection session, not inside an app pod.
# Prints metadata only. Does not change sysctls, load BPF or read Secret values.
show_file() {
  printf '\nFILE %s\n' "$1"
  if [ -r "$1" ]; then
    cat "$1" || printf 'UNKNOWN: read failed\n'
  else
    printf 'UNKNOWN: unavailable or not readable\n'
  fi
}
date -u '+UTC %Y-%m-%dT%H:%M:%SZ'
hostname
uname -r
show_file /proc/sys/kernel/random/boot_id
show_file /proc/cmdline
show_file /sys/fs/selinux/enforce
show_file /sys/kernel/security/lsm
show_file /sys/kernel/security/lockdown
for key in kernel.yama.ptrace_scope kernel.unprivileged_bpf_disabled \
  kernel.perf_event_paranoid kernel.kptr_restrict kernel.dmesg_restrict \
  kernel.randomize_va_space fs.suid_dumpable; do
  printf '\nSYSCTL %s\n' "$key"
  sysctl "$key" || printf 'UNKNOWN: sysctl failed\n'
done
for f in /sys/devices/system/cpu/vulnerabilities/*; do
  [ -e "$f" ] || continue
  show_file "$f"
done

Download the read-only host snapshot script (SH · v1). Review before use; this article does not claim it has been run on your OpenShift workers.

For ptrace restrictions, use the Yama semantics. For kernel knobs, use the sysctl reference, and map the actual OpenShift/RHCOS payload to vendor advisories instead of comparing upstream version strings alone.

11Break the path without breaking the cluster

Choose controls for the transition they constrain. Admission governs requested privileges; runtime policy governs execution; patching removes known defects. A sensor can support all three with evidence, but it does not substitute for any of them.

restricted-v2 drops capabilities and applies runtime-default seccomp; newer SCC options and user namespaces depend on release/configuration. Inspect the admitted pod rather than assuming a default. Avoid blanket changes to built-in SCCs. SCC behavior.

A scoped TuneD example—not a universal baseline

Evaluate host sysctls through the Node Tuning Operator. This example inherits openshift-node and matches one explicit label; lower priority numbers win. Adapt inheritance and conflicts to your pool, especially performance-tuned nodes. NTO profile matching.

yamlapiVersion: tuned.openshift.io/v1
kind: Tuned
metadata:
  name: k8sec-kernel-canary
  namespace: openshift-cluster-node-tuning-operator
spec:
  profile:
    - name: k8sec-kernel-canary
      data: |
        [main]
        summary=K8SEC single-worker evaluation
        include=openshift-node
        [sysctl]
        kernel.yama.ptrace_scope=1
        kernel.unprivileged_bpf_disabled=2
        kernel.perf_event_paranoid=2
        kernel.kptr_restrict=1
        kernel.dmesg_restrict=1
        fs.suid_dumpable=0
  recommend:
    - match:
        - label: k8sec.io/kernel-canary
          value: "true"
          type: node
      priority: 10
      profile: k8sec-kernel-canary

unprivileged_bpf_disabled=2 disables unprivileged BPF reversibly on supporting kernels; 1 cannot be reversed until reboot. Neither revokes all privileged BPF access. Check existing values first. kptr_restrict and dmesg_restrict reduce information exposure, not exploitability to zero. Kernel knob semantics. Profiling restrictions need separate evaluation for legitimate agents and CAP_PERFMON: perf security.

1. Baseline: choose a replaceable worker; record its active TuneD profile, original values, BPF consumers, operator health and application SLOs.
2. Review: server-side dry-run the manifest, check inherited settings and ensure only the intended node bears the canary label.
3. Change: apply only in the approved window. Confirm selected profile, live values and positive/negative control results—not just an accepted YAML object.
4. Compatibility: exercise CNI, CSI, APM, security sensors, profiling, crash collection and workload probes before widening scope.
5. Rollback: remove the canary match and restore the recorded prior profile/values through the supported operator process. Verify every value. Removing a label alone is not proof that all sysctls reverted.

Kernel arguments and RHCOS lifecycle changes need separate Machine Config Operator planning, disruption budgets and maintenance windows. MachineConfig guidance. An SELinux-disabled boot is not repaired by setenforce 1; inspect the configuration source and plan supported recovery. Do not turn broad audit2allow output into an unreviewed exception.

12Operate the detector, not just the rule

A detection is a maintained system: sensor, parser, identity mapping, correlation, enrichment, alert route and reviewer. Measure each part. A rule that never alerts because its event vanished after a kernel update is broken, not precise.

• Coverage: maintain a kernel/architecture/runtime/sensor matrix. Replay the expected allowed and denied canaries after upgrades. Record unsupported configurations.
• Evidence quality: track lost records, queue latency, attachment failures, unresolved identities and clock discontinuities. Keep UNKNOWN visible instead of counting it as a passed negative control.
• Evaluation: report precision on labeled reviewed alerts, and recall on your explicitly bounded test corpus—not all malware. Also measure false positives per workload-hour, investigation time and repeated alerts per incident.
• Privacy: default to metadata. Arguments, paths, environment and buffers may expose credentials or tenant data. Minimize collection, restrict access, redact deliberately and document the evidence lost through redaction.
• Cluster correlation: join runtime subject identity to pod UID, container ID, image digest, node and Kubernetes audit records. The ServiceAccount is not the Linux UID, and Kubernetes RBAC does not explain every kernel decision.
• Resilience: forward evidence off-node under controlled access. A compromised kernel can undermine host telemetry; a clean local event stream is not proof of kernel integrity.

Operator cheat sheet

Request only: attempted operation. Failure: rejected or failed call, cause needs attribution. Successful bytes: transfer for this call, not content classification. Approval: independent contextual record. Missing telemetry: unknown. Same PID: insufficient identity. Same time window: insufficient causality.

This checklist is a reading aid, not an automated security score. Ticks reset when the page reloads.

Source and validation boundary

Mechanism references are linked alongside the relevant claims; reviewed October 17, 2026. The correlation contract and investigation guidance are K8SEC’s analysis. Automated tests validate synthetic logic, page behavior and links; they do not validate a live kernel sensor or establish real-world malware detection rates.

Continue with Kubernetes audit-log investigations and kernel boundaries and OpenShift hardening. K8SEC Agent remains read-only Kubernetes graph intelligence: no Secret-value collection, pod exec, resource mutation or autonomous exploitation.