General

https://tetragon.io/

https://github.com/cilium/tetragon - 4.2K stars

https://isovalent.com/blog/runtime-security/ - blog posts that cover Tetragon and other solutions as well

https://isovalent.com/blog/ebpf/ - blog posts that cover Tetragon and other solutions as well

https://cilium.slack.com/signup#/domain-signup - Slack community

NOTE this is a work-in-progress. There are still several areas that I plan on further investigating

NOTE one of the best places to start is this article

first released in 2022 (and donated by Isovalent to the CNCF) and reached 1.0 in 2023. It was developed by Isovalent, who were acquired by Cisco in 2024. It was in use before then internally as mentioned here: "We are excited to announce the Tetragon open source project. Tetragon is a powerful eBPF-based security observability and runtime enforcement platform that has been part of Isovalent Cilium Enterprise for several years. Today, we are open sourcing major parts as project Tetragon and open it up for collaboration with the entire community"

NOTE for the various components, refer to the Components / Architecture section

NOTE although you will see apiVersion: cilium.io/v1alpha1 in its CRDs, it is stable and has been used in production for years

NOTE remember that "eBPF programs can efficiently collect, filter, and aggregate data directly in the kernel, sending only relevant information to user space"

NOTE it is useful for attack surface reduction (and is very useful for this) but it isn’t going to do much against something like memory corruption and most CVEs

NOTE keep in mind the many Limitations

NOTE it has a little over a dozen pre-built policies that you can use. Some of them are solid useful

NOTE out of the box and by default, it logs all process starts and exits which can actually be quite useful once you do have a rule violation. You can also build a Process Tree using this

NOTE considering how much data it can collect, you have to balance how much you want to log and send to your SIEM

NOTE you can granularly control:

NOTE one of its abilities is to let you correlate binaries + network connections (IPs). So you can limit / monitor what IPs a given binary connects to (refer to the Observe and Terminal All TCP Traffic Not to localhost section for an example)

Note

overall, it is quite powerful. Rule writing requires that you understand the function + its parameters but is then quite flexible. However, it can only look at:

  • process names

  • file names

  • IP addresses (no DNS details, no HTTP URLs, etc.)

you can also use it as a FIM (File Integrity Monitoring) solution to detect writes to files, etc. It is a lot better than using something based on inotify, etc.

it can also block actions by doing one of the following:

so overall, you can replace auditd and most similar tools. There are some nice features that Falco has that this doesn’t

NOTE one pro is that you can update rules without restarting any processes or any downtime

"Tetragon is a runtime security tool that emerged from the Cilium CNCF project, which shares a long history of co-development with eBPF itself. Similar to Falco, Tetragon generates event information:

  • enriched with Kubernetes identity information when run in Kubernetes environments

  • timing details

  • information about process ancestry (it just includes the parent process details), which is invaluable for forensic investigations to determine how a security event came to take place.

In addition to providing runtime security observability, Tetragon takes advantage of some advanced eBPF capabilities:

Capability Details

In-kernel filtering

"Tetragon has a userspace agent for:

  • management and

  • to collect event data and forward it to an event database or security information and event management (SIEM).

But in contrast to Falco, comparison with policies all takes place within the kernel. For example, you might want a policy that lets you know when certain files are accessed. With Tetragon, the comparison against the file names that you’re interested in happens within the kernel so that only notable events make it as far as user space. This makes for significantly better performance (compared to Falco)"

NOTE so only the events, not the comparisons, etc. make it to userspace

NOTE refer to the Falco …​ section for a more thorough comparison of the two options and where each on excels

NOTE refer to the Components / Architecture section for more on the architecture

LSM API

"Tetragon rules can take advantage of the Linux Security Module (LSM) API. This is the same API that tools like AppArmor use to enforce static security policies, but when it’s used with eBPF, it can be used to apply more complex, dynamic policies (for example, so that they apply to new containers as they are created)"

NOTE for example, you have security_file_permission and others

NOTE you can use any of the Hook Points and perform any of the Enforcement options on these. So it uses the LSM API under the hood (from my understanding)

Enforcement

"Tetragon policies can define whether to:

  • report on an event

  • override a return value to provide enforcement

  • send a signal

  • kill the process responsible"

NOTE refer to the matchActions - Available Actions section for details

Tetragon policies are written as Kubernetes custom resources (CRDs) in YAML files (even if you use it outside Kubernetes - refer to the Example of Using It for a Single Policy section for details). This makes them easy to manage in a Kubernetes environment, but Tetragon also works on bare-metal or virtual machines or with non-Kubernetes containers. Here’s an example policy that uses SIGKILL for enforcement:

apiVersion: cilium.io/v1alpha1
kind: TracingPolicy
metadata:
  name: "example"
spec:
  kprobes:
  - call: "security_file_permission"                   1
    syscall: false
    args:                                              2
    - index: 0
      type: "file"
    - index: 1
      type: "int" # 0x04 is MAY_READ, 0x02 is MAY_WRITE
    selectors:
    - matchArgs:                                       3
      - index: 0
        operator: "Equal"
        values:
        - "/tmp/liz"
      matchActions:                                    4
      - action: Sigkill

from the above, you have:

Marker Details

1

This policy attaches to a kernel function called security_file_permission(), which is part of the LSM API and is called by the kernel every time a process wants to access a file for reading or writing. It’s used to control whether the operation should be permitted or not. So here we aren’t looking at specific syscalls (you’ll also notice syscall: false on the line after it)

2

There are two arguments to this function:

  • the first is a pointer to a kernel file data structure, and

  • the second is a value indicating whether the file is looking for read or write access

NOTE we get these arguments from the function’s definition (so we would look this up the definition of security_file_permission and what parameters it takes and specify them) This is needed by Tetragon if we want to:

  • print these values in the output (they would not be printed if you didn’t explicitly specify them)

  • use any of these arguments in our logic

for more on this, refer to both the Arguments and the security_file_permission_example sections

3

The policy defines some selectors that are used to filter events. In this case, Tetragon will only act on the event if the file indicated by the first parameter has the name /tmp/liz. - NOTE refer to this section for more details on this

NOTE selectors can also be used to specify which containers, pods and namespaces a specify policy applies to. Refer to the Kubernetes Identity Aware Policies / Apply Policies to Specific Pods section for details

4

If the selectors match, Tetragon sends a SIGKILL to terminate the responsible process (and also reports the event to user space). - NOTE refer to the actions section for a list of all your options

When an application, possibly compromised, attempts to access this file, a Tetragon eBPF program in the kernel is triggered. Without any transition to userspace, eBPF code compares the event to the policy, and if all the conditions are met, it sends the SIGKILL signal. This ensures that the out-of-policy action is never able to complete. - NOTE so the kprobe is doing all the filtering, etc. and only when you have a match is the eBPF program run

Tetragon also generates Prometheus-format metrics that can be visualized with a tool like Grafana or Splunk to easily spot whether and when issues have taken place or to identify which workloads are affected.

My employer, Isovalent (now part of Cisco), provides an enterprise distribution of Tetragon" - Liz Rice (from Container Security, 2nd Edition)

NOTE for how to quickly test the above policy without even needing Kubernetes, refer to the Example of Using It for a Single Policy section

NOTE this is what the output would look like (outside a Kubernetes environment):

{
  "process_kprobe": {
    "process": {
      "exec_id": "bG9jYWxob3N0LmxvY2FsZG9tYWluOjg3Mjk5OTQ0MzIyNTU4OjcwMjY=",
      "pid": 7026,
      "uid": 1000,
      "cwd": "/home/vagrant",
      "binary": "/usr/bin/cat",
      "arguments": "/tmp/liz",
      "flags": "execve clone",
      "start_time": "2025-10-19T02:00:55.041947879Z",
      "auid": 1000,
      "parent_exec_id": "bG9jYWxob3N0LmxvY2FsZG9tYWluOjUxNDU0MDAwMDAwMDozMTA3",
      "refcnt": 1,
      "tid": 7026,
      "in_init_tree": false
    },
    "parent": {
      "exec_id": "bG9jYWxob3N0LmxvY2FsZG9tYWluOjUxNDU0MDAwMDAwMDozMTA3",
      "pid": 3107,
      "uid": 1000,
      "cwd": "/home/vagrant",
      "binary": "/usr/bin/bash",
      "flags": "procFS auid",
      "start_time": "2025-10-18T01:54:29.637624811Z",
      "auid": 1000,
      "parent_exec_id": "bG9jYWxob3N0LmxvY2FsZG9tYWluOjUxNDQ5MDAwMDAwMDozMTA2",
      "tid": 3107,
      "in_init_tree": false
    },
    "function_name": "security_file_permission",
    "args": [
      {
        "file_arg": {
          "path": "/tmp/liz",
          "permission": "-rw-r--r--"
        }
      },
      {
        "int_arg": 4
      }
    ],
    "action": "KPROBE_ACTION_SIGKILL",
    "policy_name": "example",
    "return_action": "KPROBE_ACTION_POST"
  },
  "node_name": "localhost.localdomain",
  "time": "2025-10-19T02:00:55.043721804Z"
}

NOTE so we can see the action that was taken: KPROBE_ACTION_SIGKILL in addition to:

  • process details

  • parent process details

NOTE notice also the args section which are the arguments that we specified (it adds a _arg to the name). So we had:

    args:
    - index: 0
      type: "file"
    - index: 1
      type: "int" # 0x04 is MAY_READ, 0x02 is MAY_WRITE

and ended up with:

    "args": [
      {
        "file_arg": {
          "path": "/tmp/liz",
          "permission": "-rw-r--r--"
        }
      },
      {
        "int_arg": 4
      }
    ]

in our output

Documentation

https://tetragon.io/docs

NOTE overall, the documentation is pretty good but not comprehensive. For example, I found Operators which had no documentation related to them

Components / Architecture

here is what it looks like:

tetragon diagram 1

so you have the following:

Component Details

Tetragon Agent

this is responsible for:

  • loading and applying configuration

  • loading and applying policies

  • reading events sent from kernel-space

  • generating metrics related to both:

    • Tetragon’s health

    • the activity of processes observed by Tetragon

Linux Kernel

here is where you have the bulk of your work. This is the reason that it quite a bit faster than something like Falco

refer to the Everything is Done in the Kernel section for details

Containers / Processes

these are what are being monitored

Everything is Done in the Kernel

"Smart in-kernel collection logic allows filtering and aggregating data as soon as they are generated in the kernel with the eBPF-based collector. This approach minimizes overhead by reducing the need for unnecessary data transfer between kernel and user space or relying on static kernel probes" - https://isovalent.com/blog/post/tetragon-release-10/

NOTE the only time it does anything in user space is when generating an event

NOTE unlike other options like Falco, it does almost everything in-kernel. This first diagram is what other solutions do:

tetragon diagram 3

and this is what Tetragon does:

tetragon diagram 4

Tetragon vs Alternatives

Falco, Tetragon, KubeArmor, or Tracee? Choosing the Right Security Agent

Falco …​

Falco vs. Tetragon: A Runtime Security Showdown for Kubernetes

What are the key differences between Falco and tetragon? · Issue #701 · cilium/tetragon

NOTE want to add more details to this section

both use eBPF

Tetragon is a low level eBPF toolkit that you can use to build a runtime detection and prevention system

Falco is an intrusion detection system designed to detect and respond (using Falco Talon). Its response system is not inline. It also comes with a lot of detection rules that can get you up and running

overall, the summary over here is pretty good:

Feature Tetragon Falco

Technology Base

eBPF

Kernel module / eBPF

Performance Impact

Very Low

Low to Moderate

Kubernetes Integration

Native

Native

Rule Language

YAML-based policies

Falco rules (YAML)

Event Types

System calls, network events, file access

System calls, network events

Real-time Enforcement

Yes

Yes (with Falco Talon)

Deployment Complexity

Low (Kubernetes-native)

Low to Moderate

Community Support

Growing (part of Cilium project)

Well-established

Cloud Provider Support

Broad

Broad
Prevention and Response Options

NOTE both of them support responses but:

  • Tetragon lets you do response inline before the corresponding syscall / action is taken by killing a process, etc. Falco doesn’t. While Tetragon lets you send to a URL or DNS, these options are limited (you can’t make them dynamic based on the data that you have)

  • Falco does let you respond via Falco Talon which was introduced in 2024. It is not inline but does offer you a lot more options in terms of response including:

    • killing a Kubernetes pod

    • invoking an AWS Lambda function

    • creating a network policy to block egress traffic to specific IPs

    • starting collecting syscalls for the pod

    • downloading files from a pod

    • etc

Performance

according to this:

Tool Overhead

Falco

adds 5-10% overhead by parsing syscalls in userspace.

so it is somewhat resource-heavy in high-throughput clusters

NOTE refer to the Falco document for more on its performance

Tetragon

runs <1% overhead via eBPF kernel hooks

so it is safe for production-grade workloads

NOTE for more on its performance, refer to the Performance section
Rulesets / Policies

Falco comes ready with out of the box policies / rules. Tetragon has a few as well

Limitations

Rules / Protection

Writing Complex Rules More Difficult Than Alternatives

it is easier to write sophisticated rules using something like Falco

on a positive note, the team knows this as mentioned here and are working on a solution. This is what the ticket says:

"The existing TracingPolicy is powerful but provides a very small abstraction over kernel mechanisms such as kprobes, uprobes, or tracepoints and it might be difficult for users to use them. We could create new policies that could eventually translate to "low-level" (existing) TracingPolicies but provide a nice UX for users"

Low Level Details

in general, you are only looking at the syscall / kprobes / etc arguments and making decisions on that. There is no support for actually parsing a given payload and extracting things from it. So you can’t inspect the hostname being queried in DNS and base your rule on that. But you can detect a DNS query (but by port and corresponding probes, etc.)

Kernel-Level Exploit Protection Very Limited

https://twitter.com/wipawel/status/1527638600267046915

grsecurity - Tetragone: A Lesson in Security Fundamentals

  • https://grsecurity.net/tetragone_a_lesson_in_security_fundamentals

  • NOTE I agree with this: "Tetragon could be useful for attack surface reduction, preventing user-land programs from leveraging certain interfaces they do not need. If a thorough analysis of a published security vulnerability is performed, like live-patching or other approaches, it could also be capable of preventing the reachability of that specific known vulnerability. But its post-exploitation mitigation certainly cannot be expected in general to mitigate nor even reliably detect kernel-level exploits" - using it for prevention of memory corruption, etc. is never going to win. But it is useful for limiting attack surface

You Specify Binaries Using the Full Path

so you have the same limitation that you have in AppArmor (SELinux is better in this case as it is label-based)

No Learning Mode / Profile Builder for Containers

you have to define this yourself. Refer to the Automatically Generating Profiles for Containers section for a semi-automated way of going about doing so

Can’t Write Rules That Look At How Long a Process Has Been Running

in tools like Falco, you have a macro such as this that takes into consideration how long a process has run (as reading config files, etc. is something typical at startup time but not afterwards):

- macro: proc_is_new
  condition: (proc.duration <= 5000000000)

and then have a rule that uses it such as this:

- rule: Read sensitive file trusted after startup
  desc: >
    An attempt to read any sensitive file (e.g. files containing user/password/authentication
    information) by a trusted program after startup. Trusted programs might read these files
    at startup to load initial state, but not afterwards. Can be customized as needed.
    In modern containerized cloud infrastructures, accessing traditional Linux sensitive files
    might be less relevant, yet it remains valuable for baseline detections. While we provide additional
    rules for SSH or cloud vendor-specific credentials, you can significantly enhance your security
    program by crafting custom rules for critical application credentials unique to your environment.
  condition: >
    open_read
    and sensitive_files
    and server_procs
    and not proc_is_new
    and proc.name!="sshd"
    and not user_known_read_sensitive_files_activities
  output: Sensitive file opened for reading by trusted program after startup | file=%fd.name pcmdline=%proc.pcmdline gparent=%proc.aname[2] ggparent=%proc.aname[3] gggparent=%proc.aname[4] evt_type=%evt.type user=%user.name user_uid=%user.uid user_loginuid=%user.loginuid process=%proc.name proc_exepath=%proc.exepath parent=%proc.pname command=%proc.cmdline terminal=%proc.tty
  priority: WARNING
  tags: [maturity_stable, host, container, filesystem, mitre_credential_access, T1555]

you don’t have the equivalent in Tetragon

No Macro Support

Falco’s macro support makes writing rules easier. While Tetragon, like Falco, supports lists, it doesn’t support macros

uprobes Aren’t Practical for Containers

NOTE at the current time, using uprobes isn’t really practical for containers due to Limitations but is valid for normal workloads

For Network-Related Rules, You Can’t Use Hostnames

you can’t use hostnames in your rules and have to use IP addresses. For an example of this, refer to the Observe and Terminal All TCP Traffic Not to localhost section

QUESTION I wonder if I can use TrackSock for this

Actions

Integrating with Webhooks is Very Limited

although it supports GetUrl, you can’t pass in parts of the payload. It also doesn’t have a PostUrl, although there is a ticket to add support for this

NOTE one workaround is to have Fluent Bit, etc. just read the Tetragon logs and send to the webhook, etc.

Can’t Run Command As Response

doing so is not one of the supported actions

it would have been useful for collecting forensics. For example, Falco Talon supports pull files out of a container as well as starting to log syscall activity

if we are able to run commands, we can look into:

  • taking snapshots of containers / processes

  • using the Checkpoint API to create a stateful copy of a running container

  • start logging syscalls made by the container

Output

No Executable Hash Details in Output

out of the box, you don’t get this like you do in Sysmon which might have been useful. It does give you this for image IDs in Kubernetes

NOTE it looks like you can get this via File Hashes but this only works for LSM Hooks

Misc

Documentation Is Incomplete

for example, there are no examples (even in their git repo) for some operators

Performance

Tetragon 1.0: Kubernetes Security Observability & Runtime Enforcement

"A core tenet of Tetragon is to optimize for collection and filtering in-kernel, and only transfer events of interest into user space. This significantly reduces the operational overhead seen in other tools, where a large amount of CPU cycles are spent in moving events from kernel space to user space before filtering"

here is what the benchmark showed:

Details

Process Execution Tracking

"In this first benchmark, we evaluate a worst-case scenario with a stress test by building the 6.1.13 Linux kernel which is an incredibly process execution heavy operation leading to approximately ~1.5M security relevant events at a rate of 2.6K events/s. A typical production workload will not expose this volume of exec and exit events, meaning we are benchmarking a worst-case scenario so this should provide a good upper ceiling of maximum overhead expected"

tetragon performance 1

NOTE in this case, it added 1.68% overhead (and 2.46% if we enabled JSON output)

File Monitoring

"Monitoring file I/O is traditionally challenging because there is typically an incredibly high number of I/O operations happening on a system, in particular when databases or other data heavy workloads are being run. This benchmark test runs a workload that performs a high volume of reads. We use a workload performing 1K reads in sequence as fast as possible, then sleeping for one nanosecond. This sequence is performed across 32 threads in parallel, resulting in a very I/O intensive workload"

tetragon performance 2

"Tetragon is configured with a File Integrity Monitoring filter that matches on no activity produced by the executed workload. This represents a policy that is looking for ''needle in a haystack" suspicious activity and quantifies Tetragon’s ability to filter out noise while monitoring a large number of file operations"

NOTE so it was very little overhead

Network Monitoring

"To benchmark this use case, we run a workload that opens new connections at a very high rate using netperf TCP_CRR to create a worst-case scenario and establish the performance overhead ceiling. A real-world scenario where this load may occur is running a L7 load-balancer or a web frontend. We then measure the overhead for two specific use-cases:

  1. monitoring all traffic for suspicious activity which involves tracking connections applying a filter to look for particular activity and

  2. logging every connection attempt as an event in the audit log"

"This benchmark demonstrates Tetragon’s minimal impact on monitoring large-scale network events. With Tetragon, the connection rate decreases by 5.88%, maintaining an average rate of approximately ~26,250 requests per second"

NOTE so at very high rates, it decreased the number of requests per second by about 6%. If you enable JSON output, that jumps to 17%

tetragon performance 3

Export Filtering - Restrict What Events Go to Output

https://tetragon.io/docs/concepts/events/#export-filtering

NOTE this is actually very flexible and you can filter on a lot of options (pod name, pod label, executable name, parent executable name, etc.). Refer to the List of Event Filters section for more

"Export filters restrict the JSON event output to a subset of desirable events. These export filters are configured as a line-separated list of JSON objects. Each object can contain:

  • one or

  • more than one

filter expressions.

Filters are combined by taking:

  • the logical OR of each line-separated filter object and

  • the logical AND of sibling expressions within a filter object"

for example:

{"event_set": ["PROCESS_EXEC", "PROCESS_EXIT"], "namespace": ["foo"]}
{"event_set": ["PROCESS_KPROBE"]}

the above means to match if:

  • the event type is PROCESS_EXEC OR PROCESS_EXEC AND the namespace is foo

  • the event type is PROCESS_KPROBE

the above two are combined using an OR

you have two options:

you can have one of the following combinations:

Option Details

No Allowlist + No Denylist

all events are exported

Allowlist Only

only events in the allowlist are exported

Denylist Only

all events not matching the denylist are exported

Allowlist + Denylist

denylist is applied first and then the allowlist is applied

NOTE as mentioned in the documentation, these apply only to events exported to JSON files and does not affect the gRPC API (so can still use the tetra CLI to get everything without any filtering)

Files

Allowlist

for non-Kubernetes setups, you would use the following file:

/etc/tetragon/tetragon.conf.d/export-allowlist

for example, to log only invocations of vim, you would have something like this:

{"event_set": ["PROCESS_EXEC", "PROCESS_EXIT"], "binary_regex": ["^/usr/bin/vim$"]}

and you would see logs like this:

{
    "process_exec": {
        "process": {
            "exec_id": "bGludXg6MzgxMjI3MzQwMDAwMDAwOjE5ODgy",
            "pid": 19882,
            "uid": 0,
            "cwd": "/etc/tetragon/tetragon.tp.d",
            "binary": "/usr/bin/vim",
            "arguments": "/tmp/network-monitoring.yaml",
            "flags": "procFS auid",
            "start_time": "2025-10-22T11:39:42.437624780Z",
            "auid": 1000,
            "parent_exec_id": "bGludXg6MzgxMjAxNzIwMDAwMDAwOjE5ODA4",
            "tid": 19882,
            "in_init_tree": false
        },
        "parent": {
            "exec_id": "bGludXg6MzgxMjAxNzIwMDAwMDAwOjE5ODA4",
            "pid": 19808,
            "uid": 0,
            "cwd": "/var/log/tetragon",
            "binary": "/usr/bin/bash",
            "flags": "procFS auid",
            "start_time": "2025-10-22T11:39:16.817624791Z",
            "auid": 1000,
            "parent_exec_id": "bGludXg6MzgxMjAxNzIwMDAwMDAwOjE5ODA3",
            "tid": 19808,
            "in_init_tree": false
        }
    },
    "node_name": "linux",
    "time": "2025-10-22T11:39:42.437624740Z"
}
Denylist
Non-Kubernetes Setups

for non-Kubernetes setups, you would use the following file:

/etc/tetragon/tetragon.conf.d/export-denylist

you would then have something like this (taken from here):

{"event_set": ["PROCESS_EXEC", "PROCESS_EXIT"], "binary_regex": ["^/bin/bash$", "^/usr/bin/bash$"], "arguments_regex": ["/usr/share/socfortress/scripts/open-audit.sh"]}
{"event_set": ["PROCESS_EXEC","PROCESS_EXIT"], "parent_binary_regex": ["^/usr/bin/bash$"], "parent_arguments_regex": ["/usr/share/socfortress/scripts/open-audit.sh"]}

this is what the first entry looks like prettified:

{
  "event_set": [
    "PROCESS_EXEC",
    "PROCESS_EXIT"
  ],
  "binary_regex": [
    "^/bin/bash$",
    "^/usr/bin/bash$"
  ],
  "arguments_regex": [
    "/usr/share/socfortress/scripts/open-audit.sh"
  ]
}
Kubernetes Setups

you would edit the ConfigMap and update it. For example:

export-denylist: |-
  {"health_check":true}
  {"namespace":["", "cilium", "kube-system","cert-manager","dynatrace"]}

List of Event Filters

https://tetragon.io/docs/concepts/events/#list-of-process-event-filters

NOTE you have a lot of options here such as:

  • filtering based on event type (process execution, process exit, kprobe, etc.)

  • binary regex

  • parent binary regex

  • ancestor binary regex (ancestor_binary_regex)

  • arguments regex

  • parent arguments regex

  • PID

  • Kubernetes namespace

  • Kubernetes pod name

  • Kubernetes label

  • container ID

  • CEL (Common Expression Library) expression

    • refer to the CEL document for more on this

Field Filters / Export Only Specific Fields

https://tetragon.io/docs/concepts/events/#field-filtering

the action field is either: INCLUDE or EXCLUDE causes the field to be either included or excluded

{"fields":"process.exec_id,process.parent_exec_id", "event_set": ["PROCESS_EXEC"], "invert_event_set": true, "action": "INCLUDE"}

this is what it looks like prettified:

{
  "fields": "process.exec_id,process.parent_exec_id",
  "event_set": [
    "PROCESS_EXEC"
  ],
  "invert_event_set": true,
  "action": "INCLUDE"
}

so here we tell it log only the process.exec_id and process.parent_exec_id fields for all event types that are not PROCESS_EXEC (because of invert_event_set: true)

Non-Kubernetes Setup
--field-filters string                       Field filters for event exports

for example:

Step Details

Define Your Field Filter

NOTE keep in mind that it only applies if there is a match. If there is no match, then the default filter is used (and it shows you all fields)

for example:

{
  "fields": "process.exec_id,process.parent_exec_id,process.binary,process.arguments,process.pid,process.uid,process.auid",
  "event_set": [
    "PROCESS_EXEC",
    "PROCESS_EXIT"
  ],
  "action": "INCLUDE"
}

JSON Escape It

you can use something like this: https://www.freeformatter.com/json-escape.html#before-output

Pass It In

for example:

tetragon --field-filters "{\"fields\":\"process.exec_id,process.parent_exec_id,process.binary,process.arguments,process.pid,process.uid,process.auid\", \"event_set\": [\"PROCESS_EXEC\", \"PROCESS_EXIT\"], \"action\": \"INCLUDE\"}\r\n"

Check the Output

and this is what the output might look like:

{
  "process_exec": {
    "process": {
      "exec_id": "bGludXg6NDE2NTIwOTM4OTgzNDU2OjIyODEx",
      "pid": 22811,
      "uid": 0,
      "binary": "/usr/bin/vim",
      "arguments": "/tmp/network-monitoring.yaml",
      "auid": 1000,
      "parent_exec_id": "bGludXg6MjYzOTU3ODEwMDAwMDAwOjE0NzY0"
    }
  },
  "node_name": "linux",
  "time": "2025-10-22T21:27:56.036608236Z"
}
{
  "process_exit": {
    "process": {
      "exec_id": "bGludXg6NDE2NTIwOTM4OTgzNDU2OjIyODEx",
      "pid": 22811,
      "uid": 0,
      "binary": "/usr/bin/vim",
      "arguments": "/tmp/network-monitoring.yaml",
      "auid": 1000,
      "parent_exec_id": "bGludXg6MjYzOTU3ODEwMDAwMDAwOjE0NzY0"
    }
  },
  "node_name": "linux",
  "time": "2025-10-22T21:27:57.229186129Z"
}
Kubernetes Setup

you would edit the Kubernetes ConfigMap and do something like this:

field-filters: |
  {"fields":"process.binary,process.arguments,process.cwd,process.pod.name,process.pod.container.name,process.pod.namespace,process.flags,process.tid,process.pod.workload_kind,parent.binary,parent.arguments,args,status,signal,message,policy_name,function_name,event,file.path"}

Sanitize / Redact Sensitive Information From Events

https://tetragon.io/docs/concepts/events/#redacting-sensitive-information

https://github.com/cilium/tetragon/issues/2241

NOTE this is supported

NOTE it uses RE2. You can test it over here

"Since Tetragon traces the entire system, event exports might sometimes contain sensitive information (for example, a secret passed via a command line argument to a process). To prevent this information from being exfiltrated via Tetragon JSON export, Tetragon provides a mechanism called Redaction Filters which can be used to string patterns to redact from exported process arguments. These filters are written in JSON and passed to the Tetragon agent via the --redaction-filters command line flag or the redactionFilters Helm value"

here is an example:

{"binary_regex": ["(?:^|/)foo$"], "redact": ["-p(?:\\s+|=)(\\S*)"]}

so the above would:

  • match binary executables that end with foo

  • if it takes a -p argument, then redact this argument

Sanitize All Arguments

here is a simple example:

tetragon --redaction-filters "{\"binary_regex\": [\"(?:^|\/)vim$\"], \"redact\": [\"(.*)\"]}"

the output would look like this:

      ...
      "binary": "/usr/bin/vim",
      "arguments": "*****",
      ...

Isovalent Enterprise Platform

https://isovalent.com/product/

Tetragon - eBPF-based Security Observability & Runtime Enforcement

NOTE want to look into this

Misc

Can You Use It In Standalone Mode (Without Kubernetes)?

https://tetragon.io/docs/installation/faq/#can-i-install-and-use-tetragon-in-standalone-mode-outside-of-k8s - yes

refer to the Standalone section for details

How to Resolve UIDs / Usernames

you can do this by:

--username-metadata	"unix"

and now a process entry looks like this:

"process": {
  "exec_id": "bGludXg6NDcyNDA1NzE2NzkwNzcwOjI3Njky",
  "pid": 27692,
  "uid": 0,
  "cwd": "/var/log/tetragon",
  "binary": "/usr/local/bin/jq",
  "flags": "execve clone",
  "start_time": "2025-10-23T12:59:20.814415941Z",
  "auid": 1000,
  "parent_exec_id": "bGludXg6NDMxODI3NjEwMDAwMDAwOjI0MjA0",
  "tid": 27692,
  "user": {
    "name": "root"
  },
  "in_init_tree": false
},

Can You Do Things Such as Kill Processes Using syscalls?

yes. Refer to the Signal - Send Specified Signal to Current Process for an example. It also supports sending any signal you want to a process

Building a Process Tree / Ancestor List

you have several options here

Using the Enterprise Version

this is included in the Enterprise version as mentioned link:https://github

Using --enable-process-ancestors
Non-Kubernetes Environments

https://github.com/cilium/tetragon/pull/2938

this was merged. So you would add as a CLI option and you would now get the entire process ancestry

NOTE I verified that this works

here is an example of the output:

{
  "process_exit": {
    "process": {
      "exec_id": "bGludXg6NDM4OTU1MjEyNDA5NjI5OjI0NjU0",
      "pid": 24654,
      "uid": 0,
      "cwd": "/",
      "binary": "/usr/lib/systemd/systemd-nsresourcework",
      "arguments": "xxxxxxxxxxxxxxxx",
      "flags": "execve rootcwd clone",
      "start_time": "2025-10-23T03:41:50.310034580Z",
      "auid": 4294967295,
      "parent_exec_id": "bGludXg6NDQxMDAwMDAwMDo2Mzk=",
      "tid": 24654,
      "in_init_tree": false
    },
    "parent": {
      "exec_id": "bGludXg6NDQxMDAwMDAwMDo2Mzk=",
      "pid": 639,
      "uid": 0,
      "cwd": "/",
      "binary": "/usr/lib/systemd/systemd-nsresourced",
      "flags": "procFS auid rootcwd",
      "start_time": "2025-10-18T01:45:59.507624780Z",
      "auid": 4294967295,
      "parent_exec_id": "bGludXg6MzAwMDAwMDAwOjE=",
      "tid": 639,
      "in_init_tree": false
    },
    "time": "2025-10-23T03:46:51.359816431Z",
    "ancestors": [
      {
        "exec_id": "bGludXg6MzAwMDAwMDAwOjE=",
        "pid": 1,
        "uid": 0,
        "cwd": "/",
        "binary": "/usr/lib/systemd/systemd",
        "arguments": "--switched-root --system --deserialize=47",
        "flags": "procFS auid rootcwd",
        "start_time": "2025-10-18T01:45:55.397624980Z",
        "auid": 4294967295,
        "parent_exec_id": "bGludXg6MTow",
        "tid": 1,
        "in_init_tree": false
      }
    ]
  },
  "node_name": "linux",
  "time": "2025-10-23T03:46:51.359816222Z"
}

here we have one parent and one ancestor but it can have as many ancestors as you have

Kubernetes

you would edit the ConfigMap. By default, it looks like this:

enable-ancestors: ""

you would change this to:

enable-ancestors: "true"

and then:

kubectl rollout restart daemonset tetragon -n kube-system
kubectl rollout status daemonset tetragon -n kube-system
Building It Based on Normal Event Logs

NOTE because it logs process starts and exits out of the box, once you have a security violation, you can backtrack the who, what, where. In their Enterprise version, they give you a UI for this that looks like this but you can do this manually if you are sending to your SIEM, etc (but they support the process tree if you are using Splunk)

tetragon process tree 1

also, it generates a unique exec_id for processes, so you don’t have to worry about PID reuse:

"process": {
  "exec_id": "bG9jYWxob3N0LmxvY2FsZG9tYWluOjEwMDAzMjQ2MDY4NDAyODo3Njky",
  "pid": 7692,
  "uid": 0,
  "cwd": "/home/vagrant",
  "binary": "/usr/bin/sudo",
  "arguments": "cat /var/log/tetragon/tetragon.log",
  "flags": "execve",
  "start_time": "2025-10-19T05:33:07.558308959Z",
  "auid": 1000,
  "parent_exec_id": "bG9jYWxob3N0LmxvY2FsZG9tYWluOjEwMDAzMjQ1OTM2ODEzMzo3Njkx",
  "refcnt": 1,
  "tid": 7692,
  "in_init_tree": false
}

Metrics

https://tetragon.io/docs/reference/metrics/

https://tetragon.io/docs/installation/metrics/

it can generate Prometheus-format metrics for:

  • its own health

  • the activity of processes observed by it

"Tetragon generates Prometheus-format metrics that can be visualized with a tool like Grafana or Splunk to easily spot whether and when issues have taken place or to identify which workloads are affected"

Kubernetes

"in a Kubernetes installation, metrics are enabled by default and exposed via the endpoint /metrics.

The tetragon service exposes the Tetragon Agent metrics on port 2112, and

the tetragon-operator-metrics service the Tetragon Operator metrics on port 2113"

this is what it looks like:

# HELP certwatcher_read_certificate_errors_total Total number of certificate read errors
# TYPE certwatcher_read_certificate_errors_total counter
certwatcher_read_certificate_errors_total 0
# HELP certwatcher_read_certificate_total Total number of certificate reads
# TYPE certwatcher_read_certificate_total counter
certwatcher_read_certificate_total 0
# HELP controller_runtime_webhook_panics_total Total number of webhook panics
# TYPE controller_runtime_webhook_panics_total counter
controller_runtime_webhook_panics_total 0

# a bunch of go_* metrics

# HELP go_gc_duration_seconds A summary of the wall-time pause (stop-the-world) duration in garbage collection cycles.
# TYPE go_gc_duration_seconds summary
go_gc_duration_seconds{quantile="0"} 3.207e-05
go_gc_duration_seconds{quantile="0.25"} 6.9831e-05
go_gc_duration_seconds{quantile="0.5"} 0.000102112
go_gc_duration_seconds{quantile="0.75"} 0.000134482
go_gc_duration_seconds{quantile="1"} 0.014899851
go_gc_duration_seconds_sum 0.111819246
go_gc_duration_seconds_count 376
...

# a bunch of tetragon_* metrics

# TYPE tetragon_build_info gauge
tetragon_build_info{commit="",go_version="go1.24.5",modified="",time="",version="v1.5.0"} 1
# HELP tetragon_events_exported_total Total number of events exported
# TYPE tetragon_events_exported_total counter
tetragon_events_exported_total 2210
# HELP tetragon_events_last_exported_timestamp Timestamp of the most recent event to be exported
# TYPE tetragon_events_last_exported_timestamp gauge
tetragon_events_last_exported_timestamp 1.761284959e+09
# HELP tetragon_events_missing_process_info_total Number of events missing process info.
# TYPE tetragon_events_missing_process_info_total counter
tetragon_events_missing_process_info_total 0
# HELP tetragon_events_total The total number of Tetragon events
# TYPE tetragon_events_total counter
tetragon_events_total{binary="/bin/bash",namespace="",pod="",type="PROCESS_EXEC",workload=""} 1
tetragon_events_total{binary="/bin/bash",namespace="",pod="",type="PROCESS_EXIT",workload=""} 1
tetragon_events_total{binary="/etc/NetworkManager/dispatcher.d/90-console-login-helper-messages-gensnippet_if",namespace="",pod="",type="PROCESS_EXEC",workload=""} 17
tetragon_events_total{binary="/etc/NetworkManager/dispatcher.d/90-console-login-helper-messages-gensnippet_if",namespace="",pod="",type="PROCESS_EXIT",workload=""} 17
tetragon_events_total{binary="/proc/self/fd/16",namespace="",pod="",type="PROCESS_EXEC",workload=""} 19
tetragon_events_total{binary="/proc/self/fd/16",namespace="",pod="",type="PROCESS_EXIT",workload=""} 1
tetragon_events_total{binary="/proc/self/fd/9",namespace="",pod="",type="PROCESS_EXEC",workload=""} 1
tetragon_events_total{binary="/usr/bin/bash",namespace="",pod="",type="PROCESS_EXEC",workload=""} 4
tetragon_events_total{binary="/usr/bin/chronyc",namespace="",pod="",type="PROCESS_EXEC",workload=""} 34
tetragon_events_total{binary="/usr/bin/chronyc",namespace="",pod="",type="PROCESS_EXIT",workload=""} 34
tetragon_events_total{binary="/usr/bin/conmon",namespace="",pod="",type="PROCESS_EXEC",workload=""} 1
tetragon_events_total{binary="/usr/bin/curl",namespace="",pod="",type="PROCESS_EXEC",workload=""} 3
tetragon_events_total{binary="/usr/bin/curl",namespace="",pod="",type="PROCESS_EXIT",workload=""} 2
tetragon_events_total{binary="/usr/bin/curl",namespace="",pod="",type="PROCESS_KPROBE",workload=""} 3
tetragon_events_total{binary="/usr/bin/dbus-broker",namespace="",pod="",type="PROCESS_EXEC",workload=""} 1
tetragon_events_total{binary="/usr/bin/dbus-broker-launch",namespace="",pod="",type="PROCESS_EXEC",workload=""} 1
tetragon_events_total{binary="/usr/bin/grep",namespace="",pod="",type="PROCESS_EXEC",workload=""} 1
tetragon_events_total{binary="/usr/bin/grep",namespace="",pod="",type="PROCESS_EXIT",workload=""} 1
tetragon_events_total{binary="/usr/bin/mkdir",namespace="",pod="",type="PROCESS_EXEC",workload=""} 17
tetragon_events_total{binary="/usr/bin/mkdir",namespace="",pod="",type="PROCESS_EXIT",workload=""} 17
tetragon_events_total{binary="/usr/bin/rm",namespace="",pod="",type="PROCESS_EXEC",workload=""} 17
tetragon_events_total{binary="/usr/bin/rm",namespace="",pod="",type="PROCESS_EXIT",workload=""} 17
tetragon_events_total{binary="/usr/bin/sudo",namespace="",pod="",type="PROCESS_EXEC",workload=""} 4
tetragon_events_total{binary="/usr/bin/systemd-tmpfiles",namespace="",pod="",type="PROCESS_EXEC",workload=""} 2
tetragon_events_total{binary="/usr/bin/systemd-tmpfiles",namespace="",pod="",type="PROCESS_EXIT",workload=""} 2
tetragon_events_total{binary="/usr/bin/udevadm",namespace="",pod="",type="PROCESS_EXEC",workload=""} 1
tetragon_events_total{binary="/usr/bin/vim",namespace="",pod="",type="PROCESS_EXEC",workload=""} 1
...

Using It for Virtual Patching

eBPF and Tetragon: Tools for Detecting XZ Utils CVE 2024-3094 Exploit

NOTE you can but only for very few exploits. You can’t for memory corruption bugs, etc. Refer to the Limitations section for details

What They Mean by "Security Observability"

"Security Observability in general is about providing more context into events involving an incident. We realize security observability by utilizing eBPF, a Linux kernel technology, to allow Security and DevOps teams, SREs, Cloud Engineers, and Solution Architects to gain real-time visibility into Kubernetes and workloads running on top of it. This helps them to secure their production environment"

Default Security Observability Events Logged

refer to the Default Policy (Track Process Starts and Exits) section

Automatically Generating Profiles for Containers

you could do this for things such as binaries executed by using the default profile and then having a script process them and generate policies based on what executables, etc. a given container ran and then applying these policies

Can It Extract HTTP Requests, etc.?

no. It looks like something like Cilium Hubble is better for this. Also refer to this

Can You Block Execution of a Given Syscall Without Killing the Process?

yes. You can do this using the Override - Not Run Function & Return Error to Caller action

" Override action allows to modify the return value of call. While Sigkill will terminate the entire process responsible for making the call, Override will run in place of the original kprobed function and return the value specified in the argError field. It’s then up to the code path or the user space process handling the returned value to whether stop or proceed with the execution"

Can You Use Regular Expressions / Regex?

NOTE this is not supported

Logging

https://tetragon.io/docs/troubleshooting/loglevel/

you have the following levels:

  • trace

  • debug

  • info (this is the default)

  • warning

  • error

  • fatal

  • panic

to modify it:

Option Details

systemd

 
tetra loglevel set debug

and to get it:

tetra loglevel get

you can also modify this file:

/etc/tetragon/tetragon.conf.d/log-level

to view the logs:

journalctl -r -u tetragon.service

Helm

update the Helm value

Using it with Wazuh

Mastering Linux Monitoring with Tetragon and Wazuh | by SOCFortress | Medium

Sending to Crowdstrike Falcon LogScale NG-SIEM

https://github.com/sween/tetragon-crwd-logscale

Resources

Labs

Detecting a Container Escape with Tetragon and eBPF

Getting Started with Tetragon

  • https://isovalent.com/labs/tetragon-getting-started/

  • done

  • NOTE it was decent (about an hour or so)

  • "The best way to have your first experience with Cilium Tetragon is to walk through this lab, which takes the first part of the Real World Attack example out of the book and teaches you how to detect a container escape step by step:

    • running a pod to gain root privileges

    • escaping the pod onto the host

    • persisting the attack with an invisible pod

    • execute a fileless script in memory"

  • "This lab takes the first part of the Real World Attack out of the book and teaches you how to detect a container escape step by step. During the attack you will take advantage of a pod with an overly permissive configuration ("privileged" in Kubernetes) to enter into all the host namespaces with the nsenter command.

    From there, you will write a static pod manifest in the /etc/kubernetes/manifests directory that will cause the kubelet to run that pod. Here you actually take advantage of a Kubernetes bug where you define a Kubernetes namespace that doesn’t exist for your static pod, which makes the pod invisible to the Kubernetes API server. This makes your stealthy pod invisible to kubectl commands.

    After persisting the breakout by spinning up an invisible container, you are going to download and execute a malicious script in memory that never touches disk. Note that this simple python script represents a fileless malware which is almost impossible to detect by using traditional userspace tools"

Books

Books Resource Library - Isovalent

Buzzing Across Space: The Illustrated Children’s Guide to eBPF

Security Observability with eBPF

Courses / Learning Tracks

https://isovalent.com/learning-tracks/ - these cover a number of topics and they have some for Tetragon

To Process

Kubernetes Identity Aware Policies | Tetragon - eBPF-based Security Observability and Runtime Enforcement

Lightning Talk: Don’t Get Blown up! Avoiding Configuration Gotchas for Tetragon Newb…​ Pratik Lotia

Tetragon: Runtime Observability and Security

Detection and Blocking with BPF via YAML

Why Tetragon Should Be Standard in Every Kubernetes Cluster: The Missing Runtime Security Layer

Security Superpowers With eBPF and Tetragon - Liz Rice, Isovalent at Cisco

What are your users really doing within GitHub Actions? | Some Natalie’s corner of the internet

Telemetry to Tactics: Tetragon Through the Lens of the MITRE ATT&CK Framework

Exploring Tetragon - A Security Observability Tool for Kubernetes, Docker, and Linux

Quick Exploration of Tetragon - A Security Observability and Execution Tool Based on eBPF

Ingest, enrich, and transform Tetragon agent logs with Cribl Edge

A lab showing how to inject mTLS certs into requests with Cilium Service Mesh L7 Envoy advance feature

Automated Kubernetes Threat Detection with Tetragon and Azure Sentinel | by Saraswathi Lakshman | Medium

Running PCI-DSS Certified Kubernetes Workloads in the Public Cloud

  • https://www.youtube.com/watch?v=OiO7ySxWiQs

  • "To protect sensitive cardholder data, Stephen Hoekstra and Marcel Bezemer from Schuberg Philis use Tetragon for runtime security visibility to achieve PCI-DSS compliance in AWS"

eBPF: Tetragon Security for IRIS Workloads | InterSystems Developer

Tetragon, EC2 Image Builder and Network Flow Monitor

Processed

Security superpowers with eBPF and Tetragon

Can I Use Tetragon without Cilium? Yes!

eBPF-Powered Kubernetes Security: A Complete Guide to Tetragon

Use Tetragon to Limit Network Usage for a set of Binary - DEV Community

Enhancing Cloud-Native Security with Tetragon

Installation

Standalone

Container

https://tetragon.io/docs/installation/container/

Package

https://tetragon.io/docs/installation/package/

Tetragon will be managed as a systemd service

NOTE this is pretty straight-forward. You just:

  • download a package

  • untar it

  • run the ./install.sh script

and you are good to go. But upgrading requires you basically repeat the above. Would have been preferable if they gave you a repo so you can just update using your normal package manager

Configuration

https://tetragon.io/docs/reference/daemon-configuration/

it looks in the following locations:

--config-dir
/usr/lib/tetragon/tetragon.conf.d/*
/usr/local/lib/tetragon/tetragon.conf.d/*
/etc/tetragon/tetragon.yaml
/etc/tetragon/tetragon.conf.d

for example:

$ sudo ls /etc/tetragon/tetragon.conf.d

and you find the following:

File Value / Details

bpf-lib

 
/usr/local/lib/tetragon/bpf/

export-file-compress

 
true

export-filename

 
/var/log/tetragon/tetragon.log

gops-address

log-format

 
text

log-level

 
info

metrics-server

server-address

this is the socket it listens on

unix:///var/run/tetragon/tetragon.sock

by default, its permissions are:

# ls -l /var/run/tetragon/tetragon.sock
srw-rw----. 1 root root 0 Oct 20 05:10 /var/run/tetragon/tetragon.sock

verbose

 
0

here is an example

Restrict gRPC API Access

https://tetragon.io/docs/reference/daemon-configuration/#restrict-grpc-api-access

Where It Looks for Tracing Policies

it looks for them under:

/etc/tetragon/tetragon.tp.d/

you can also use --tracing-policy-dir for a directory or --tracing-policy for a single policy

View Events

NOTE to view events, you can do one of the following:

Option Details

Access the Socket

 
sudo tetra --server-address unix:///var/run/tetragon/tetragon.sock getevents

NOTE the above outputs it in JSON format. If you want a single line instead, you would add this to it:

-o compact

Tail the Log File

 
sudo tail -f /var/log/tetragon/tetragon.log | jq
Pass in Arguments
systemd
systemctl edit tetragon.service

and then add something like this at the top:

[Service]
ExecStart=
ExecStart=/usr/local/bin/tetragon --enable-process-ancestors

followed by:

systemctl daemon-reload
Example of Using It for a Single Policy

here we do the following:

File / Task Details

Create File

 
echo "hello" > /tmp/liz

example.yml

 
apiVersion: cilium.io/v1alpha1
kind: TracingPolicy
metadata:
  name: "example"
spec:
  kprobes:
  - call: "security_file_permission"
    syscall: false
    args:
    - index: 0
      type: "file"
    - index: 1
      type: "int" # 0x04 is MAY_READ, 0x02 is MAY_WRITE
    selectors:
    - matchArgs:
      - index: 0
        operator: "Equal"
        values:
        - "/tmp/liz"
      matchActions:
      - action: Sigkill

Stop the Tetragon Daemon

 
systemctl stop tetragon.service

Start Tetragon and Point to the Configuration File

 
tetragon --tracing-policy example.yml

Tail the Tetragon Log

in another terminal, run the following:

sudo tail -f /var/log/tetragon/tetragon.log | ./jq

Trigger It

in a third terminal, run the following:

cat /tmp/liz

the above would kill the process and you would have a log entry shown in the General section

Kubernetes

NOTE it runs as a DaemonSet which implements the eBPF logic for:

  • extracting the Security Observability events as well as

  • event filtering

  • event aggregation and

  • event export to external event collectors

installing it installs the following components:

Component Details

Tetragon Operator

this introduces new CRDs that help us create TracingPolicy and TracingPolicyNamespaced resources

Tetragon Engine

this is the DaemonSet that is installed

this also adds the eBPF probes to collect all the kernel events

Services

it installs two services that mainly expose Prometheus metrics:

  • one for Tetragon

  • one for the Tetragon Operator’s

Helm

it is available as this

for example:

helm repo add cilium https://helm.cilium.io
helm repo update
helm install tetragon cilium/tetragon -n kube-system

to monitor its installation:

$ kubectl rollout status -n kube-system ds/tetragon -w

Waiting for daemon set "tetragon" rollout to finish: 0 of 1 updated pods are available...
daemon set "tetragon" successfully rolled out
Helm Chart Values

https://tetragon.io/docs/reference/helm-chart/#values - NOTE there is very little documentation on these here

NOTE to get some documentation, you can run the following:

helm -n kube-system show values cilium/tetragon | vim -

Configuration

Get Configuration

NOTE this will only show you the daemon’s configuration, not the CRDs. For the CRDs:

$ kubectl get tracingpolicy
NAME      AGE
connect   3h30m

NOTE the configuration is saved in a ConfigMap named tetragon-config

to view it:

kubectl edit cm tetragon-config -n kube-system

this is what it looks like by default:

# Please edit the object below. Lines beginning with a '#' will be ignored,
# and an empty file will abort the edit. If an error occurs while saving this file will be
# reopened with the relevant failures.
#
apiVersion: v1
data:
  cluster-name: ""
  debug: "false"
  enable-ancestors: ""
  enable-cgidmap: "false"
  enable-cri: "false"
  enable-k8s-api: "true"
  enable-pod-annotations: "false"
  enable-pod-info: "false"
  enable-policy-filter: "true"
  enable-process-cred: "false"
  enable-process-ns: "false"
  enable-tracing-policy-crd: "true"
  event-cache-retries: "15"
  event-cache-retry-delay: "2"
  export-allowlist: '{"event_set":["PROCESS_EXEC", "PROCESS_EXIT", "PROCESS_KPROBE",
    "PROCESS_UPROBE", "PROCESS_TRACEPOINT", "PROCESS_LSM"]}'
  export-denylist: |-
    {"health_check":true}
    {"namespace":["", "cilium", "kube-system"]}
  export-file-compress: "false"
  export-file-max-backups: "5"
  export-file-max-size-mb: "10"
  export-file-perm: "600"
  export-filename: /var/run/cilium/tetragon/tetragon.log
  export-rate-limit: "-1"
  field-filters: ""
  gops-address: localhost:8118
  health-server-address: :6789
  health-server-interval: "10"
  metrics-label-filter: namespace,workload,pod,binary
  metrics-server: :2112
  process-cache-gc-interval: 30s
  process-cache-size: "65536"
  procfs: /procRoot
  redaction-filters: ""
  server-address: localhost:54321
kind: ConfigMap
metadata:
  annotations:
    meta.helm.sh/release-name: tetragon
    meta.helm.sh/release-namespace: kube-system
  creationTimestamp: "2025-10-23T08:38:25Z"
  labels:
    app.kubernetes.io/component: agent
    app.kubernetes.io/instance: tetragon
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: tetragon
    app.kubernetes.io/part-of: tetragon
    app.kubernetes.io/version: 1.5.0
    helm.sh/chart: tetragon-1.5.0
  name: tetragon-config
  namespace: kube-system
  resourceVersion: "831025"
  uid: ec42b135-d608-4c95-a3b2-d16826bee210

NOTE you will notice that the actual CRDs are not shown here

Update Configuration

you have two options here:

Option Details

Update the Helm Chart

for example, by default, it:

  • doesn’t log capabilities in events

  • doesn’t log namespaces related to network, etc.

so it looks like this:

# -- Enable Capabilities visibility in exec and kprobe events.
enableProcessCred: false
# -- Enable Namespaces visibility in exec and kprobe events.
enableProcessNs: false

to enable these, we would update the Helm chart as follows:

$ helm upgrade -n kube-system --set tetragon.enableProcessCred="true" --set tetragon.enableProcessNs="true" tetragon cilium/tetragon

Release "tetragon" has been upgraded. Happy Helming!
NAME: tetragon
LAST DEPLOYED: Thu Oct 23 20:46:03 2025
NAMESPACE: kube-system
STATUS: deployed
REVISION: 2
TEST SUITE: None

$ kubectl rollout restart daemonset tetragon -n kube-system
$ kubectl rollout status daemonset tetragon -n kube-system

NOTE if you want to test it out, you can use the following manifest

Modify the ConfigMap

NOTE the Helm chart route is better as you get a history of changes

you can edit the ConfigMap and then restart the daemonset:

kubectl edit cm tetragon-config -n kube-system
kubectl rollout restart ds/tetragon -n kube-system

View Logs

Task Details

Install tetra CLI on Host

 
curl -L https://github.com/cilium/tetragon/releases/latest/download/tetra-linux-amd64.tar.gz | tar -xz
sudo mv tetra /usr/local/bin

View the Logs

NOTE if we want to export in compact mode, we would do the following:

$ kubectl logs -n kube-system -l app.kubernetes.io/name=tetragon -c export-stdout -f | tetra getevents -o compact

💥 exit    default/elasticsearch-quickstart-es-default-0 /usr/bin/nc -z -v -w5 127.0.0.1 8080 0
💥 exit    default/elasticsearch-quickstart-es-default-0 /usr/bin/bash /mnt/elastic-internal/scripts/readiness-port-script.sh 0
🚀 process default/elasticsearch-quickstart-es-default-0 /usr/bin/bash /mnt/elastic-internal/scripts/readiness-port-script.sh
🚀 process default/elasticsearch-quickstart-es-default-0 /usr/bin/nc -z -v -w5 127.0.0.1 8080

NOTE the above would export the logs for the pod with the label app.kubernetes.io/name=tetragon from the export-stdout container

NOTE we could also filter based on namespace and pod if we wanted to. For example:

kubectl logs -n kube-system -l app.kubernetes.io/name=tetragon -c export-stdout -f | tetra getevents -o compact --namespaces default --pods elasticsearch-quickstart-es-default-0

NOTE if we want to view it in JSON mode, we would do the following:

kubectl logs -n kube-system -l app.kubernetes.io/name=tetragon -c export-stdout -f | jq

we can use jq to filter in or filter out logs:

kubectl logs -n kube-system -l app.kubernetes.io/name=tetragon -c export-stdout -f | jq 'select(.process_exec.process.pod.name=="xwing" or .process_exit.process.pod.name=="xwing")'

another option is to view the .log file on the hosts themselves. For example:

/var/run/cilium/tetragon/tetragon.log

Managed Kubernetes Clusters

EKS

How to Deploy Tetragon on an EKS Cluster

Misc

Does It Log Events on the Underlying Host By Default?

from my testing, the answer is: no

Misc

Building from Source Code

https://tetragon.io/docs/installation/faq/#can-i-install-and-use-tetragon-in-standalone-mode-outside-of-k8s

Minimum Kernel Version / Features

https://tetragon.io/docs/installation/faq/#can-i-install-and-use-tetragon-in-standalone-mode-outside-of-k8s

Configuration

"Tetragon’s TracingPolicy is a user-configurable Kubernetes custom resource (CR) that allows users to trace arbitrary events in the kernel and optionally define actions to take on a match. Policies consist of:

Item Details

Hook Points

it supports a number of hook points including:

and others

Selectors

this is used for:

  • in-kernel filtering and

  • specifying actions

  • specifying which pods and containers a given policy applies to

Tracing Policy API / CRD Documentation

https://tetragon.io/docs/reference/tracing-policy/

https://github.com/cilium/tetragon/blob/main/pkg/k8s/apis/cilium.io/client/crds/v1alpha1/cilium.io_tracingpolicies.yaml

Tracing Policies (TracingPolicy)

https://tetragon.io/docs/concepts/tracing-policy/

TracingPolicy is a user-configurable Kubernetes custom resource definition (CRD) that allows you to:

  • trace arbitrary events in the kernel and

  • define actions to take on match

" TracingPolicy is fully Kubernetes Identity Aware (which means that you can apply policies to specify pods, etc), so it can enforce on arbitrary kernel events and system calls after the Pod has reached a READY state. This allows you to prevent system calls that are required by the container runtime but should be restricted at application runtime. You can also make changes to the TracingPolicy that dynamically update the eBPF programs in the kernel without needing to restart your application or node.

Once there is an event triggered by a TracingPolicy and the corresponding signature, you can either send an alert to a Security Analyst or prevent the behaviour with a SIGKILL signal to the process"

Hook Points

https://tetragon.io/docs/concepts/tracing-policy/hooks/ - read

Linux Kernel Fundamentals for Effectively Writing Tetragon Tracing Policies

Hook points | Tetragon - eBPF-based Security Observability and Runtime Enforcement

"Tetragon can hook into the:

Layer Details

Kernel

you can do this using:

Userspace

you can use this using:

users can configure these hook points using the corresponding sections of the TracingPolicy specification (.spec).

These hook points include:

  • arguments and

  • return values that can be specified using the:

  • args and

  • returnArg fields"

NOTE the above are used as hooks and then your eBPF program is then run. This is why it is able to kill processes, etc. if you instrument the right system call, etc.

"Tetragon abstracts raw attachment points into policy targets. A hook point in Tetragon is simply a declaration of:

  • where you want to monitor and

  • what arguments you want extracted.

With a kprobe spec for example, you declare, and Tetragon takes care of attaching to the kernel function and extracting the arguments"

- Summary of Hook Points -

you have the following options:

Option Details

kprobes

"Kprobes enables you to:

  • dynamically hook into any kernel function and

  • execute eBPF code"

NOTE keep in mind that kernel functions might change across kernel versions

NOTE it can also be used to trace syscalls

tracepoints

"Tracepoints are statically defined in the kernel and have the advantage of being stable across kernel versions and thus more portable than kprobes"

NOTE it can also be used to trace syscalls

lsm-bpf

instrument LSM hooks

uprobes

NOTE allow you to dynamically hook into any user-space function and execute BPF code

NOTE these are tied to the binary version of the user-space program, so they may not be portable across different versions or architectures

NOTE at the current time, it isn’t really practical for containers due to Limitations but is valid for normal workloads

usdts

useful if your application has these

would typically requires that you either:

  • recompile your application and enable dtrace support

  • check to see if your repository comes with a version that has dtrace support enabled out of the box

user-space
- List of Hook Points -
 — Kernel-space — 
kprobes

https://tetragon.io/docs/concepts/tracing-policy/hooks/#kprobes

"Kprobes enables you to:

  • dynamically hook into any kernel function and

  • execute BPF code"

NOTE keep in mind that kernel functions might change across versions

NOTE these require Arguments. Refer to the section on it for more

Using it for kprobes

an example is fd_install:

spec:
  kprobes:
  - call: "fd_install"
    syscall: false
Using It for Syscalls

NOTE you can also use it for syscalls:

spec:
  kprobes:
  - call: "sys_read"
    syscall: true
    # [...]
  - call: "sys_write"
    syscall: true
    # [...]
tracepoints

https://tetragon.io/docs/concepts/tracing-policy/hooks/#tracepoints

"Tracepoints are statically defined in the kernel and have the advantage of being stable across kernel versions and thus more portable than kprobes"

NOTE these require Arguments. Refer to the section on it for more

Using it for Tracepoints

NOTE these provide access to arguments through the tracepoint format (more on this below)

to do so, you would do the following:

Step Details

List Subsystems

 
$ sudo ls -w60 /sys/kernel/debug/tracing/events

here is an example of the output:

alarmtimer        i2c             qrtr
amd_cpu           i2c_slave       ras
amdgpu            icmp            raw_syscalls
amdgpu_dm         initcall        rcu
amdxdna           intel_iommu     regmap
asoc              interconnect    regulator
avc               iocost          resctrl
binder            iomap           rpm
block             iommu           rseq
bpf_test_run      io_uring        rtc
bpf_trace         ipi             rv
bridge            irq             sched
btrfs             irq_matrix      sched_ext
capability        irq_vectors     scsi
cfg80211          jbd2            sd
cgroup            kmem            signal
clk               ksm             skb
cma               kvm             smbus
compaction        kvmmmu          sock
context_tracking  kyber           sof
cpuhp             libata          spi
csd               lock            sunrpc
damon             mac80211        swiotlb
dev               maple_tree      syscalls
devfreq           mce             task
devlink           mctp            tcp
dma               mdio            thermal
dma_fence         memcg           thp
drm               migrate         thunderbolt
enable            mmap            timer
error_report      mmap_lock       timer_migration
exceptions        mmc             timestamp
ext4              module          tlb
fib               mptcp           tls
fib6              msr             tsm_mr
filelock          napi            ucsi
filemap           neigh           udp
fs_dax            net             v4l2
ftrace            netlink         vb2
fuse              nmi             vmalloc
gpio              notifier        vmscan
gpu_scheduler     nvme            vsyscall
handshake         oom             watchdog
hda               osnoise         wbt
hda_controller    page_isolation  workqueue
hda_intel         pagemap         writeback
header_event      page_pool       x86_fpu
header_page       percpu          xdp
huge_memory       power           xen
hugetlbfs         printk          xhci-hcd
hwmon             pwm
hyperv            qdisc

List Events Under Subsystem

for example:

$ sudo ls -w60 /sys/kernel/debug/tracing/events/net

enable                  net_dev_xmit_timeout
filter                  netif_receive_skb
napi_gro_frags_entry    netif_receive_skb_entry
napi_gro_frags_exit     netif_receive_skb_exit
napi_gro_receive_entry  netif_receive_skb_list_entry
napi_gro_receive_exit   netif_receive_skb_list_exit
net_dev_queue           netif_rx
net_dev_start_xmit      netif_rx_entry
net_dev_xmit            netif_rx_exit

Get Event Details

for example:

$ sudo cat /sys/kernel/debug/tracing/events/net/netif_receive_skb/format

name: netif_receive_skb
ID: 1879
format:
        field:unsigned short common_type;       offset:0;       size:2; signed:0;
        field:unsigned char common_flags;       offset:2;       size:1; signed:0;
        field:unsigned char common_preempt_count;       offset:3;       size:1; signed:0;
        field:int common_pid;   offset:4;       size:4; signed:1;

        field:void * skbaddr;   offset:8;       size:8; signed:0;
        field:unsigned int len; offset:16;      size:4; signed:0;
        field:__data_loc char[] name;   offset:20;      size:4; signed:0;

print fmt: "dev=%s skbaddr=%p len=%u", __get_str(name), REC->skbaddr, REC->len
Using it for Raw Tracepoints

https://tetragon.io/docs/concepts/tracing-policy/hooks/#raw-tracepoints

raw tracepoints are attach directly without the perf layer in the middle, so should be bit faster than standard tracepoints

Using it for Syscalls

NOTE similar to kprobes, tracepoints can also hook into system calls. For example:

spec:
  tracepoints:
  - subsystem: "raw_syscalls"
    event: "sys_enter"
    args:
    - index: 4
      type: "int64"
lsm-bpf - Instrument LSM Hooks

https://tetragon.io/docs/concepts/tracing-policy/hooks/#lsm-bpf

NOTE this is what AppArmor and SELinux hook into

"LSM hooks are exposed at critical positions within the kernel and can control operations through hooks, such as:

  • File system operations

  • Opening, creating, moving, and deleting files

  • Mounting and unmounting file systems

  • Task/process operations

  • Allocating and freeing tasks, changing task user and group identities

  • Socket operations

  • Creating and binding sockets

  • Receiving and sending messages"

to check if it is enabled:

$ cat /boot/config-$(uname -r) | grep BPF_LSM
CONFIG_BPF_LSM=y

for an example, refer to the file_open - Called Before Opening a File section

List of LSM Hooks

https://www.kernel.org/doc/html/v5.2/security/LSM.html

these can be found in include/linux/lsm_hooks.h

here is an example of an online version for kernel 5.18

NOTE you have dozens of LSM hooks

for example:

$ grep "Security hooks for" include/linux/lsm_hooks.h
 * Security hooks for program execution operations.
 * Security hooks for mount using fs_context.
 * Security hooks for filesystem operations.
 * Security hooks for inode operations.
 * Security hooks for kernfs node operations
 * Security hooks for file operations
 * Security hooks for task operations.
 * Security hooks for Netlink messaging.
 * Security hooks for Unix domain networking.
 * Security hooks for socket operations.
 * Security hooks for SCTP
 * Security hooks for Infiniband
 * Security hooks for XFRM operations.
 * Security hooks for individual messages held in System V IPC message queues
 * Security hooks for System V IPC Message Queues
 * Security hooks for System V Shared Memory Segments
 * Security hooks for System V Semaphores
 * Security hooks for Audit
 * Security hooks for the general notification queue:
 * Security hooks for using the eBPF maps and programs functionalities through
 * Security hooks for perf events
 * Security hooks for io_uring

the LSM_HOOK definition format looks like this:

LSM_HOOK(<return_type>, <default_value>, <hook_name>, args...)
- Useful LSM Hooks -

NOTE this section needs some work

bprm_check_security - Called Before a New Program is Executed

this is called before a new program is executed

file_open - Called Before Opening a File

this is called before opening a file

here is an example:

apiVersion: cilium.io/v1alpha1
kind: TracingPolicy
metadata:
  name: "lsm-file-open"
spec:
  lsmhooks:
  - hook: "file_open"
    args:
      - index: 0
        type: "file"
    selectors:
    - matchBinaries:
      - operator: "In"
        values:
        - "/usr/bin/cat"
      matchArgs:
      - index: 0
        operator: "Equal"
        values:
        - "/etc/passwd"
        - "/etc/shadow"
 — User-space — 
uprobes

https://tetragon.io/docs/concepts/tracing-policy/hooks/#uprobes

NOTE allow you to dynamically hook into any user-space function and execute BPF code

N)TE this feature isn’t as mature as other hooks

NOTE these are tied to the binary version of the user-space program, so they may not be portable across different versions or architectures

NOTE these require Arguments. Refer to the section on it for more

NOTE at the current time, it isn’t really practical for containers due to Limitations but is valid for normal workloads

to use it, you would do the following:

Step Details

List Functions

 
$ nm -D /bin/bash | grep readline

0000000000155400 B bash_readline_initialized
0000000000155178 B current_readline_line
0000000000155170 B current_readline_line_index
0000000000154ec0 B current_readline_prompt
0000000000079dc0 T initialize_readline
000000000007a530 T pcomp_set_readline_variables
00000000000713d0 T posix_readline_initialize
00000000000b2a70 T readline
00000000000b2770 T readline_internal_char
00000000000b2100 T readline_internal_setup
00000000000b2330 T readline_internal_teardown
000000000015476c D rl_gnu_readline_p
000000000014e180 D rl_readline_name
00000000001553f8 B rl_readline_state
0000000000154768 D rl_readline_version

Use It

here is an example:

apiVersion: cilium.io/v1alpha1
kind: TracingPolicy
metadata:
  name: "example-uprobe"
spec:
  uprobes:
  - path: "/bin/bash"
    symbols:
    - "readline"
Limitations

some of its limitations include:

Limitation Details

Requires Full Path - Not Useful for Containers

https://github.com/cilium/tetragon/issues/3572

NOTE because you need the full path, it isn’t practical for containers because the full path (from the host’s point of view) changes for each container
usdts - User Statically-Defined Tracing

https://tetragon.io/docs/concepts/tracing-policy/hooks/#usdts

NOTE refer also to the Set - Write Value to Configured USDT Probe Argument section

Requires Application Be Compiled with DTrace Support

How to trace a Python application with eBPF/BCC | by Isham Araia | Medium

- Misc -
Arguments

https://tetragon.io/docs/concepts/tracing-policy/hooks/#arguments

arguments are required for:

this does the following:

  • tells Tetragon what type each argument is (so it prints it out correctly if needed)

  • which arguments to add to the event output (if you don’t specify an argument, then it won’t be printed in the output)

  • required if you want to use the argument in your logic

for example:

apiVersion: cilium.io/v1alpha1
kind: TracingPolicy
metadata:
  name: "example"
spec:
  kprobes:
  - call: "security_file_permission"
    syscall: false
    args:
    - index: 0
      type: "file"
    - index: 1
      type: "int" # 0x04 is MAY_READ, 0x02 is MAY_WRITE
    selectors:
    - matchArgs:
      - index: 0
        operator: "Equal"
        values:
        - "/tmp/liz"
      matchActions:
      - action: Sigkill

in the above, if we commented out the args section, it would fail to run the policy. If we commented out just the index: 0 section, it would fail to run because we use it in the selectors section. However, if we disable index: 1, it works fine but it wouldn’t get included in the output

here is another example taken from here:

tetragon tracing policy arguments 1

Monitoring Multiple Syscalls (Using Lists, etc)

you can do this using either:

Option Details

Lists

 
spec:
  lists:
  - name: "dups"
    type: "syscalls"
    values:
    - "sys_dup"
    - "sys_dup2"
    - "sys_dup3"
  kprobes:
  - call: "list:dups"

NOTE you can have multiple lists. For example:

spec:
  lists:
  - name: "dups"
    type: "syscalls"
    values:
    - "sys_dup"
    - "sys_dup2"
    - "sys_dup3"
    name: "another"
    - "sys_open"
    - "sys_close"

Normal

 
spec:
  kprobes:
  - call: "sys_dup"
    syscall: true
  - call: "sys_dup2"
    syscall: true
  - call: "sys_dup3"
    syscall: true
Lists

NOTE these are useful for providing a list of syscalls, etc. to monitor. Refer to the Monitoring Multiple Syscalls (Using Lists, etc) section for details

Include Return Values in Output

https://tetragon.io/docs/concepts/tracing-policy/hooks/#return-values

you can do this using returnArg

for example:

- call: "sk_alloc"
  syscall: false
  return: true
  args:
  - index: 1
    type: int
    label: "family"
  returnArg:
    index: 0
    type: sock
Labels

you can add these and they would be included in the output

for example:

args:
- index: 0
  type: "int"
  label: "FD"
- index: 1
  type: "file"
Attribute Resolution

https://tetragon.io/docs/concepts/tracing-policy/hooks/#attribute-resolution

allows you to dynamically extract specific attributes from kernel structures passed as parameters to:

"For example, when using the bprm_check_security LSM hook, you can access and display attributes from the parameter struct linux_binprm *bprm at index 0. This parameter contains many informations you may want to access. With the resolve flag, you can easily access fields such as mm.owner.real_parent.comm, which provides the parent process comm "

so running a policy such as this:

apiVersion: cilium.io/v1alpha1
kind: TracingPolicy
metadata:
  name: "lsm"
spec:
  lsmhooks:
  - hook: "bprm_check_security"
    args:
    - index: 0 # struct linux_binprm *bprm
      type: "string"
      resolve: "mm.owner.real_parent.comm"
    selectors:
    - matchActions:
      - action: Post

would give us something like this:

{
  "process_lsm": {
    "process": {
      "exec_id": "bG9jYWxob3N0LmxvY2FsZG9tYWluOjEwMDAzMjQ2MDY4NDAyODo3Njky",
      "pid": 7692,
      "uid": 0,
      "cwd": "/home/vagrant",
      "binary": "/usr/bin/sudo",
      "arguments": "cat /var/log/tetragon/tetragon.log",
      "flags": "execve",
      "start_time": "2025-10-19T05:33:07.558308959Z",
      "auid": 1000,
      "parent_exec_id": "bG9jYWxob3N0LmxvY2FsZG9tYWluOjEwMDAzMjQ1OTM2ODEzMzo3Njkx",
      "refcnt": 1,
      "tid": 7692,
      "in_init_tree": false
    },
    "parent": {
      "exec_id": "bG9jYWxob3N0LmxvY2FsZG9tYWluOjEwMDAzMjQ1OTM2ODEzMzo3Njkx",
      "pid": 7691,
      "uid": 0,
      "cwd": "/home/vagrant",
      "binary": "/usr/bin/sudo",
      "arguments": "cat /var/log/tetragon/tetragon.log",
      "flags": "execve",
      "start_time": "2025-10-19T05:33:07.556994376Z",
      "auid": 1000,
      "parent_exec_id": "bG9jYWxob3N0LmxvY2FsZG9tYWluOjEwMDAzMjQzMDQ2MzE5Mzo3Njg3",
      "refcnt": 1,
      "tid": 7691,
      "in_init_tree": false
    },
    "function_name": "bprm_check_security",
    "policy_name": "lsm",
    "args": [
      {
        "string_arg": "sudo"
      }
    ],
    "action": "KPROBE_ACTION_POST"
  },
  "node_name": "localhost.localdomain",
  "time": "2025-10-19T05:33:07.558568815Z"
}

NOTE notice at the end the args: section which has:

"string_arg": "sudo"
Which Hook Point Should You Use?

NOTE want to improve this section

NOTE in most cases, it is pretty straight-forward. Just refer to the Summary of Hook Points section

Selectors - Filter Events and Take Action

https://tetragon.io/docs/concepts/tracing-policy/selectors/ - read

NOTE these are performed in-kernel so this is very fast (refer to the Performance section for details)

"Selectors enable per-hook in-kernel BPF filtering and actions.

Each selector defines a combination of:

  • a set of filters (such as matchBinaries, etc)

  • a set of (optional) actions to be performed if the selector filters match.

    • NOTE the default action is Post which prints the event to the console"

Note
 each hook can contain up to 5 selectors. Refer to the example below for what a selector looks like
Note

some things to keep in mind:

  • if no selectors are defined on a hook, the default action (Post, i.e., post an event) will be used. This prints the event to the console / log file

  • for a selector to match, all of its filters must match (AND operator)

  • if a selector defines no filters, it matches

  • if multiple selectors are defined in a hook, only the action defined in the first matching selector will be applied (short-circuited OR)

  • if a selector defines no action, the default action (Post) is applied
Important

each selector starts with a -

here is an example:

apiVersion: cilium.io/v1alpha1
kind: TracingPolicy
metadata:
  name: "mount-example"
spec:
  kprobes:
  - call: "sys_mount"
    syscall: true
    selectors:
      # first selector
      - matchBinaries:
        - operator: In
          values:
          - "/usr/bin/mount"
        matchActions:
        - action: NoPost
      # second selector
      - matchActions:
        - action: Post

in the above, we have two selectors:

Selector Details

1st

this is what it looks like:

# first selector
- matchBinaries:
  - operator: In
    values:
    - "/usr/bin/mount"
  matchActions:
  - action: NoPost

so we use the matchBinaries filter to look for the value: /usr/bin/mount. If we get a match, then our action is NoPost (we don’t print anything)

NOTE the operators that you can use depend on the filter

2nd

here is our second selector:

# second selector
- matchActions:
  - action: Post

we don’t have any filters so it will always match the kprobe (sys_mount syscall). Instead, we just have the default action which is Post (refer to the matchActions section for more on matchActions)

NOTE remember that first selector match wins. So in the above, it filters for all sys_mount syscalls. It then looks at the first selector and looks to see if the binary doing this call is /usr/bin/mount. If we have a match, it doesn’t print anything and the second selector is not evaluated. So the second selector would only actually run if the first selector is not a match

- Summary of Filters -

here is a list of them:

Filter Details

matchArgs

filter based on the value of a function’s arguments

matchReturnArgs

filter on the function’s return value (success, error, etc)

NOTE "a use case for this would be to detect the failed access to certain files, like /etc/shadow. Doing cat /etc/shadow will use a openat syscall that will returns -1 for a failed attempt with an unprivileged user"

matchData

filter on the value of data fields

NOTE this is basically data from task_struct related to the process that you can access

matchPIDs

filter on PID

matchBinaries

filter on binary path (NOTE you have to put in the full path). This also means that if someone copies the binary somewhere, it is not applicable anymore (similar to AppArmor)

matchNamespaces

filter on Linux namespaces

matchNamespaceChanges

filter on Linux namespaces changes

matchCapabilities

filter on Linux capabilities

matchCapabilityChanges

filter on Linux capabilities changes

matchActions

apply an action on selector matching

matchReturnActions

apply an action on return selector matching
- List of Filters -
matchArgs - Filter on Value of Function’s Argument

https://tetragon.io/docs/concepts/tracing-policy/selectors/#arguments-filter

this lets you filter on value of a function’s arguments

NOTE when multiple operators are supported under matchArgs, it will logically AND them

for example, for a syscall, the first argument has to equal one of a list of values

you have two options:

Option Details

index

this denotes the argument’s position within the function arguments

args

this denotes the argument’s position within the spec arguments

NOTE if you have both index and args, the args takes precedence

arguments are zero-based

here is a simple partial example:

selectors:
- matchArgs:
  - index: 1
    operator: "Equal"
    values:
    - "/etc/passwd"
    - "/etc/shadow"

NOTE for a full example, refer to the Detect Reads and Writes to /etc/kubernetes/manifests/ Directory section

NOTE the available operators include:

matchReturnArgs - Filter on Function Return Value (Success, Error, etc)

https://tetragon.io/docs/concepts/tracing-policy/selectors/#return-args-filter

NOTE I think you would use matchReturnActions instead of matchActions

this lets you filter on a function’s return value (success, error, etc)

here is an example:

matchReturnArgs:
- operator: "NotEqual"
  values:
  - 0

NOTE "a use case for this would be to detect the failed access to certain files, like /etc/shadow. Doing cat /etc/shadow will use a openat syscall that will returns -1 for a failed attempt with an unprivileged user"

it supports the following operators:

matchData - Filter Based on Value of Specified data (task_struct) Field

https://tetragon.io/docs/concepts/tracing-policy/selectors/#data-filter

https://tetragon.io/docs/concepts/tracing-policy/hooks/#data

NOTE this lets us to retrieve data from the kernel current task object. For a list of the fields that you can access, refer to the task_struct section

NOTE this is useful because you can look at all the fields extracted related to the PID, etc. and use that

for example:

data:
- type: "string"
  source: "current_task"
  resolve: "pid"
selectors:
- matchData:
  - index: 0
    operator: "NotEqual"
    values:
    - "1"

in the above example we extract pid value from current_task and filter on all values except for 1. So we would only print if the process that is currently running is not PID 1

NOTE "all data field spec definitions need to define source (ATM only available value is current_task) and resolve string based on kernel struct task_struct object" - https://tetragon.io/docs/concepts/tracing-policy/hooks/#data

here is an example of what an event looks like. Notice that the PID is not 1

{
  "process_exit": {
    "process": {
      "exec_id": "bG9jYWxob3N0LmxvY2FsZG9tYWluOjgzMTYxMDgyNDM5MjA6MzUxMA==",
      "pid": 3510,
      "uid": 0,
      "binary": "[kthreadd]",
      "flags": "execve",
      "start_time": "2025-10-18T04:04:22.329706699Z",
      "auid": 4294967295,
      "parent_exec_id": "bG9jYWxob3N0LmxvY2FsZG9tYWluOjMwMDAwMDAwMDoy",
      "refcnt": 1,
      "tid": 3510,
      "in_init_tree": false
    },
    "parent": {
      "exec_id": "bG9jYWxob3N0LmxvY2FsZG9tYWluOjMwMDAwMDAwMDoy",
      "pid": 2,
      "uid": 0,
      "binary": "[kthreadd]",
      "flags": "procFS",
      "start_time": "2025-10-18T01:45:46.521461396Z",
      "auid": 4294967295,
      "parent_exec_id": "bG9jYWxob3N0LmxvY2FsZG9tYWluOjE6MA==",
      "tid": 2,
      "in_init_tree": false
    },
    "time": "2025-10-18T04:14:57.209243030Z"
  },
  "node_name": "localhost.localdomain",
  "time": "2025-10-18T04:14:57.209241777Z"
}
task_struct Definition

https://www.cs.fsu.edu/~baker/opsys/examples/task_struct.html

https://elixir.bootlin.com/linux/v6.2-rc1/source/include/linux/sched.h#L737

https://docs.huihoo.com/doxygen/linux/kernel/3.7/structtask__struct.html - this is for an older version of the kernel

Processes | 50.005 CSE

Kernel task_struct | Alex XU

it is defined in linux/sched.h

the below is from AI:

"Each process and thread in the Linux kernel is represented by its own task_struct instance. This structure contains a vast amount of data crucial for managing the task’s execution and its relationship with other tasks.

Key information stored within the task_struct includes:"

Item Details

Identifiers

here you have:

  • Process ID (PID)

  • Task Group ID (TGID)

  • Parent Process ID (PPID)

State Information

here you have current state of the task (e.g., running, sleeping, stopped, zombie)

Scheduling Information

here you have:

  • priority

  • scheduling class, and

  • details relevant to how the scheduler manages the task

Resource Management

here you have pointers to structures managing:

  • open files

  • memory space, and

  • other resources

Relationships

here you have pointers to the:

  • parent process and

  • lists of child and sibling processes

establishing the process hierarchy

CPU-Specific State

here you have information like:

  • registers

  • stack pointers, and

  • other CPU-specific details

Credentials

here you have User and group IDs associated with the task

Signal Handling

here you have Information about:

  • signal handlers and

  • pending signals

this is what it looks like:

kernel task struct 1

When Would You Use This vs matchArgs?

NOTE when you want to use some context related to the task_struct, not arguments, etc. For example, the pid, the auid, the uid, etc.

matchPIDs - Filter Based on PID of Process

https://tetragon.io/docs/concepts/tracing-policy/selectors/#pids-filter

this lets us filter in or filter out specific PIDs

NOTE when multiple operators are supported under matchPIDs, it would logically AND them

it supports the following operators:

Using It with Containers

NOTE it supports this

for example, what if we want to filter based on all PIDs not 1 in a container and not children of PID 1? We can do something like this:

- matchPIDs:
  - operator: NotIn
    followForks: false
    isNamespacePID: true
    values:
    - 1

so the above would detect kubectl exec as it meets the requirements of the above

matchBinaries - Match Binaries

https://tetragon.io/docs/concepts/tracing-policy/selectors/#binaries-filter

this filters based on a binary’s path

this is an example:

- matchBinaries:
  - operator: "In"
    values:
    - "/usr/bin/cat"
    - "/usr/bin/tail"

NOTE it supports the following operators:

here it its definition:

matchBinaries:
  description: A list of binary exec name filters.
  items:
    properties:
      followChildren:
        default: false
        description: In addition to binaries, match children
          processes of specified binaries.
        type: boolean
      operator:
        description: Filter operation.
        enum:
        - In
        - NotIn
        - Prefix
        - NotPrefix
        - Postfix
        - NotPostfix
        type: string
      values:
        description: Value to compare the argument against.
        items:
          type: string
        type: array
    required:
    - operator
    - values
    type: object
  type: array

NOTE so they added Prefix which they didn’t have (as mentioned here)

Requires Realpath

NOTE it requires the real path, not a link. So you would do something like this:

$ ls -l /etc/alternatives/javac
lrwxrwxrwx. 1 root root 38 May  2 10:51 /etc/alternatives/javac -> /usr/lib/jvm/java-21-openjdk/bin/javac

$ realpath /etc/alternatives/javac
/usr/lib/jvm/java-21-openjdk/bin/javac

there is an issue here to have it resolve the for you

and then use that

Follow Children

https://tetragon.io/docs/concepts/tracing-policy/selectors/#follow-children

this is supported:

- matchBinaries:
  - operator: "In"
    values:
    - "/usr/sbin/sshd"
    followChildren: true

NOTE only works on children started after Tetragon has started

NOTE for a more complete example, refer to the Monitor All write Calls from sshd and Its Child Processes section

Bypassed with Executable Copy

NOTE you have to put in the full path. This also means that if someone copies the binary somewhere, it is not applicable anymore (similar to AppArmor)

matchNamespaces - Filter Based on Linux Namespace

https://tetragon.io/docs/concepts/tracing-policy/selectors/#namespaces-filter

NOTE for Kubernetes Namespaces, refer to the TracingPolicyNamespaced section

NOTE this lets you filter based on any namespace including:

  • Uts

  • Ipc

  • Mnt

  • Pid

  • PidForChildren

  • Net

  • Cgroup

  • User

  • Time

  • TimeForChildren

NOTE you would pass in either:

  • host_ns - for host namespaces

  • the namespace ID

NOTE so it isn’t that useful

here is an example:

selectors:
- matchNamespaces:
  - namespace: Pid
    operator: In
    values:
    - "4026531836"
    - "4026531835"
  - namespace: Mnt
    operator: In
    values:
    - "4026531833"
    - "4026531834"

NOTE also look at the Detect Opening of /etc/shadow on Host (Not Containers) example

matchNamespaceChanges - Filter on Namespace Changes

https://tetragon.io/docs/concepts/tracing-policy/selectors/#namespace-changes-filter

NOTE this filter can be useful to track execution of code in a new namespace or even container escapes that change their namespaces

for example:

matchNamespaceChanges:
- operator: In
  values:
  - "Mnt"
matchCapabilities - Filter Based on Linux Capabilities

https://tetragon.io/docs/concepts/tracing-policy/selectors/#capabilities-filter

here is a simple example:

- matchCapabilities:
  - type: Effective
    operator: In
    values:
    - "CAP_CHOWN"
    - "CAP_NET_RAW"

the type can be one of the following:

  • Effective

  • Inheritable

  • Permitted

matchCapabilityChanges - Filter on Linux Capability Changes

https://tetragon.io/docs/concepts/tracing-policy/selectors/#capability-changes-filter

here is an example:

matchCapabilityChanges:
- type: Effective
  operator: In
  isNamespaceCapability: false
  values:
  - "CAP_SETUID"
matchActions - Available Actions

https://tetragon.io/docs/concepts/tracing-policy/selectors/#actions-filter

"The matchActions section can be used to specify the action Tetragon should apply when an event occurs"

this lets you apply an action on selector matching

- Summary of Actions -

it supports the following actions:

Action Details

Post

print event

NOTE this is the default

NoPost

don’t print event but execute any other actions

Signal

this can be used to send a signal to the process

if you want to kill a process, use Sigkill instead

Sigkill

this can be used to kill the process immediately

executed directly in the kernel BPF code

Override

this can be used to override the function return arguments

" Override action allows to modify the return value of call. While Sigkill will terminate the entire process responsible for making the call, Override will run in place of the original kprobed function and return the value specified in the argError field. It’s then up to the code path or the user space process handling the returned value to whether stop or proceed with the execution"

GetUrl

this can be used to send a GET request to a known URL

NOTE actually kind of limited

DnsLookup

this can be used to send a DNS request

TrackSock

NOTE not that useful

UntrackSock

NOTE not that useful

NotifyEnforcer

NOTE not that useful

Set

lets you override values in USDTs
- List -
Post - Transmit Event to the Agent

https://tetragon.io/docs/concepts/tracing-policy/selectors/#post-action

allows an event to be transmitted to the agent, from kernel-space to userspace

by default, all TracingPolicy hooks will create an event with the Post action except in those situations:

NoPost - Suppress the Event Being Generated But Execute Action

https://tetragon.io/docs/concepts/tracing-policy/selectors/#nopost-action

"the NoPost action can be used to suppress the event to be generated, but at the same time, all its defined actions are performed.

It’s useful when you are not interested in the event itself, just in the action being performed"

for example:

selectors:
- matchPIDs:
  matchArgs:
  - index: 1
    operator: "Equal"
    values:
    - "/etc/passwd"
  matchActions:
  - action: Override
    argError: -2
  - action: NoPost

so here it would override the return value of the call but not Post it

Signal - Send Specified Signal to Current Process

https://tetragon.io/docs/concepts/tracing-policy/selectors/#signal-action

this lets you send a specified signal (you pass in the number)

here is an example:

- call: "sys_write"
  syscall: true
  args:
  - index: 0
    type: "fd"
  - index: 1
    type: "char_buf"
    sizeArgIndex: 3
  - index: 2
    type: "size_t"
  selectors:
  - matchPIDs:
    - operator: NotIn
      followForks: true
      isNamespacePID: true
      values:
      - 0
      - 1
    matchArgs:
    - index: 0
      operator: "Prefix"
      values:
      - "/etc/passwd"
    matchActions:
    - action: Signal
      argSig: 9

for example, the above would send the signal 9 (SIGKILL) to a process if:

  • it tries to write to /etc/passwd

  • is not PID 0 or PID 1

NOTE in the above example, it would have been better to just use Sigkill - Kill Process instead

Sigkill - Kill Process

https://tetragon.io/docs/concepts/tracing-policy/selectors/#sigkill-action

this can be used to kill the process immediately

executed directly in the kernel BPF code

terminates synchronously the process that made the call that matches the appropriate selectors from the kernel

tetragon diagram 5

Override - Not Run Function & Return Error to Caller

https://tetragon.io/docs/concepts/tracing-policy/selectors/#override-action

NOTE only available if the kernel has CONFIG_BPF_KPROBE_OVERRIDE enabled

" Override action allows to modify the return value of call. While Sigkill will terminate the entire process responsible for making the call, Override will run in place of the original kprobed function and return the value specified in the argError field. It’s then up to the code path or the user space process handling the returned value to whether stop or proceed with the execution"

NOTE this can be used for:

for example:

selectors:
- matchArgs:
  - index: 0
    operator: "Equal"
    values:
    - "/etc/passwd\0"
  matchActions:
  - action: Override
    argError: -1

so whatever action was going to be down (reading the file, etc.), the actual function is not run and we just return the error -1 instead

GetUrl - Send HTTP Request

https://tetragon.io/docs/concepts/tracing-policy/selectors/#geturl-action

Tutorial: Setting Up a Cybersecurity Honeypot with Tetragon to Trigger Canary Tokens

NOTE this isn’t that powerful. An alternative is to just have something like Fluent Bit monitor the logs and forward the ones you are interested in to the webhook. This would give you more control + you can use POST requests as well

useful for triggering Thinkst canaries or any system that can be triggered via an URL request

for example:

matchActions:
- action: GetUrl
  argUrl: http://ebpf.io

NOTE notice that it is a GET and not a POST

NOTE it looks like you can’t pass dynamic parameters (refer to the Can the GetUrl Action Take Dynamic Parameters? section for more)

DnsLookup - Perform DNS Request

https://tetragon.io/docs/concepts/tracing-policy/selectors/#dnslookup-action

can be used to perform a remote interaction such as triggering Thinkst canaries or any system that can be triggered via an DNS entry request

for example:

matchActions:
- action: DnsLookup
  argFqdn: ebpf.io
TrackSock

https://tetragon.io/docs/concepts/tracing-policy/selectors/#tracksock-action

allows you to create a mapping using a BPF map between sockets and processes

for an example, refer to this

UntrackSock

https://tetragon.io/docs/concepts/tracing-policy/selectors/#untracksock-action

remove a mapping from a BPF map

for an example, refer to this

NotifyEnforcer

"The NotifyEnforcer action notifies the enforcer program to kill or override a syscall.

It’s meant to be used on systems with kernel that lacks multi kprobe feature, that allows to attach many kprobes quickly. To workaround that the enforcer sensor uses the raw syscall tracepoint and attaches simple program to syscalls that we need to kill or override"

Set - Write Value to Configured USDT Probe Argument

https://tetragon.io/docs/concepts/tracing-policy/selectors/#usdt-set-action

NOTE it has some limitations (refer to the documentation)

- Misc -
Can You Have Multiple Actions?

yes. For example:

matchActions:
- action: Override
  argError: -2
- action: NoPost
Can the GetUrl Action Take Dynamic Parameters?

NOTE want to check this out (refer also to the GetUrl section)

NOTE according to this, it looks like you can’t which is why he is trying to get a PostUrl option: "Arming Tetragon with the ability to send data as part of the GetUrl action (or through addition of a new action e.g. PostUrl) would enable this to leverage slack webhooks and others"

Can You POST to Webhooks?

not as of Oct 2025. This is an issue trying to get this feature added

Execute AWS Lambda Function

you can do this using GetUrl - Send HTTP Request but given that you can’t send parameters from the process, etc. it isn’t very useful

Rate Limiting / Throttling

https://tetragon.io/docs/concepts/tracing-policy/selectors/#rate-limiting

"When this parameter is specified for an action, that action will check if the same action has fired, for the same thread, within the time window, with the same inspected arguments (only the first 40 bytes of each inspected argument is used in the matching)

here is an example:

matchActions:
- action: Post
  rateLimit: 5m

by default, it takes seconds but you can use m or h

by default, the rate limiting is applied per thread, meaning that only repeated actions by the same thread will be rate limited. This can be expanded to:

Option Details

All Threads

for all threads for a process by specifying a rateLimitScope with value "process"

All Processes

for all processes by specifying the same with the value "global"
File Hashes

https://tetragon.io/docs/concepts/tracing-policy/selectors/#file-hash-collection-with-ima

Post takes the imaHash parameter, when turned to true (by default to false) it adds file hashes in LSM events calculated by Linux integrity subsystem

NOTE it requires kernel 5.11+ and a few kernel features

NOTE it only works for LSM hooks

here is an example:

apiVersion: cilium.io/v1alpha1
kind: TracingPolicy
spec:
  lsmhooks:
  - hook: "bprm_check_security"
    args:
      - index: 0
        type: "linux_binprm"
    selectors:
    - matchBinaries:
      - operator: "In"
        values:
        - "/usr/bin/zsh"
        - "/usr/bin/bash"
      matchActions:
        - action: Post
          imaHash: true

the output would add something like this:

"ima_hash": "sha256:73abb4280520053564fd4917286909ba3b054598b32c9cdfaf1d733e0202cc96"
What is isNamespacePID?

this should be useful for determining if PIDs are in a container or not. Want to test it out

isNamespacePID:
  default: false
  description: Indicates whether PIDs are namespace PIDs.
  type: boolean
matchReturnActions

https://tetragon.io/docs/concepts/tracing-policy/selectors/#return-actions-filter

NOTE I think this only works with matchReturnArgs - Filter on Function Return Value (Success, Error, etc)

it supports the same actions as matchActions - Available Actions

for example (ignore the {{}} sections as the below was taken from some Tetragon tests and these are filled in at test time using Go’s template package):

# test for the sigkill action
apiVersion: cilium.io/v1alpha1
kind: TracingPolicy
metadata:
  name: "sigkilltest"
spec:
  kprobes:
  - call: "sys_lseek"
    syscall: true
    return: true
    args:
    - index: 2
      type: int
    returnArg:
      index: 0
      type: int
    selectors:
    - matchPIDs:
      - operator: In
        values:
        - {{.MatchedPID}}
        isNamespacePID: {{.NamespacePID}}
      matchArgs:
      - index: 2
        operator: Equal
        values:
        - "5555" # magic value, see also sigkill-tester
      matchReturnArgs:
      - index: 0
        operator: "Equal"
        values:
        - "-9" # -EBADF
      matchReturnActions:
      - action: Sigkill
- List of Operators -

https://tetragon.io/docs/concepts/tracing-policy/selectors/#operator-types

Note
 which ones you can use depends on the selector

NOTE the documentation doesn’t have a lot of examples of many of these. To find them, do the following:

git clone --depth=1 https://github.com/cilium/tetragon
cd tetragon/examples

and then search for your operator. For example, to find Mask, you would do the following:

ag "operator: \"Mask\""

but even then, there are some for which there are no examples

 — General Operators — 
Equal

for example:

selectors:
- matchArgs:
  - index: 0
    operator: "Equal"
    values:
    - "/tmp/liz"
NotEqual
Prefix

this is an example:

selectors:
- matchArgs:
  - index: 1
    operator: "Prefix"
    values:
    - "/etc"
NotPrefix
Postfix

here is an example:

matchArgs:
- index: 0
  operator: "Postfix"
  values:
  - "` + testEcho + `"
NotPostfix
Mask

for example:

- index: 2
  operator: "Mask"
  values:
  - "1" # MAP_SHARED
In

here is an example:

- matchPIDs:
  - operator: "In"
    followForks: true
    values:
    - "pid1"
    - "pid2"
    - "pid3"
NotIn

here is an example:

- matchPIDs:
  - operator: NotIn
    followForks: false
    isNamespacePID: true
    values:
    - 1
Gt - Greater Than

for example:

matchReturnArgs:
- index: 0
  operator: "gt"
  values:
  - "0"
Lt - Less Than

for example:

selectors:
- matchPIDs:
  - operator: In
    followForks: true
    values:
    - pid1
    - pid2
    - pid3
  matchArgs:
  - index: 0
    operator: "Equal"
    values:
    - 1
  - index: 2
    operator: "lt"
    values:
    - 500
InRange - In Interval range

for example:

matchData:
- index: 0
  operator: InRange
  values:
  - "0:187904818"
  - "1879048189:4294967295"
NotInRange - Not in Interval range

for example:

matchData:
- index: 0
  operator: NotInRange
  values:
  - "0:187904818"
  - "187904818:1879048187"
  - "1879048189:4294967295"
 — Network-Related — 
SPort - Source Port

for example:

- index: 1
  operator: "SPort"
  values:
  - "80"
NotSPort - Not Source Port
SPortPriv - Source Port is Privileged (0-1023)
NotSPortPriv - Source Port is Not Privileged (Not 0-1023)
DPort - Destination Port
NotDPort - Not Destination Port
DPortPriv - Destination Port is Privileged (0-1023)
NotDPortPriv - Destination Port is Not Privileged (Not 0-1023)
SAddr - Source Address

this can be:

  • IPv4/6 address or

  • IPv4/6 CIDR (for ex 1.2.3.4/24 or 2a1:56::1/128)

NotSAddr - Not Source Address
DAddr - Destination Address

for example:

- index: 1
  operator: "DAddr"
  values:
  - "127.0.0.1/8"

or:

selectors:
- matchArgs:
  - index: 0
    operator: "DAddr"
    values:
    - "127.0.0.1/8"
    - "192.168.0.0/16"
NotDAddr - Not Destination Address

here is an example:

selectors:
  - matchArgs: # How to filter events
      - index: 0
        operator: 'NotDAddr'
        values:
          - 127.0.0.1
Protocol - IP Protocol (TCP, UDP, etc)

for example:

- index: 1
  operator: "Protocol"
  values:
  - "IPPROTO_UDP"
Family (AF_INET, AF_UNIX, etc)

for example:

- index: 0
  operator: "Family"
  values:
  - "AF_INET"
State (TCP_SYN_RECV, etc.)

for example:

- index: 0
  operator: "State"
  values:
  - "TCP_SYN_RECV"

Examples

Examples Repository

https://github.com/cilium/tetragon/tree/main/examples/tracingpolicy - you have a few dozen examples here

NOTE also make sure to check out the Capabilities / Use Cases section which has some examples

Misc Examples
Exclude Events Based on auid

they recently added this. Refer to this for an example

"Within tracing policies which monitor the setuid, setgid, setreuid, setregid, setresui`d `setresgid, setfsuid syscalls, such as the default monitor-uid-gid-changes, we would like the ability to exclude events where the auid equals a specific number. Such as 0 (root) or 4294967295 (unset)"

the below should be able to do so:

apiVersion: cilium.io/v1alpha1
kind: TracingPolicy
metadata:
  name: "example"
spec:
  kprobes:
  - call: "sys_setuid"
    syscall: true
    args:
    - index: 0
      type: int
    data:
    - type: uint32
      index: 0 ## NB: this is to satisfy a validation bug (we can remove when the bug is fixed)
      source: current_task
      resolve: "loginuid.val"
    selectors:
    - matchData:
      - args: [0]
        operator: "NotEqual"
        values: ["1000"]
Monitor Specific Processes Only
kubectl exec -n kube-system -ti daemonset/tetragon -c tetragon -- \
  tetra getevents -o compact --processes curl,python

and you might get something like this:

🚀 process kind-control-plane /usr/bin/python              🛑 CAP_SYS_ADMIN
🚀 process kind-control-plane /usr/bin/curl https://raw.githubusercontent.com/realpython/python-scripts/master/scripts/18_zipper.py 🛑 CAP_SYS_ADMIN
🔌 connect kind-control-plane /usr/bin/curl tcp 10.244.0.6:35596 -> 185.199.110.133:443 🛑 CAP_SYS_ADMIN
🧹 close   kind-control-plane /usr/bin/curl tcp 10.244.0.6:35596 -> 185.199.110.133:443 🛑 CAP_SYS_ADMIN
💥 exit    kind-control-plane /usr/bin/curl https://raw.githubusercontent.com/realpython/python-scripts/master/scripts/18_zipper.py 0 🛑 CAP_SYS_ADMIN
💥 exit    kind-control-plane /usr/bin/python  0  🛑 CAP_SYS_ADMIN
Detect Calls to setns (Set Namespace)

the below is taken from here

if we applied the following TracingPolicy:

apiVersion: cilium.io/v1alpha1
kind: TracingPolicy
metadata:
  name: "sys-setns"
spec:
  kprobes:
  - call: "sys_setns"
    syscall: true
    args:
    - index: 0
      type: "int"
    - index: 1
      type: "int"

and then did the following:

Task Details

Create Privileged Pod with Shared Namespaces

NOTE it should also contain the nsenter binary as we will then use it

NOTE the reason that it is privileged is that we will need this to use nsenter

the below gives us:

  • a privileged container

  • allows it to share the host’s network namespace + PID namespace

apiVersion: v1
kind: Pod
metadata:
  name: sith-infiltrator
  labels:
    org: empire
spec:
  hostPID: true
  hostNetwork: true
  containers:
  - name: sith-infiltrator
    image: nginx:latest
    ports:
    - containerPort: 80
    securityContext:
      privileged: true

NOTE I think nginx:lastest is actually some local copy they made and included nsenter in it

Exec Into Container

 
kubectl exec -it sith-infiltrator -- /bin/bash

Run nsenter

within the pod, we then run the following:

nsenter -t 1 -a bash

which tells it to enter the namespace of the process with PID 1

and then ran it, we would see the following:

🚀 process default/sith-infiltrator /usr/bin/nsenter -t 1 -a bash 🛑 CAP_SYS_ADMIN
🔧 setns   default/sith-infiltrator /usr/bin/nsenter                🛑 CAP_SYS_ADMIN
🚀 process default/sith-infiltrator /usr/bin/bash          🛑 CAP_SYS_ADMIN

NOTE the 2nd event (setns) is the one that was generated due to this rule

Using It with Kubernetes

Automatic Enrichment When Used in Kubernetes

"All security observability events are automatically enriched with Kubernetes metadata such as:

  • pod names

  • labels

  • namespace information, and

  • container SHAs"

Kubernetes Identity Aware Policies / Apply Policies to Specific Pods

https://tetragon.io/docs/concepts/tracing-policy/k8s-filtering/#container-field-filters

here you have several options (you can use them together if you want to):

Option Details

Using Namespaces (TracingPolicyNamespaced)

we can tell it to apply to only specific namespaces in our cluster

Using podSelector (Match Pod Labels)

we can match on specific pod labels

Container Field Filters

we can match on specific containers
Using Namespaces (TracingPolicyNamespaced)

https://tetragon.io/docs/concepts/tracing-policy/k8s-filtering/#namespace-filtering

we can do this using TracingPolicyNamespaced instead of TracingPolicy

"By providing labels and namespace filters in eBPF, Tetragon not only enables teams to filter on Kubernetes metadata and to apply policies to particular Kubernetes workloads, it does so incredibly efficiently. If a particular policy should only be applied to Kubernetes workloads with a particular label, then this logic is directly encoded in the kernel using eBPF which allows to observe and enforce runtime behavior with incredible performance"

for example:

apiVersion: cilium.io/v1alpha1
kind: TracingPolicyNamespaced
metadata:
  name: "sys-read-follow-prefix"
spec:
  kprobes:
    - call: "sys_read"
      syscall: true
      args:
        - index: 0
          type: "fd"
        - index: 1
          type: "char_buf"
          returnCopy: true
        - index: 2
          type: "size_t"
    - call: "sys_write"
      syscall: true
      args:
        - index: 0
          type: "fd"
        - index: 1
          type: "char_buf"
          sizeArgIndex: 3
        - index: 2
          type: "size_t"

so the only different is the kind (TracingPolicy → TracingPolicyNamespaced)

Policy Names Should Be Unique Across All Policies

keep this in mind (refer to this for more). So a TracingPolicy and TracingPolicyNamespaced policies can’t have the same name

Using podSelector (Match Pod Labels)

https://tetragon.io/docs/concepts/tracing-policy/k8s-filtering/#pod-label-filters

here is an example taken from here:

apiVersion: cilium.io/v1alpha1
kind: TracingPolicy
metadata:
  name: detect-shell-in-app-pods
spec:
  kprobes:
    - call: 'execve'
      syscall: true
      args:
        - index: 0
          type: string
      selectors:
        - matchBinaries:
            - operator: In
              values:
                - /bin/bash
                - /bin/sh
                - /bin/zsh
                - /usr/bin/bash
                - /usr/bin/sh
                - /usr/bin/zsh
          matchActions:
            - action: Post
              rateLimit: 1m
  podSelector:
    matchLabels:
      app: your-application

notice the podSelector section at the end

NOTE the above would cause false positives if this is part of how your application runs (by running some bash script as the entrypoint)

here is another example:

apiVersion: cilium.io/v1alpha1
kind: TracingPolicy
metadata:
  name: "file-write-etc-kubernetes-manifests"
spec:
  podSelector:
    matchLabels:
     org: empire
  options:
  - name: "disable-kprobe-multi"
    value: "1"
  kprobes:
  - call: "security_file_permission"
    syscall: false
    args:
    - index: 0
      type: "file" # (struct file *) used for getting the path
    - index: 1
      type: "int" # 0x04 is MAY_READ, 0x02 is MAY_WRITE
    selectors:
    - matchArgs:
      - index: 0
        operator: "Prefix"
        values:
        - "/etc/kubernetes/manifests"
Container Field Filters

https://tetragon.io/docs/concepts/tracing-policy/k8s-filtering/#container-field-filters

https://tetragon.io/docs/concepts/tracing-policy/k8s-filtering/#container-field-filters-1 "For container field filters, we use the containerSelector field of tracing policies to select the containers that the policy is applied to. At the moment, the only supported fields are name and repo which refers to the container repository"

for example:

spec:
  containerSelector:
    matchExpressions:
      - key: name
        operator: In
        values:
        - main
  kprobes:
  ...

Policies

you have the following:

Option Details

Default Policy

this is the default policy that tracks process starts and terminations

Policy Library

this is a library of structured examples that you can use / tweak as needed. You have to explicitly download and enable these

Default Policy (Track Process Starts and Exits)

NOTE the default policy just tracks:

  • process starts and

  • process exits

this is what the compact output looks like:

🚀 process default/sith-infiltrator /kind/bin/mount-product-files.sh /kind/bin/mount-product-files.sh 🛑 CAP_SYS_ADMIN
🚀 process default/sith-infiltrator /usr/bin/jq -r .bundle 🛑 CAP_SYS_ADMIN
💥 exit    default/sith-infiltrator /usr/bin/jq -r .bundle 0 🛑 CAP_SYS_ADMIN
🚀 process default/sith-infiltrator /usr/bin/cp /kind/product_name /kind/product_uuid /run/containerd/io.containerd.runtime.v2.task/k8s.io/323bd684d621510af170e9d4e42bd2acd4523511a586654935bb51b09a9e822b/rootfs/ 🛑 CAP_SYS_ADMIN
💥 exit    default/sith-infiltrator /usr/bin/cp /kind/product_name /kind/product_uuid /run/containerd/io.containerd.runtime.v2.task/k8s.io/323bd684d621510af170e9d4e42bd2acd4523511a586654935bb51b09a9e822b/rootfs/ 0 🛑 CAP_SYS_ADMIN
🚀 process default/sith-infiltrator /usr/bin/mount -o ro,bind /run/containerd/io.containerd.runtime.v2.task/k8s.io/323bd684d621510af170e9d4e42bd2acd4523511a586654935bb51b09a9e822b/rootfs/product_name /run/containerd/io.containerd.runtime.v2.task/k8s.io/323bd684d621510af170e9d4e42bd2acd4523511a586654935bb51b09a9e822b/rootfs/sys/class/dmi/id/product_name 🛑 CAP_SYS_ADMIN

Policy Library

https://tetragon.io/docs/policy-library/

https://tetragon.io/docs/policy-library/observability/

https://github.com/cilium/tetragon/tree/main/examples/policylibrary

"The Policy Library provides structured examples of Tetragon policies paired with relevant use cases"

NOTE some of these are useful

NOTE for each of these, it:

  • tells you what the rule does

  • how to reproduce an event that matches

  • how to search your logs using jq filters

NOTE they only have about 15 policies out of the box covering:

Policy Details

Binary Execution in /tmp

https://tetragon.io/docs/policy-library/observability/#tmp-execs

monitor execution of a binary in the /tmp directory

sudo Monitoring

https://tetragon.io/docs/policy-library/observability/#sudo

monitor sudo invocations

Privileges Escalation via SUID Binary Execution

https://tetragon.io/docs/policy-library/observability/#privileges-suid

monitor execution of SUID binaries

Privileges Escalation via File Capabilities Execution

https://tetragon.io/docs/policy-library/observability/#privileges-fscaps

monitor execution of binaries with file capabilities

Privileges Escalation via Setuid system calls

https://tetragon.io/docs/policy-library/observability/#privileges-setuid

monitor execution of the setuid() system calls family

Privileges Escalation via Unprivileged User Namespaces

https://tetragon.io/docs/policy-library/observability/#privileges-userns

https://raw.githubusercontent.com/cilium/tetragon/main/examples/policylibrary/privileges/privileges-raise.yaml

monitor creation of User namespaces by unprivileged users

Privileges Change via Capset system call

https://tetragon.io/docs/policy-library/observability/#privileges-capset

monitor the execution of the capset() system call

Fileless Execution

https://tetragon.io/docs/policy-library/observability/#exec-fileless

monitor the execution of binaries that exist exclusively as a computer memory-based artifact

Execution of Deleted Binaries

https://tetragon.io/docs/policy-library/observability/#exec-unlinked

monitor the execution of deleted binaries

eBPF System Activity

https://tetragon.io/docs/policy-library/observability/#ebpf

audit BPF program loads

Kernel Module Audit trail

https://tetragon.io/docs/policy-library/observability/#kernel-module

audit loading of kernel modules

Shared Library Loading

https://tetragon.io/docs/policy-library/observability/#library

monitor loading of libraries

Network Activity of SSH daemon

https://tetragon.io/docs/policy-library/observability/#ssh-network

https://raw.githubusercontent.com/cilium/tetragon/main/examples/policylibrary/sshd.yaml

audit remote connections into a shell server

Outbound Connections

https://tetragon.io/docs/policy-library/observability/#egress-connections

monitor all cluster egress connections

it ignores connections to:

  • localhost

  • the POD’s CIDR

  • the service CIDR

Capabilities / Use Cases

https://tetragon.io/docs/use-cases/

https://github.com/cilium/tetragon/tree/main/examples/tracingpolicy - you have a few dozen examples here

eBPF Security Observability: Top Tetragon Use Cases (Part 1)

eBPF Runtime Security at Scale: Top Tetragon Use Cases (Part 2)

Process Lifecycle

it automatically logs process starts and exists. This can be very useful when looking at the process tree after detecting a possible intrusion. Refer to the Building a Process Tree / Ancestor List section for details

- Hook Points -

Default Events

this is what you typically see (in non-Kubernetes environments). For an example of what you see when using it with Kubernetes, refer to this section

{
  "process_exec": {
    "process": {
      "exec_id": "bGludXg6ODExMDAwMDAwMDo3OTk=",
      "pid": 799,
      "uid": 0,
      "cwd": "/",
      "binary": "/usr/lib/systemd/systemd-logind",
      "flags": "procFS auid rootcwd",
      "start_time": "2025-10-18T01:46:03.207624781Z",
      "auid": 4294967295,
      "parent_exec_id": "bGludXg6MzAwMDAwMDAwOjE=",
      "tid": 799,
      "in_init_tree": false
    },
    "parent": {
      "exec_id": "bGludXg6MzAwMDAwMDAwOjE=",
      "pid": 1,
      "uid": 0,
      "cwd": "/",
      "binary": "/usr/lib/systemd/systemd",
      "arguments": "--switched-root --system --deserialize=47",
      "flags": "procFS auid rootcwd",
      "start_time": "2025-10-18T01:45:55.397626724Z",
      "auid": 4294967295,
      "parent_exec_id": "bGludXg6MTow",
      "tid": 1,
      "in_init_tree": false
    }
  },
  "node_name": "linux",
  "time": "2025-10-18T01:46:03.207625001Z"
}

security_bprm_check - Verify Permission Before Executing a Binary via execve or execveat

https://aquasecurity.github.io/tracee/v0.19/docs/events/builtin/extra/security_bprm_check/

for example:

apiVersion: cilium.io/v1alpha1
kind: TracingPolicy
metadata:
  name: "sample-no-exec-id"
spec:
  kprobes:
  - call: "security_bprm_check"
    syscall: false
    args:
    - index: 0
      type: "linux_binprm"
    returnArg:
      index: 0
      type: "int"
    selectors:
      - matchArgs:
          - index: 0
            operator: "Equal"
            values:
              - "/usr/bin/vim"

and this is an example of what we would see:

{
  "process_kprobe": {
    "process": {
      "exec_id": "bGludXg6NTQzNTMwMTQxODc3NTMxOjM0OTU2",
      "pid": 34956,
      "uid": 0,
      "cwd": "/home/vagrant",
      "binary": "/bin/bash",
      "flags": "execve",
      "start_time": "2025-10-24T08:44:45.239502502Z",
      "auid": 1000,
      "parent_exec_id": "bGludXg6NTQzNTE2MDYxNTMxMzg3OjM0OTI5",
      "refcnt": 1,
      "tid": 34956,
      "in_init_tree": false
    },
    "parent": {
      "exec_id": "bGludXg6NTQzNTE2MDYxNTMxMzg3OjM0OTI5",
      "pid": 34929,
      "uid": 0,
      "cwd": "/home/vagrant",
      "binary": "/bin/bash",
      "flags": "execve clone",
      "start_time": "2025-10-24T08:44:31.159156358Z",
      "auid": 1000,
      "parent_exec_id": "bGludXg6NTQzNTE2MDU5NDgyMzkxOjM0OTI4",
      "tid": 34929,
      "in_init_tree": false
    },
    "function_name": "security_bprm_check",
    "args": [
      {
        "linux_binprm_arg": {
          "path": "/usr/bin/vim",
          "permission": "-rwxr-xr-x"
        }
      }
    ],
    "action": "KPROBE_ACTION_POST",
    "policy_name": "sample-no-exec-id",
    "return_action": "KPROBE_ACTION_POST"
  },
  "node_name": "linux",
  "time": "2025-10-24T08:44:45.240121542Z"
}

Network Observability

you can use it to detect and prevent unexpected traffic (such as containers requesting access to public IP addresses when they are not expected to). For an example of this, refer to the Observe and Terminal All TCP Traffic Not to localhost section

if you purchase the Enterprise version, you get parsers to parse traffic and get additional details

- Hook Points -

sk_alloc - All Socket Objects Allocated Here

https://manpages.debian.org/testing/linux-manual-4.8/sk_alloc.9.en.html

refer to this for an example

tcp_connect - Open Connection to Remote Host

here is example that will kill any TCP connections to other than 127.0.0.1/8

apiVersion: cilium.io/v1alpha1
kind: TracingPolicy
metadata:
  name: "connect-only-local-addrs"
spec:
  kprobes:
  - call: "tcp_connect"
    syscall: false
    args:
    - index: 0
      type: "sock"
    selectors:
    - matchArgs:
      - index: 0
        operator: "NotDAddr"
        values:
        - "127.0.0.0/8"
      matchBinaries:
      - operator: "In"
        values:
        - "/usr/bin/curl"
      matchActions:
      - action: Sigkill

tcp_sendmsg - Send Data Over TCP Connection

Kernel analysis with bpftrace [LWN.net]

this is what it looks like:

int tcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t size);

tcp_recvmsg - Recieve Data Over TCP Connection

tcp_close - TCP Connection Close

refer to the Observe TCP connect() and close() Syscalls section for an example

- Examples -

Observe TCP connect() and close() Syscalls

this is what it would look like:

apiVersion: cilium.io/v1alpha1
kind: TracingPolicy
metadata:
  name: "networking"
spec:
  kprobes:
  - call: "tcp_connect"
    syscall: false
    args:
     - index: 0
       type: "sock"
  - call: "tcp_close"
    syscall: false
    args:
     - index: 0
       type: "sock"

after applying the above manifest, you can then inspect the Security Observability events by running:

kubectl exec -n kube-system -ti daemonset/tetragon -c tetragon -- \
	tetra getevents -o compact

we would see output such as this:

...
🔌 connect kind-control-plane /usr/lib/systemd/systemd tcp 127.0.0.1:46318 -> 127.0.0.1:10259 🛑 CAP_SYS_ADMIN
🧹 close   kind-control-plane /usr/lib/systemd/systemd tcp 127.0.0.1:46318 -> 127.0.0.1:10259 🛑 CAP_SYS_ADMIN
🧹 close   kind-control-plane  tcp 127.0.0.1:0 -> 127.0.0.1:46318
🔌 connect kind-control-plane  tcp 127.0.0.1:40446 -> 127.0.0.1:8080
🧹 close   kind-control-plane  tcp 127.0.0.1:40446 -> 127.0.0.1:8080
🧹 close   kind-control-plane  tcp 127.0.0.1:8080 -> 127.0.0.1:40446
🔌 connect kind-control-plane  tcp 127.0.0.1:40452 -> 127.0.0.1:8080
🧹 close   kind-control-plane  tcp 127.0.0.1:40452 -> 127.0.0.1:8080
🧹 close   kind-control-plane  tcp 127.0.0.1:8080 -> 127.0.0.1:40452
🔌 connect kind-control-plane /usr/lib/systemd/systemd tcp 127.0.0.1:58178 -> 127.0.0.1:2381 🛑 CAP_SYS_ADMIN
🧹 close   kind-control-plane  tcp 127.0.0.1:2381 -> 127.0.0.1:58178
🧹 close   kind-control-plane /usr/lib/systemd/systemd tcp 127.0.0.1:58178 -> 127.0.0.1:2381 🛑 CAP_SYS_ADMIN
...

NOTE if we wanted to limit this to a specific pod, we would add it at the end using --pods. For example:

kubectl exec -n kube-system -ti daemonset/tetragon -c tetragon \
  -- tetra getevents -o compact --pods sith-infiltrator

Observe and Terminal All TCP Traffic Not to localhost

this is taken from here:

apiVersion: cilium.io/v1alpha1
kind: TracingPolicy
metadata:
  name: 'monitor-network'
spec:
  kprobes: # How to inject (kprobes, tracepoints, uprobes)
    - call: 'tcp_connect' # Where to inject (syscall, kernel function, tracepoint)
      syscall: false
      args: # Extra data to include in the event
        - index: 0
          type: 'sock'
      selectors:
        - matchArgs: # How to filter events
            - index: 0
              operator: 'NotDAddr'
              values:
                - 127.0.0.1
          matchActions: # How to react to events (in addition to logging)
            - action: Sigkill

this is an example of the output we would see (if we aren’t running it in Kubernetes):

{
  "process_kprobe": {
    "process": {
      "exec_id": "bG9jYWxob3N0LmxvY2FsZG9tYWluOjI1OTg1NDk2NjkwMjA2NDoxNDQ5MA==",
      "pid": 14490,
      "uid": 1000,
      "cwd": "/home/vagrant",
      "binary": "/usr/bin/curl",
      "arguments": "https://httpbin.org/get",
      "flags": "execve clone",
      "start_time": "2025-10-21T01:56:50.064527155Z",
      "auid": 1000,
      "parent_exec_id": "bG9jYWxob3N0LmxvY2FsZG9tYWluOjUxNDU0MDAwMDAwMDozMTA3",
      "refcnt": 1,
      "tid": 14490,
      "in_init_tree": false
    },
    "parent": {
      "exec_id": "bG9jYWxob3N0LmxvY2FsZG9tYWluOjUxNDU0MDAwMDAwMDozMTA3",
      "pid": 3107,
      "uid": 1000,
      "cwd": "/home/vagrant",
      "binary": "/usr/bin/bash",
      "flags": "procFS auid",
      "start_time": "2025-10-18T01:54:29.637624921Z",
      "auid": 1000,
      "parent_exec_id": "bG9jYWxob3N0LmxvY2FsZG9tYWluOjUxNDQ5MDAwMDAwMDozMTA2",
      "tid": 3107,
      "in_init_tree": false
    },
    "function_name": "tcp_connect",
    "args": [
      {
        "sock_arg": {
          "family": "AF_INET",
          "type": "SOCK_STREAM",
          "protocol": "IPPROTO_TCP",
          "saddr": "192.168.121.141",
          "daddr": "35.171.138.34",
          "sport": 45346,
          "dport": 443,
          "cookie": "18446624719831238976",
          "state": "TCP_SYN_SENT"
        }
      }
    ],
    "action": "KPROBE_ACTION_SIGKILL",
    "policy_name": "monitor-network",
    "return_action": "KPROBE_ACTION_POST"
  },
  "node_name": "localhost.localdomain",
  "time": "2025-10-21T01:56:50.087614070Z"
}

the above was in response to running:

curl https://httpbin.org/get

NOTE notice that we get the following:

  • the socket details including saddr, daddr, etc.

    • 35.171.138.34 is one of the IPs returned by host httpbin.org

  • the name and path of the binary involved (curl)

Kill Connections to DNS Servers Other Than Approved List

https://github.com/cilium/tetragon/blob/main/examples/tracingpolicy/dns-only-specified-servers.yaml

NOTE remember that DoT and DoH can get around this

Log HTTP(S) Destination

if the tool takes it from the CLI, then you just look at the corresponding process log. For example:

curl https://httpbin.org/get

would show us the following in the process section:

"process": {
  "exec_id": "bG9jYWxob3N0LmxvY2FsZG9tYWluOjI1OTg1NDk2NjkwMjA2NDoxNDQ5MA==",
  "pid": 14490,
  "uid": 1000,
  "cwd": "/home/vagrant",
  "binary": "/usr/bin/curl",
  "arguments": "https://httpbin.org/get",
  "flags": "execve clone",
  "start_time": "2025-10-21T01:56:50.064527155Z",
  "auid": 1000,
  "parent_exec_id": "bG9jYWxob3N0LmxvY2FsZG9tYWluOjUxNDU0MDAwMDAwMDozMTA3",
  "refcnt": 1,
  "tid": 14490,
  "in_init_tree": false
},

notice the arguments

NOTE of course, if our binary reads this from a file (like hurl does), you won’t see this in the output:

"process": {
  "exec_id": "bG9jYWxob3N0LmxvY2FsZG9tYWluOjI2NDc4ODM1NjM3ODk3MjoxNDg2NQ==",
  "pid": 14865,
  "uid": 0,
  "cwd": "/home/vagrant/hurl-7.0.0-x86_64-unknown-linux-gnu/bin",
  "binary": "/home/vagrant/hurl-7.0.0-x86_64-unknown-linux-gnu/bin/hurl",
  "arguments": "input.txt",
  "flags": "execve clone",
  "start_time": "2025-10-21T03:19:03.454004373Z",
  "auid": 1000,
  "parent_exec_id": "bG9jYWxob3N0LmxvY2FsZG9tYWluOjI2Mzk1NzgyMDIzOTU5ODoxNDc2NA==",
  "tid": 14865,
  "in_init_tree": false
},

and this is what input.txt looks like:

GET https://httpbin.org/get

- Misc -

Logs Number of Bytes Sent

NOTE you will find this in the output under int_arg. For example, if the compat output looks like this:

📤 sendmsg default/elasticsearch-quickstart-es-default-0 /usr/bin/curl tcp 10.42.0.11:49958 -> 54.157.190.211:443 bytes 38

NOTE the above shows you how many bytes were sent in the sendmsg syscall

we would see the following in the JSON output:

"function_name": "tcp_sendmsg",
"args": [
  {
    "sock_arg": {
      "family": "AF_INET",
      "type": "SOCK_STREAM",
      "protocol": "IPPROTO_TCP",
      "saddr": "10.42.0.11",
      "daddr": "35.171.138.34",
      "sport": 40350,
      "dport": 443,
      "cookie": "18446619037430048896",
      "state": "TCP_ESTABLISHED"
    }
  },
  {
    "int_arg": 38
  }
],
"action": "KPROBE_ACTION_POST",
"policy_name": "connect",
"return_action": "KPROBE_ACTION_POST"

File Integrity Monitoring (FIM) / Filename Access

https://tetragon.io/docs/use-cases/filename-access/

File Monitoring with eBPF and Tetragon (Part 1)

NOTE for why eBPF is a better option than traditional FIMs that use inotify, etc., refer to the Why It is Better Than Other Options section

NOTE for why monitoring syscalls such as open and openat is dangerous and may miss things, refer to the Why Checking the open / openat Syscall Isn’t Recommended section

- Hook Points -

NOTE there are two main kprobes that you would use here:

Option Details

security_file_permission - Intercept All File Reads and Writes

this lets you intercept all of these syscalls:

  • using read and write system calls

  • using copy_file_range and sendfile system calls

  • asynchronous I/O system calls such as aio and io_uring

security_path_truncate - Intercept File Truncations

check permission before truncating the file indicated by path

the truncate syscall is used to truncate a file to a specified length and is one syscall that would be intercepted

security_file_permission - Intercept All File Reads and Writes

https://docs.kernel.org/core-api/kernel-api.html#c.security_file_permission

NOTE this is part of the LSM API is called by the kernel every time a process wants to access a file for reading or writing. It’s used to control whether the operation should be permitted or not

NOTE something to keep in mind is that this is called on every file access in the system. You might not want this. If you want to just know when a file is opened the first time by a process, use security_file_open instead

returns 0 if permission is granted

"Check file permissions before accessing an open file. This hook is called by various operations that read or write files. A security module can use this hook to perform additional checking on these operations, e.g. to revalidate permissions on use to support privilege bracketing or policy changes. Notice that this hook is used when the actual read/write operations are performed, whereas the inode_security_ops hook is called when a file is opened (as well as many other operations)"

NOTE the signature of the function looks like this:

int security_file_permission(struct file *file, int mask)

NOTE for examples, refer to the Kill Processes Trying to Access File section

security_file_open - Executed Whenever a File is Opened by a Process

NOTE unlike security_file_permission, this is only run once every time a process open a file. Any subsequent writes, etc. to the file will not go through this LSM hook. But keep in mind that if the process had already opened the file before you started Tetragon, then you would not see this

security_mmap_file - Monitor / Control Mapping of Files Into Process’s Virtual Memory

"`security_mmap_file` is a Linux kernel security hook that monitors and controls the mapping of files into a process’s virtual memory. It is used by security modules, such as SELinux, to enforce access policies, for example, preventing a file from being mapped with both read and write and execute permissions (W^X). This hook is crucial for runtime security and observability, as it provides a way to audit and control file access at a low level"

security_path_truncate - Intercept File Truncations

https://docs.kernel.org/core-api/kernel-api.html#c.security_path_truncate

"Check permission before truncating the file indicated by path. Note that truncation permissions may also be checked based on already opened files, using the security_file_truncate() hook"

returns 0 if permission is granted

Other security_* Hooks

https://docs.kernel.org/core-api/kernel-api.html

NOTE search for security_ in the above link

fd_install - Open File / Install File Descriptor to FD Table

"The fd_install kernel function is called each time a file descriptor is installed into the file descriptor table of a process, typically referenced within system calls like:

  • open or

  • openat.

Hooking fd_install has its benefits and limitations, which are out of the scope of this guide" - https://tetragon.io/docs/concepts/tracing-policy/hooks/

"The routine fd_install() is found in include/linux/file.h. It just stores the information returned by filp_open() " as mentioned here

this is what its code looks like:

/*
 * Install a file pointer in the fd array.
 *
 * The VFS is full of places where we drop the files lock between
 * setting the open_fds bitmap and installing the file in the file
 * array.  At any such point, we are vulnerable to a dup2() race
 * installing a file in the array before us.  We need to detect this and
 * fput() the struct file we are about to overwrite in this case.
 *
 * It should never happen - if we allow dup2() do it, _really_ bad things
 * will follow.
 *
 * This consumes the "file" refcount, so callers should treat it
 * as if they had called fput(file).
 */

void fd_install(unsigned int fd, struct file *file)
{
	struct files_struct *files = current->files;
	struct fdtable *fdt;

	rcu_read_lock_sched();

	if (unlikely(files->resize_in_progress)) {
		rcu_read_unlock_sched();
		spin_lock(&files->file_lock);
		fdt = files_fdtable(files);
		BUG_ON(fdt->fd[fd] != NULL);
		rcu_assign_pointer(fdt->fd[fd], file);
		spin_unlock(&files->file_lock);
		return;
	}
	/* coupled with smp_wmb() in expand_fdtable() */
	smp_rmb();
	fdt = rcu_dereference_sched(files->fdt);
	BUG_ON(fdt->fd[fd] != NULL);
	rcu_assign_pointer(fdt->fd[fd], file);
	rcu_read_unlock_sched();
}

EXPORT_SYMBOL(fd_install);

- Misc -

What About Symbolic Links?

this won’t bypass our policies

Inode-based FIM

NOTE this allows us to monitor inodes instead of path-based filenames (it does the translation on your behalf). This is meant to handle scenarios where someone uses a hard link or bind mount or chroot to access the files using a different path (refer to the How to Bypass Path-Based Filtering section for details)

NOTE this feature is part of Isovalent Enterprise for Tetragon and not in the OSS version. Refer to this for details

How to Bypass Path-Based Filtering

to bypass path-based filtering as shown in the examples, we have several options:

Option Details

Hard Links

when fs.protected_hardlinks is set to 1, then it requires that the user has at least read + write permissions on the file

this requires

Bind Mounts

bind mount requires CAP_SYS_ADMIN

chroot

chroot requires CAP_CHROOT

Why It is Better Than Other Options

you had two traditional options before Tetragon / eBPF solutions:

Option Details

Periodic Scan

here you would just periodically test to see if a file’s hash was changed or not

an attacker can modify a file and then update it again between these scans

you also wouldn’t know who made the modifications and when

you might also end up with multiple modifications and you would only see the one that was last made

inotify

this has limitations including:

  • if a user creates a sub-directory, it is not automatically monitored. Even if you had something in user-space that added this sub-directory, an attacker could have written a file to this sub-directory before the sub-directory was added to the watch list

  • not container-aware

  • can’t tell you are only interested in specific PIDs accessing / modifying a file

  • you don’t get the execution context (which process, which user, which container, etc.)

Why Checking the open / openat Syscall Isn’t Recommended

as mentioned here, you can run into TOCTOU "because the memory location with the pathname to be accessed belongs to user-space, and user-space can change it after the hook runs, but before the pathname is used to perform the actual open in-kernel operation"

tetragon open openat syscall toctou 1

instead, using an LSM hook such as security_file_permission which allows us to see the final path

- Examples -

Kill Processes Trying to Access File

here is a simple example:

apiVersion: cilium.io/v1alpha1
kind: TracingPolicy
metadata:
  name: "example"
spec:
  kprobes:
  - call: "security_file_permission"
    syscall: false
    args:
    - index: 0
      type: "file"
    - index: 1
      type: "int" # 0x04 is MAY_READ, 0x02 is MAY_WRITE
    selectors:
    - matchArgs:
      - index: 0
        operator: "Equal"
        values:
        - "/tmp/liz"
      matchActions:
      - action: Sigkill

in the above, we have the following:

Section Details

kprobes

NOTE so we tell it we will use a kprobe hook type

call

we then specify the actual hook point. Here we define it as security_file_permission:

- call: "security_file_permission"
  syscall: false

notice that syscall is set to false because we are interested in the LSM hook and not a raw syscall

args

here we have:

args:
- index: 0
  type: "file"
- index: 1
  type: "int" # 0x04 is MAY_READ, 0x02 is MAY_WRITE

so we define two arguments:

  • the first argument is the file

  • the second argument is an int

NOTE notice the args section matches the signature of the function:

int security_file_permission(struct file *file, int mask)

selectors.matchArgs

here we can specify what we are interested in. NOTE the index field tells it which of the arguments that we specified in the args section above

- matchArgs:
  - index: 0
    operator: "Equal"
    values:
    - "/tmp/liz"

so here we tell it to match when the file name is equal to /tmp/liz

NOTE if we wanted to specify that we are interested in only reads or only writes, we would look at index: 1 as well. Refer to the Log Reads to Specific Files / Directories and Writes to Specific Files / Directories section for an example of this

selectors.matchActions

finally, we have what action to do. In this case, we specified Sigkill which tells it to kill the process immediately

  matchActions:
  - action: Sigkill

Log Reads to Specific Files / Directories and Writes to Specific Files / Directories

NOTE the below is taken from here:

apiVersion: cilium.io/v1alpha1
kind: TracingPolicy
metadata:
  name: "file-monitoring-filtered"
spec:
  kprobes:
  - call: "security_file_permission"
    syscall: false
    return: true
    args:
    - index: 0
      type: "file" # (struct file *) used for getting the path
    - index: 1
      type: "int" # 0x04 is MAY_READ, 0x02 is MAY_WRITE
    returnArg:
      index: 0
      type: "int"
    returnArgAction: "Post"
    selectors:
    - matchArgs:
      - index: 0
        operator: "Prefix"
        values:
        - "/boot"           # Reads to sensitive directories
        - "/root/.ssh"      # Reads to sensitive files we want to know about
        - "/etc/shadow"
        - "/etc/profile"
        - "/etc/sudoers"
        - "/etc/pam.conf"   # Reads global shell configs bash/csh supported
        - "/etc/bashrc"
        - "/etc/csh.cshrc"
        - "/etc/csh.login"  # Add additional sensitive files here
      - index: 1
        operator: "Equal"
        values:
        - "4" # MAY_READ
    - matchArgs:
      - index: 0
        operator: "Prefix"
        values:
        - "/etc"              # Writes to sensitive directories
        - "/boot"
        - "/lib"
        - "/lib64"
        - "/bin"
        - "/usr/lib"
        - "/usr/local/lib"
        - "/usr/local/sbin"
        - "/usr/local/bin"
        - "/usr/bin"
        - "/usr/sbin"
        - "/var/log"          # Writes to logs
        - "/dev/log"
        - "/root/.ssh"        # Writes to sensitive files add here.
      - index: 1
        operator: "Equal"
        values:
        - "2" # MAY_WRITE

NOTE notice in the above that we split the reads by using a different index: 1 section for it and then another for the writes

Log Access to Any / All Files

NOTE the below will not intercept the truncate syscall and others that are covered using security_path_truncate - Intercept File Truncations

NOTE to track all file access (not recommended that you do this):

apiVersion: cilium.io/v1alpha1
kind: TracingPolicy
metadata:
  name: "file-all"
spec:
  kprobes:
  - call: "security_file_permission"
    syscall: false

NOTE the above has the following limitations:

  • it does not tell us which files were accessed

  • it could hammer our servers due to how many files are typically open / being written to

to have it print out the filenames, we would add the following argument (refer to the Arguments section for details)

	args:
	- index: 0
  	   type: "file" # (struct file *) used for getting the path

Detect Reads and Writes to /etc/kubernetes/manifests/ Directory

this is taken from here:

NOTE this is important as users that are able to create manifests here get containers to run that are invisible to the API Server (because they are static pods):

apiVersion: cilium.io/v1alpha1
kind: TracingPolicy
metadata:
  name: "file-write-etc-kubernetes-manifests"
spec:
  podSelector:
    matchLabels:
     org: empire
  options:
  - name: "disable-kprobe-multi"
    value: "1"
  kprobes:
  - call: "security_file_permission"
    syscall: false
    args:
    - index: 0
      type: "file" # (struct file *) used for getting the path
    - index: 1
      type: "int" # 0x04 is MAY_READ, 0x02 is MAY_WRITE
    selectors:
    - matchArgs:
      - index: 0
        operator: "Prefix"
        values:
        - "/etc/kubernetes/manifests"

"To be able to detect creating an invisible Pod, we will need to apply this TracingPolicy. This TracingPolicy is going to be used to monitor read and write access to sensitive files"

if we then applied it and then either:

  • went to the host

  • did some privilege escalation to the host

and then went to /etc/kubernetes/manifests/ and then ran the below:

cat << EOF > hack-latest.yaml
apiVersion: v1
kind: Pod
metadata:
  name: hack-latest
  hostNetwork: true
  # define in namespace that doesn't exist so
  # workload is invisible to the API server
  namespace: doesnt-exist
spec:
  containers:
  - name: hack-latest
    image: sublimino/hack:latest
    command: ["/bin/sh"]
    args: ["-c", "while true; do sleep 10;done"]
    securityContext:
      privileged: true
EOF

we would see the following Security Observability events:

📚 read    default/sith-infiltrator /usr/bin/ls /etc/kubernetes/manifests 🛑 CAP_SYS_ADMIN
📚 read    default/sith-infiltrator /usr/bin/ls /etc/kubernetes/manifests 🛑 CAP_SYS_ADMIN
📚 read    default/sith-infiltrator /usr/bin/ls /etc/kubernetes/manifests 🛑 CAP_SYS_ADMIN
📚 read    default/sith-infiltrator /usr/bin/ls /etc/kubernetes/manifests 🛑 CAP_SYS_ADMIN
📚 read    default/sith-infiltrator /usr/bin/ls /etc/kubernetes/manifests 🛑 CAP_SYS_ADMIN
📚 read    default/sith-infiltrator /usr/bin/ls /etc/kubernetes/manifests 🛑 CAP_SYS_ADMIN
💥 exit    default/sith-infiltrator /usr/bin/ls -la 0 🛑 CAP_SYS_ADMIN
🚀 process default/sith-infiltrator /usr/bin/cat           🛑 CAP_SYS_ADMIN
📝 write   default/sith-infiltrator /usr/bin/cat /etc/kubernetes/manifests/hack-latest.yaml 🛑 CAP_SYS_ADMIN
📝 write   default/sith-infiltrator /usr/bin/cat /etc/kubernetes/manifests/hack-latest.yaml 🛑 CAP_SYS_ADMIN
💥 exit    default/sith-infiltrator /usr/bin/cat  0 🛑 CAP_SYS_ADMIN

NOTE if we update the policy and add this:

      matchActions:
      - action: Override
        argError: -1

the call would then fail if someone tried to write a file to this directory

root@server:~# kubectl exec -it sith-infiltrator -- /bin/bash
root@kind-control-plane:/# nsenter -t 1 -a bash
root@kind-control-plane:/# cd /etc/kubernetes/manifests/
ls -la
ls: reading directory '.': Operation not permitted
total 0
root@kind-control-plane:/etc/kubernetes/manifests#
root@server:~# kubectl exec -n kube-system -ti daemonset/tetragon -c tetragon -- \
  tetra getevents -o compact --pods sith-infiltrator
🚀 process default/sith-infiltrator /bin/bash              🛑 CAP_SYS_ADMIN
🚀 process default/sith-infiltrator /usr/bin/nsenter -t 1 -a bash 🛑 CAP_SYS_ADMIN
🔧 setns   default/sith-infiltrator /usr/bin/nsenter                🛑 CAP_SYS_ADMIN
🚀 process default/sith-infiltrator /usr/bin/bash          🛑 CAP_SYS_ADMIN
🚀 process default/sith-infiltrator /usr/bin/ls -la        🛑 CAP_SYS_ADMIN
📚 read    default/sith-infiltrator /usr/bin/ls /etc/kubernetes/manifests 🛑 CAP_SYS_ADMIN
💥 exit    default/sith-infiltrator /usr/bin/ls -la 2 🛑 CAP_SYS_ADMIN

NOTE notice that the return status code is 2

Detect Opening of /etc/shadow on Host (Not Containers)

NOTE this is taken from this section

apiVersion: cilium.io/v1alpha1
kind: TracingPolicy
metadata:
  name: "example_ns_1"
spec:
  kprobes:
    - call: "fd_install"
      syscall: false
      args:
        - index: 0
          type: int
        - index: 1
          type: "file"
      selectors:
        - matchBinaries:
          - operator: "In"
            values:
            - "/bin/cat"
          matchArgs:
          - index: 1
            operator: "Equal"
            values:
            - "/etc/shadow"
          matchNamespaces:
          - namespace: Mnt
            operator: In
            values:
            - "host_ns"
        - matchBinaries:
          - operator: "In"
            values:
            - "/bin/cat"
          matchArgs:
          - index: 1
            operator: "Equal"
            values:
            - "/etc/shadow"
          matchNamespaces:
          - namespace: Net
            operator: In
            values:
            - "host_ns"

so we:

  • look for file opens using fd_install

    • NOTE this is called when an entry is added to the file descriptors table

  • only look at it if the binary calling it is /bin/cat

  • only look at it if the file being opened is /etc/shadow

  • we then further filter on it by looking at this when it is on the host by looking at the namespaces

Return Error Code If Specific Files Opened Using cat Binary

taken from here:

apiVersion: cilium.io/v1alpha1
kind: TracingPolicy
metadata:
  name: "file-rsa-keys-open"
spec:
  kprobes:
  - call: "security_file_open"
    syscall: false
    args:
    - index: 0
      type: "file" # (struct file *) used for getting the path
    selectors:
    - matchArgs:
      - index: 0
        operator: "Equal"
        values:
        - "/etc/ssh/ssh_host_rsa_key"
        - "/etc/ssh/ssh_host_ed25519_key"
        - "/etc/ssh/ssh_host_ecdsa_key"
      matchBinaries:
      - operator: "In"
        values:
        - "/usr/bin/cat"
      matchActions:
      - action: Override
        argError: -1

NOTE the above uses security_file_open, so it:

  • only logs once for each of these files

  • would not detect a process that already had one of these files open before you ran Tetragon

Monitor All write Calls from sshd and Its Child Processes

NOTE this is taken from here

this is what it would look like:

- call: "sys_write"
  syscall: true
  args:
  - index: 0
    type: "int"
  - index: 1
    type: "char_buf"
    sizeArgIndex: 3
  - index: 2
    type: "size_t"
  selectors:
  # match to /sbin/sshd
  - matchBinaries:
    - operator: "In"
      values:
      - "/usr/sbin/sshd"
    followChildren: true
  # match to stdin/stdout/stderr
    matchArgs:
    - index: 0
      operator: "Equal"
      values:
      - "1"
      - "2"
      - "3"

so you:

  • filter based on syscall (sys_write)

  • use a selector with the matchBinaries filter and specify the path to /usr/sbin/sshd and its child processes

  • match stdin, stdout and stderr

Misc

Linux Process Credentials

this is when a process tries to:

  • increase its capabilities

  • new credentials are given to a task

refer to these for examples

Output

Formats

Compact (-o compact)

this is what it looks like. So you don’t get a lot of details:

📚 read    default/sith-infiltrator /usr/bin/ls /etc/kubernetes/manifests 🛑 CAP_SYS_ADMIN
📚 read    default/sith-infiltrator /usr/bin/ls /etc/kubernetes/manifests 🛑 CAP_SYS_ADMIN
📚 read    default/sith-infiltrator /usr/bin/ls /etc/kubernetes/manifests 🛑 CAP_SYS_ADMIN
📚 read    default/sith-infiltrator /usr/bin/ls /etc/kubernetes/manifests 🛑 CAP_SYS_ADMIN
📚 read    default/sith-infiltrator /usr/bin/ls /etc/kubernetes/manifests 🛑 CAP_SYS_ADMIN
📚 read    default/sith-infiltrator /usr/bin/ls /etc/kubernetes/manifests 🛑 CAP_SYS_ADMIN
💥 exit    default/sith-infiltrator /usr/bin/ls -la 0 🛑 CAP_SYS_ADMIN
🚀 process default/sith-infiltrator /usr/bin/cat           🛑 CAP_SYS_ADMIN
📝 write   default/sith-infiltrator /usr/bin/cat /etc/kubernetes/manifests/hack-latest.yaml 🛑 CAP_SYS_ADMIN
📝 write   default/sith-infiltrator /usr/bin/cat /etc/kubernetes/manifests/hack-latest.yaml 🛑 CAP_SYS_ADMIN
💥 exit    default/sith-infiltrator /usr/bin/cat  0 🛑 CAP_SYS_ADMIN

JSON

For Non-Containers

here is an example:

{
  "process_kprobe": {
    "process": {
      "exec_id": "bG9jYWxob3N0LmxvY2FsZG9tYWluOjg3Mjk5OTQ0MzIyNTU4OjcwMjY=",
      "pid": 7026,
      "uid": 1000,
      "cwd": "/home/vagrant",
      "binary": "/usr/bin/cat",
      "arguments": "/tmp/liz",
      "flags": "execve clone",
      "start_time": "2025-10-19T02:00:55.041947879Z",
      "auid": 1000,
      "parent_exec_id": "bG9jYWxob3N0LmxvY2FsZG9tYWluOjUxNDU0MDAwMDAwMDozMTA3",
      "refcnt": 1,
      "tid": 7026,
      "in_init_tree": false
    },
    "parent": {
      "exec_id": "bG9jYWxob3N0LmxvY2FsZG9tYWluOjUxNDU0MDAwMDAwMDozMTA3",
      "pid": 3107,
      "uid": 1000,
      "cwd": "/home/vagrant",
      "binary": "/usr/bin/bash",
      "flags": "procFS auid",
      "start_time": "2025-10-18T01:54:29.637624811Z",
      "auid": 1000,
      "parent_exec_id": "bG9jYWxob3N0LmxvY2FsZG9tYWluOjUxNDQ5MDAwMDAwMDozMTA2",
      "tid": 3107,
      "in_init_tree": false
    },
    "function_name": "security_file_permission",
    "args": [
      {
        "file_arg": {
          "path": "/tmp/liz",
          "permission": "-rw-r--r--"
        }
      },
      {
        "int_arg": 4
      }
    ],
    "action": "KPROBE_ACTION_SIGKILL",
    "policy_name": "example",
    "return_action": "KPROBE_ACTION_POST"
  },
  "node_name": "localhost.localdomain",
  "time": "2025-10-19T02:00:55.043721804Z"
}
For Containers

NOTE for containers, it gives you additional details such as:

  • pod name

  • pod namespace

  • container image

  • pod labels

  • etc

{
  "process_exec": {
    "process": {
      "exec_id": "Z2tlLWpvaG4tNjMyLWRlZmF1bHQtcG9vbC03MDQxY2FjMC05czk1OjEzNTQ4Njc0MzIxMzczOjUyNjk5",
      "pid": 52699,
      "uid": 0,
      "cwd": "/",
      "binary": "/usr/bin/curl",
      "arguments": "https://ebpf.io/applications/#...",
      "flags": "execve rootcwd",
      "start_time": "2023-10-06T22:03:57.700327580Z",
      "auid": 4294967295,
      "pod": {
        "namespace": "default",
        "name": "xwing",
        "container": {
          "id": "containerd://551e161c47d8ff0eb665438a7bcd5b4e3ef5a297282b40a92b7c77d6bd168eb3",
          "name": "spaceship",
          "image": {
            "id": "docker.io/tgraf/netperf@sha256:8e86f744bfea165fd4ce68caa05abc96500f40130b857773186401926af7e9e6",
            "name": "docker.io/tgraf/netperf:latest"
          },
          "start_time": "2023-10-06T21:52:41Z",
          "pid": 49
        },
        "pod_labels": {
          "app.kubernetes.io/name": "xwing",
          "class": "xwing",
          "org": "alliance"
        },
        "workload": "xwing"
      },
      "docker": "551e161c47d8ff0eb665438a7bcd5b4",
      "parent_exec_id": "Z2tlLWpvaG4tNjMyLWRlZmF1bHQtcG9vbC03MDQxY2FjMC05czk1OjEzNTQ4NjcwODgzMjk5OjUyNjk5",
      "tid": 52699
    },
    "parent": {
      "exec_id": "Z2tlLWpvaG4tNjMyLWRlZmF1bHQtcG9vbC03MDQxY2FjMC05czk1OjEzNTQ4NjcwODgzMjk5OjUyNjk5",
      "pid": 52699,
      "uid": 0,
      "cwd": "/",
      "binary": "/bin/bash",
      "arguments": "-c \"curl https://ebpf.io/applications/#...\"",
      "flags": "execve rootcwd clone",
      "start_time": "2023-10-06T22:03:57.696889812Z",
      "auid": 4294967295,
      "pod": {
        "namespace": "default",
        "name": "xwing",
        "container": {
          "id": "containerd://551e161c47d8ff0eb665438a7bcd5b4e3ef5a297282b40a92b7c77d6bd168eb3",
          "name": "spaceship",
          "image": {
            "id": "docker.io/tgraf/netperf@sha256:8e86f744bfea165fd4ce68caa05abc96500f40130b857773186401926af7e9e6",
            "name": "docker.io/tgraf/netperf:latest"
          },
          "start_time": "2023-10-06T21:52:41Z",
          "pid": 49
        },
        "pod_labels": {
          "app.kubernetes.io/name": "xwing",
          "class": "xwing",
          "org": "alliance"
        },
        "workload": "xwing"
      },
      "docker": "551e161c47d8ff0eb665438a7bcd5b4",
      "parent_exec_id": "Z2tlLWpvaG4tNjMyLWRlZmF1bHQtcG9vbC03MDQxY2FjMC05czk1OjEzNTQ4NjQ1MjQ1ODM5OjUyNjg5",
      "tid": 52699
    }
  },
  "node_name": "gke-john-632-default-pool-7041cac0-9s95",
  "time": "2023-10-06T22:03:57.700326678Z"
}

NOTE for a single process, if you combine the process start + exit, you get about 10K! as shown in the below example:

{
  "process_exit": {
    "process": {
      "exec_id": "azNkLWVsYXN0aWMtc2VydmVyLTA6MTU1MjE3ODU1MjQxNDI5Nzo5NTU3MzA=",
      "pid": 955730,
      "uid": 1000,
      "cwd": "/usr/share/elasticsearch",
      "binary": "/usr/bin/nc",
      "arguments": "-z -v -w5 127.0.0.1 8080",
      "flags": "execve clone",
      "start_time": "2025-10-23T08:42:43.200470544Z",
      "auid": 4294967295,
      "pod": {
        "namespace": "default",
        "name": "elasticsearch-quickstart-es-default-0",
        "container": {
          "id": "containerd://b0feb994b531d61af43add66ae0be1fe29271c67c3def8eff8dacb2ca172380e",
          "name": "elasticsearch",
          "image": {
            "id": "docker.elastic.co/elasticsearch/elasticsearch@sha256:e0dd6421674d6b28224ca38cb9a19e79c1f365361e3e5b295cb87eca7ecfdad9",
            "name": "docker.elastic.co/elasticsearch/elasticsearch:8.18.1"
          },
          "start_time": "2025-10-17T03:32:45Z",
          "pid": 798764,
          "security_context": {}
        },
        "pod_labels": {
          "apps.kubernetes.io/pod-index": "0",
          "common.k8s.elastic.co/type": "elasticsearch",
          "controller-revision-hash": "elasticsearch-quickstart-es-default-6756799d95",
          "elasticsearch.k8s.elastic.co/cluster-name": "elasticsearch-quickstart",
          "elasticsearch.k8s.elastic.co/http-scheme": "https",
          "elasticsearch.k8s.elastic.co/node-data": "true",
          "elasticsearch.k8s.elastic.co/node-data_cold": "true",
          "elasticsearch.k8s.elastic.co/node-data_content": "true",
          "elasticsearch.k8s.elastic.co/node-data_frozen": "true",
          "elasticsearch.k8s.elastic.co/node-data_hot": "true",
          "elasticsearch.k8s.elastic.co/node-data_warm": "true",
          "elasticsearch.k8s.elastic.co/node-ingest": "true",
          "elasticsearch.k8s.elastic.co/node-master": "true",
          "elasticsearch.k8s.elastic.co/node-ml": "true",
          "elasticsearch.k8s.elastic.co/node-remote_cluster_client": "true",
          "elasticsearch.k8s.elastic.co/node-transform": "true",
          "elasticsearch.k8s.elastic.co/node-voting_only": "false",
          "elasticsearch.k8s.elastic.co/statefulset-name": "elasticsearch-quickstart-es-default",
          "elasticsearch.k8s.elastic.co/version": "8.18.1",
          "statefulset.kubernetes.io/pod-name": "elasticsearch-quickstart-es-default-0"
        },
        "workload": "elasticsearch-quickstart-es-default",
        "workload_kind": "StatefulSet"
      },
      "docker": "b0feb994b531d61af43add66ae0be1f",
      "parent_exec_id": "azNkLWVsYXN0aWMtc2VydmVyLTA6MTU1MjE3ODU1MTA1NTAxOTo5NTU3MjM=",
      "tid": 955730,
      "in_init_tree": false
    },
    "parent": {
      "exec_id": "azNkLWVsYXN0aWMtc2VydmVyLTA6MTU1MjE3ODU1MTA1NTAxOTo5NTU3MjM=",
      "pid": 955723,
      "uid": 1000,
      "cwd": "/usr/share/elasticsearch",
      "binary": "/usr/bin/bash",
      "arguments": "/mnt/elastic-internal/scripts/readiness-port-script.sh",
      "flags": "execve",
      "start_time": "2025-10-23T08:42:43.199111326Z",
      "auid": 4294967295,
      "pod": {
        "namespace": "default",
        "name": "elasticsearch-quickstart-es-default-0",
        "container": {
          "id": "containerd://b0feb994b531d61af43add66ae0be1fe29271c67c3def8eff8dacb2ca172380e",
          "name": "elasticsearch",
          "image": {
            "id": "docker.elastic.co/elasticsearch/elasticsearch@sha256:e0dd6421674d6b28224ca38cb9a19e79c1f365361e3e5b295cb87eca7ecfdad9",
            "name": "docker.elastic.co/elasticsearch/elasticsearch:8.18.1"
          },
          "start_time": "2025-10-17T03:32:45Z",
          "pid": 798758,
          "security_context": {}
        },
        "pod_labels": {
          "apps.kubernetes.io/pod-index": "0",
          "common.k8s.elastic.co/type": "elasticsearch",
          "controller-revision-hash": "elasticsearch-quickstart-es-default-6756799d95",
          "elasticsearch.k8s.elastic.co/cluster-name": "elasticsearch-quickstart",
          "elasticsearch.k8s.elastic.co/http-scheme": "https",
          "elasticsearch.k8s.elastic.co/node-data": "true",
          "elasticsearch.k8s.elastic.co/node-data_cold": "true",
          "elasticsearch.k8s.elastic.co/node-data_content": "true",
          "elasticsearch.k8s.elastic.co/node-data_frozen": "true",
          "elasticsearch.k8s.elastic.co/node-data_hot": "true",
          "elasticsearch.k8s.elastic.co/node-data_warm": "true",
          "elasticsearch.k8s.elastic.co/node-ingest": "true",
          "elasticsearch.k8s.elastic.co/node-master": "true",
          "elasticsearch.k8s.elastic.co/node-ml": "true",
          "elasticsearch.k8s.elastic.co/node-remote_cluster_client": "true",
          "elasticsearch.k8s.elastic.co/node-transform": "true",
          "elasticsearch.k8s.elastic.co/node-voting_only": "false",
          "elasticsearch.k8s.elastic.co/statefulset-name": "elasticsearch-quickstart-es-default",
          "elasticsearch.k8s.elastic.co/version": "8.18.1",
          "statefulset.kubernetes.io/pod-name": "elasticsearch-quickstart-es-default-0"
        },
        "workload": "elasticsearch-quickstart-es-default",
        "workload_kind": "StatefulSet"
      },
      "docker": "b0feb994b531d61af43add66ae0be1f",
      "parent_exec_id": "azNkLWVsYXN0aWMtc2VydmVyLTA6MTU1MjE3ODU1MDU1NDAzMDo5NTU3MjM=",
      "tid": 955723,
      "in_init_tree": false
    },
    "time": "2025-10-23T08:42:43.201610160Z"
  },
  "node_name": "k3d-elastic-server-0",
  "time": "2025-10-23T08:42:43.201610080Z",
  "node_labels": {
    "beta.kubernetes.io/arch": "amd64",
    "beta.kubernetes.io/instance-type": "k3s",
    "beta.kubernetes.io/os": "linux",
    "kubernetes.io/arch": "amd64",
    "kubernetes.io/hostname": "k3d-elastic-server-0",
    "kubernetes.io/os": "linux",
    "node-role.kubernetes.io/control-plane": "true",
    "node-role.kubernetes.io/master": "true",
    "node.kubernetes.io/instance-type": "k3s"
  }
}
{
  "process_exit": {
    "process": {
      "exec_id": "azNkLWVsYXN0aWMtc2VydmVyLTA6MTU1MjE3ODU1MTA1NTAxOTo5NTU3MjM=",
      "pid": 955723,
      "uid": 1000,
      "cwd": "/usr/share/elasticsearch",
      "binary": "/usr/bin/bash",
      "arguments": "/mnt/elastic-internal/scripts/readiness-port-script.sh",
      "flags": "execve",
      "start_time": "2025-10-23T08:42:43.199111326Z",
      "auid": 4294967295,
      "pod": {
        "namespace": "default",
        "name": "elasticsearch-quickstart-es-default-0",
        "container": {
          "id": "containerd://b0feb994b531d61af43add66ae0be1fe29271c67c3def8eff8dacb2ca172380e",
          "name": "elasticsearch",
          "image": {
            "id": "docker.elastic.co/elasticsearch/elasticsearch@sha256:e0dd6421674d6b28224ca38cb9a19e79c1f365361e3e5b295cb87eca7ecfdad9",
            "name": "docker.elastic.co/elasticsearch/elasticsearch:8.18.1"
          },
          "start_time": "2025-10-17T03:32:45Z",
          "pid": 798758,
          "security_context": {}
        },
        "pod_labels": {
          "apps.kubernetes.io/pod-index": "0",
          "common.k8s.elastic.co/type": "elasticsearch",
          "controller-revision-hash": "elasticsearch-quickstart-es-default-6756799d95",
          "elasticsearch.k8s.elastic.co/cluster-name": "elasticsearch-quickstart",
          "elasticsearch.k8s.elastic.co/http-scheme": "https",
          "elasticsearch.k8s.elastic.co/node-data": "true",
          "elasticsearch.k8s.elastic.co/node-data_cold": "true",
          "elasticsearch.k8s.elastic.co/node-data_content": "true",
          "elasticsearch.k8s.elastic.co/node-data_frozen": "true",
          "elasticsearch.k8s.elastic.co/node-data_hot": "true",
          "elasticsearch.k8s.elastic.co/node-data_warm": "true",
          "elasticsearch.k8s.elastic.co/node-ingest": "true",
          "elasticsearch.k8s.elastic.co/node-master": "true",
          "elasticsearch.k8s.elastic.co/node-ml": "true",
          "elasticsearch.k8s.elastic.co/node-remote_cluster_client": "true",
          "elasticsearch.k8s.elastic.co/node-transform": "true",
          "elasticsearch.k8s.elastic.co/node-voting_only": "false",
          "elasticsearch.k8s.elastic.co/statefulset-name": "elasticsearch-quickstart-es-default",
          "elasticsearch.k8s.elastic.co/version": "8.18.1",
          "statefulset.kubernetes.io/pod-name": "elasticsearch-quickstart-es-default-0"
        },
        "workload": "elasticsearch-quickstart-es-default",
        "workload_kind": "StatefulSet"
      },
      "docker": "b0feb994b531d61af43add66ae0be1f",
      "parent_exec_id": "azNkLWVsYXN0aWMtc2VydmVyLTA6MTU1MjE3ODU1MDU1NDAzMDo5NTU3MjM=",
      "tid": 955723,
      "in_init_tree": false
    },
    "parent": {
      "exec_id": "azNkLWVsYXN0aWMtc2VydmVyLTA6MTU1MjE3ODU1MDU1NDAzMDo5NTU3MjM=",
      "pid": 955723,
      "uid": 1000,
      "cwd": "/usr/share/elasticsearch",
      "binary": "/mnt/elastic-internal/scripts/readiness-port-script.sh",
      "arguments": "bash /mnt/elastic-internal/scripts/readiness-port-script.sh",
      "flags": "execve",
      "start_time": "2025-10-23T08:42:43.198610668Z",
      "auid": 4294967295,
      "pod": {
        "namespace": "default",
        "name": "elasticsearch-quickstart-es-default-0",
        "container": {
          "id": "containerd://b0feb994b531d61af43add66ae0be1fe29271c67c3def8eff8dacb2ca172380e",
          "name": "elasticsearch",
          "image": {
            "id": "docker.elastic.co/elasticsearch/elasticsearch@sha256:e0dd6421674d6b28224ca38cb9a19e79c1f365361e3e5b295cb87eca7ecfdad9",
            "name": "docker.elastic.co/elasticsearch/elasticsearch:8.18.1"
          },
          "start_time": "2025-10-17T03:32:45Z",
          "pid": 798758,
          "security_context": {}
        },
        "pod_labels": {
          "apps.kubernetes.io/pod-index": "0",
          "common.k8s.elastic.co/type": "elasticsearch",
          "controller-revision-hash": "elasticsearch-quickstart-es-default-6756799d95",
          "elasticsearch.k8s.elastic.co/cluster-name": "elasticsearch-quickstart",
          "elasticsearch.k8s.elastic.co/http-scheme": "https",
          "elasticsearch.k8s.elastic.co/node-data": "true",
          "elasticsearch.k8s.elastic.co/node-data_cold": "true",
          "elasticsearch.k8s.elastic.co/node-data_content": "true",
          "elasticsearch.k8s.elastic.co/node-data_frozen": "true",
          "elasticsearch.k8s.elastic.co/node-data_hot": "true",
          "elasticsearch.k8s.elastic.co/node-data_warm": "true",
          "elasticsearch.k8s.elastic.co/node-ingest": "true",
          "elasticsearch.k8s.elastic.co/node-master": "true",
          "elasticsearch.k8s.elastic.co/node-ml": "true",
          "elasticsearch.k8s.elastic.co/node-remote_cluster_client": "true",
          "elasticsearch.k8s.elastic.co/node-transform": "true",
          "elasticsearch.k8s.elastic.co/node-voting_only": "false",
          "elasticsearch.k8s.elastic.co/statefulset-name": "elasticsearch-quickstart-es-default",
          "elasticsearch.k8s.elastic.co/version": "8.18.1",
          "statefulset.kubernetes.io/pod-name": "elasticsearch-quickstart-es-default-0"
        },
        "workload": "elasticsearch-quickstart-es-default",
        "workload_kind": "StatefulSet"
      },
      "docker": "b0feb994b531d61af43add66ae0be1f",
      "parent_exec_id": "azNkLWVsYXN0aWMtc2VydmVyLTA6MTU1MjE3ODU0ODc5MjA3Nzo5NTU3MjM=",
      "tid": 955723,
      "in_init_tree": false
    },
    "time": "2025-10-23T08:42:43.201790949Z"
  },
  "node_name": "k3d-elastic-server-0",
  "time": "2025-10-23T08:42:43.201791320Z",
  "node_labels": {
    "beta.kubernetes.io/arch": "amd64",
    "beta.kubernetes.io/instance-type": "k3s",
    "beta.kubernetes.io/os": "linux",
    "kubernetes.io/arch": "amd64",
    "kubernetes.io/hostname": "k3d-elastic-server-0",
    "kubernetes.io/os": "linux",
    "node-role.kubernetes.io/control-plane": "true",
    "node-role.kubernetes.io/master": "true",
    "node.kubernetes.io/instance-type": "k3s"
  }
}