ebpf概要
eBPF是几项颠覆性的掌控技术，源于 Linux Mach，能在作业系统Mach等权利语句中运转沙箱流程。它能安全可靠有效地扩充Mach的机能，而无须更动Mach源码或读取Mach模块。 比如说，采用ebpf能追踪任何人Mach求出表达式的模块，codice，以同时实现kernel hook 的效用；透过ebpf还能在互联网数据包抵达Mach协定栈以后就展开处置，这能同时实现互联网流量掌控，即使隐密通讯。
ebpf追踪
ebpf其本质上而已运转在linux Mach中的软件包，要充分发挥其强悍的潜能却是要跟linux kernel 便携式的追踪机能配搭：
kprobe
uprobe
tracepoint
USDT
bcc
BCC 是一个用于创建高效Mach追踪和操作流程的辅助工具包，包括几个有用的辅助工具和示例。它利用扩充的 BPF（Berkeley Packet Filters），正式名称为 eBPF，这是 Linux 3.15 中首次添加的新机能。BCC 采用的大部分内容都需要 Linux 4.1 及更高版本。
源码安装bcc v0.25.0
git clonehttps://github.com/iovisor/bcc.git git checkout v0.25.0 git submodule init git submodule update
bcc 从v0.10.0开始采用libbpf 并透过submodule 的形式加入源码树，所以这里需要更新并拉取子模块
apt install flex bison libdebuginfod-dev libclang-14-dev
mkdir build && cd build cmake -DCMAKE_BUILD_TYPE=Release .. make -j #n取决于机器的cpu核心数
编译安装完成后，在python3中就能采用bcc模块了 安装bcc时会在/usr/share/bcc目录下安装bcc便携式的示例脚本和辅助工具脚本，以及manual 文档 能直接采用man -M /usr/share/bcc/man<keyword>来查询
采用python + bcc 追踪Mach表达式
bcc 便携式的辅助工具execsnoop能追踪execv系统调用，其源码如下：
#!/usr/bin/python# @lint-avoid-python-3-compatibility-imports## execsnoop Trace new processes via exec() syscalls.# For Linux, uses BCC, eBPF. Embedded C.## USAGE: execsnoop [-h] [-T] [-t] [-x] [-q] [-n NAME] [-l LINE]# [–max-args MAX_ARGS]## This currently will print up to a maximum of 19 arguments, plus the process# name, so 20 fields in total (MAXARG).## This wont catch all new processes: an application may fork() but not exec().## Copyright 2016 Netflix, Inc.# Licensed under the Apache License, Version 2.0 (the “License”)## 07-Feb-2016 Brendan Gregg Created this.from__future__importprint_functionfrombccimportBPFfrombcc.containersimportfilter_by_containersfrombcc.utilsimportArgString, printbimportbcc.utilsasutilsimportargparseimportreimporttimeimportpwdfromcollectionsimportdefaultdictfromtimeimportstrftimedefparse_uid(user):try:result=int(user)exceptValueError:try:user_info=pwd.getpwnam(user)exceptKeyError:raiseargparse.ArgumentTypeError(“{0!r} is not valid UID or user entry”.format(user))else:returnuser_info.pw_uidelse:# Maybe validate if UID < 0 ?returnresult# argumentsexamples=“””examples:./execsnoop # trace all exec() syscalls ./execsnoop -x # include failed exec()s./execsnoop -T # include time (HH:MM:SS) ./execsnoop -U # include UID ./execsnoop -u 1000 # only trace UID 1000./execsnoop -u user # get user UID and trace only them ./execsnoop -t # include timestamps./execsnoop -q # add “quotemarks” around arguments ./execsnoop -n main # only print command lines containing “main”./execsnoop -l tpkg # only print command where arguments contains “tpkg”./execsnoop –cgroupmap mappath # only trace cgroups in this BPF map./execsnoop –mntnsmap mappath # only trace mount namespaces in the map“””parser=argparse.ArgumentParser(description=“Trace exec() syscalls”,formatter_class=argparse.RawDescriptionHelpFormatter,epilog=examples)parser.add_argument(“-T”, “–time”, action=“store_true”,help=“include time column on output (HH:MM:SS)”)parser.add_argument(“-t”, “–timestamp”, action=“store_true”,help=“include timestamp on output”)parser.add_argument(“-x”, “–fails”, action=“store_true”,help=“include failed exec()s”)parser.add_argument(“–cgroupmap”,help=“trace cgroups in this BPF map only”)parser.add_argument(“–mntnsmap”,help=“trace mount namespaces in this BPF map only”)parser.add_argument(“-u”, “–uid”, type=parse_uid, metavar=USER,help=“trace this UID only”)parser.add_argument(“-q”, “–quote”, action=“store_true”,help=“Add quotemarks (\”) around arguments.” )parser.add_argument(“-n”, “–name”,type=ArgString,help=“only print commands matching this name (regex), any arg”)parser.add_argument(“-l”, “–line”,type=ArgString,help=“only print commands where arg contains this line (regex)”)parser.add_argument(“-U”, “–print-uid”, action=“store_true”,help=“print UID column”)parser.add_argument(“–max-args”, default=“20”,help=“maximum number of arguments parsed and displayed, defaults to 20”)parser.add_argument(“–ebpf”, action=“store_true”,help=argparse.SUPPRESS)args=parser.parse_args()# define BPF programbpf_text=“””#include <uapi/linux/ptrace.h>#include <linux/sched.h>#include <linux/fs.h>#define ARGSIZE 128enum event_type { EVENT_ARG, EVENT_RET,};struct data_t {u32 pid; // PID as in the userspace term (i.e. task->tgid in kernel)u32 ppid; // Parent PID as in the userspace term (i.e task->real_parent->tgid in kernel) u32 uid; char comm[TASK_COMM_LEN]; enum event_type type; char argv[ARGSIZE];int retval;};BPF_PERF_OUTPUT(events);static int __submit_arg(struct pt_regs *ctx, void *ptr, struct data_t *data){ bpf_probe_read_user(data->argv, sizeof(data->argv), ptr);events.perf_submit(ctx, data, sizeof(struct data_t)); return 1;}static int submit_arg(struct pt_regs *ctx, void *ptr, struct data_t *data){ const char *argp = NULL; bpf_probe_read_user(&argp, sizeof(argp), ptr); if (argp) {return __submit_arg(ctx, (void *)(argp), data); } return 0;}int syscall__execve(struct pt_regs *ctx,const char __user *filename, const char __user *const __user *__argv, const char __user *const __user *__envp){u32 uid = bpf_get_current_uid_gid() & 0xffffffff; UID_FILTER if (container_should_be_filtered()) {return 0; } // create data here and pass to submit_arg to save stack space (#555) struct data_t data = {};struct task_struct *task; data.pid = bpf_get_current_pid_tgid() >> 32;task = (struct task_struct *)bpf_get_current_task(); // Some kernels, like Ubuntu 4.13.0-generic, return 0 // as the real_parent->tgid.// We use the get_ppid function as a fallback in those cases. (#1883)data.ppid = task->real_parent->tgid; bpf_get_current_comm(&data.comm, sizeof(data.comm)); data.type = EVENT_ARG;__submit_arg(ctx, (void *)filename, &data); // skip first arg, as we submitted filename #pragma unrollfor (int i = 1; i < MAXARG; i++) { if (submit_arg(ctx, (void *)&__argv[i], &data) == 0) goto out; }// handle truncated argument list char ellipsis[] = “…”; __submit_arg(ctx, (void *)ellipsis, &data);out: return 0;}int do_ret_sys_execve(struct pt_regs *ctx){ if (container_should_be_filtered()) { return 0; }struct data_t data = {}; struct task_struct *task; u32 uid = bpf_get_current_uid_gid() & 0xffffffff; UID_FILTERdata.pid = bpf_get_current_pid_tgid() >> 32; data.uid = uid; task = (struct task_struct *)bpf_get_current_task();// Some kernels, like Ubuntu 4.13.0-generic, return 0 // as the real_parent->tgid.// We use the get_ppid function as a fallback in those cases. (#1883) data.ppid = task->real_parent->tgid;bpf_get_current_comm(&data.comm, sizeof(data.comm)); data.type = EVENT_RET; data.retval = PT_REGS_RC(ctx);events.perf_submit(ctx, &data, sizeof(data)); return 0;}“””bpf_text=bpf_text.replace(“MAXARG”, args.max_args)ifargs.uid:bpf_text=bpf_text.replace(UID_FILTER,if (uid != %s) { return 0; }%args.uid)else:bpf_text=bpf_text.replace(UID_FILTER, )bpf_text=filter_by_containers(args) +bpf_textifargs.ebpf:print(bpf_text)exit()# initialize BPFb=BPF(text=bpf_text)execve_fnname=b.get_syscall_fnname(“execve”)b.attach_kprobe(event=execve_fnname, fn_name=“syscall__execve”)b.attach_kretprobe(event=execve_fnname, fn_name=“do_ret_sys_execve”)# headerifargs.time:print(“%-9s”% (“TIME”), end=“”)ifargs.timestamp:print(“%-8s”% (“TIME(s)”), end=“”)ifargs.print_uid:print(“%-6s”% (“UID”), end=“”)print(“%-16s %-7s %-7s %3s %s”% (“PCOMM”, “PID”, “PPID”, “RET”, “ARGS”))classEventType(object):EVENT_ARG=0EVENT_RET=1start_ts=time.time()argv=defaultdict(list)# This is best-effort PPID matching. Short-lived processes may exit# before we get a chance to read the PPID.# This is a fallback for when fetching the PPID from task->real_parent->tgip# returns 0, which happens in some kernel versions.defget_ppid(pid):try:withopen(“/proc/%d/status”%pid) asstatus:forlineinstatus:ifline.startswith(“PPid:”):returnint(line.split()[1])exceptIOError:passreturn0# process eventdefprint_event(cpu, data, size):event=b[“events”].event(data)skip=Falseifevent.type==EventType.EVENT_ARG:argv[event.pid].append(event.argv)elifevent.type==EventType.EVENT_RET:ifevent.retval!=0andnotargs.fails:skip=Trueifargs.nameandnotre.search(bytes(args.name), event.comm):skip=Trueifargs.lineandnotre.search(bytes(args.line),b .join(argv[event.pid])):skip=Trueifargs.quote:argv[event.pid] = [b”\””+arg.replace(b”\””, b”\\\””) +b”\””forarginargv[event.pid] ]ifnotskip:ifargs.time:printb(b”%-9s”%strftime(“%H:%M:%S”).encode(ascii), nl=“”)ifargs.timestamp:printb(b”%-8.3f”% (time.time() –start_ts), nl=“”)ifargs.print_uid:printb(b”%-6d”%event.uid, nl=“”)ppid=event.ppidifevent.ppid>0elseget_ppid(event.pid)ppid=b”%d”%ppidifppid>0elseb”?”argv_text=b .join(argv[event.pid]).replace(b\n, b\\n)printb(b”%-16s %-7d %-7s %3d %s”% (event.comm, event.pid,ppid, event.retval, argv_text))try:del(argv[event.pid])exceptException:pass# loop with callback to print_eventb[“events”].open_perf_buffer(print_event)while1:try:b.perf_buffer_poll()exceptKeyboardInterrupt:exit()
此辅助工具采用kprobe和kretprobe追踪execv系统调用的进入和退出事件，并将进程名，进程模块，pid，ppid以及返回代码输出到终端。
① 网安学习成长路径思维导图② 60+网安经典常用辅助工具包③ 100+SRC漏洞分析报告④ 150+网安攻防实战掌控技术电子书⑤ 最权威CISSP 认证考试指南+题库⑥ 超1800页CTF实战技巧手册⑦ 最新网安大厂面试题合集（含答案）⑧ APP客户端安全可靠检测指南（安卓+IOS）
使用python + bcc 追踪用户表达式
bcc中采用uprobe追踪glibc malloc 表达式的辅助工具，并统计malloc 内存的总量。
#!/usr/bin/python## mallocstacks Trace malloc() calls in a process and print the full# stack trace for all callsites.# For Linux, uses BCC, eBPF. Embedded C.## This script is a basic example of the new Linux 4.6+ BPF_STACK_TRACE# table API.## Copyright 2016 GitHub, Inc.# Licensed under the Apache License, Version 2.0 (the “License”)from__future__importprint_functionfrombccimportBPFfrombcc.utilsimportprintbfromtimeimportsleepimportsysiflen(sys.argv) <2:print(“USAGE: mallocstacks PID [NUM_STACKS=1024]”)exit()pid=int(sys.argv[1])iflen(sys.argv) ==3:try:assertint(sys.argv[2]) >0, “”except (ValueError, AssertionError) ase:print(“USAGE: mallocstacks PID [NUM_STACKS=1024]”)print(“NUM_STACKS must be a non-zero, positive integer”)exit()stacks=sys.argv[2]else:stacks=“1024”# load BPF programb=BPF(text=“””#include <uapi/linux/ptrace.h>BPF_HASH(calls, int);BPF_STACK_TRACE(stack_traces, “””+stacks+“””);int alloc_enter(struct pt_regs *ctx, size_t size) { int key = stack_traces.get_stackid(ctx, BPF_F_USER_STACK); if (key < 0) return 0;// could also use `calls.increment(key, size);` u64 zero = 0, *val;val = calls.lookup_or_try_init(&key, &zero); if (val) { (*val) += size; } return 0;};“””)b.attach_uprobe(name=“c”, sym=“malloc”, fn_name=“alloc_enter”, pid=pid)print(“Attaching to malloc in pid %d, Ctrl+C to quit.”%pid)# sleep until Ctrl-Ctry:sleep(99999999)exceptKeyboardInterrupt:passcalls=b.get_table(“calls”)stack_traces=b.get_table(“stack_traces”)fork, vinreversed(sorted(calls.items(), key=lambdac: c[1].value)):print(“%d bytes allocated at:”%v.value)ifk.value>0 :foraddrinstack_traces.walk(k.value):printb(b”\t%s”%b.sym(addr, pid, show_offset=True))
libbpf
libbpf是linux 源码树中的ebpf 开发包。同时在github上也有独立的代码仓库。 这里推荐采用libbpf-bootstrap这个项目
libbpf-bootstrap
libbpf-bootstrap是采用 libbpf 和 BPF CO-RE 展开 BPF 应用流程开发的脚手架项目 首先克隆libbpf-bootstrap仓库
git clone https://github.com/libbpf/libbpf-bootstrap.git
cd libbpf-bootstrap git submodule init git submodule update
注意，子模块中包含bpftool，bpftool中还有子模块需要同步 在bpftool目录下重复以上步骤
这里进入example/c中，这里包含一些示例工具 直接make编译 等编译完成后，在此目录下会生成可执行文件
先运转一下bootstrap，这里要用root权限运转
bootstrap流程会追踪所有的exec和exit系统调用，每次流程运转时，bootstrap就会输出运转流程的信息。
运转后输出大量信息，最后有提示让我们运转sudo cat /sys/kernel/debug/tracing/trace_pipe来查看输出 运转这个命令
minimal 会追踪所有的write系统调用，并打印出调用write的进程的pid 这里看到pid为11494，ps 查询一下这个进程，发现就是minimal
来看看minimal的源码，这个流程主要有两个C文件组成，minimal.c和minimal.bpf.c前者为此流程的源码，后者为插入Mach软件包的ebpf代码。
// SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)/* Copyright (c) 2020 Facebook */#include <stdio.h>#include <unistd.h>#include <sys/resource.h>#include <bpf/libbpf.h>#include “minimal.skel.h”staticintlibbpf_print_fn(enumlibbpf_print_levellevel, constchar*format, va_listargs){returnvfprintf(stderr, format, args);}intmain(intargc, char**argv){structminimal_bpf*skel;interr;libbpf_set_strict_mode(LIBBPF_STRICT_ALL);/* Set up libbpf errors and debug info callback */libbpf_set_print(libbpf_print_fn);/* Open BPF application */skel=minimal_bpf__open();if (!skel) {fprintf(stderr, “Failed to open BPF skeleton\n”);return1; }/* ensure BPF program only handles write() syscalls from our process */skel->bss->my_pid=getpid();/* Load & verify BPF programs */err=minimal_bpf__load(skel);if (err) {fprintf(stderr, “Failed to load and verify BPF skeleton\n”);gotocleanup; }/* Attach tracepoint handler */err=minimal_bpf__attach(skel);if (err) {fprintf(stderr, “Failed to attach BPF skeleton\n”);gotocleanup; }printf(“Successfully started! Please run `sudo cat /sys/kernel/debug/tracing/trace_pipe` ““to see output of the BPF programs.\n”);for (;;) {/* trigger our BPF program */fprintf(stderr, “.”);sleep(1); }cleanup:minimal_bpf__destroy(skel);return–err;}
首先看一下minimal.c的内容，在main表达式中首先调用了libbpf_set_strict_mode(LIBBPF_STRICT_ALL);设置为libbpf v1.0模式。此模式下错误代码直接透过表达式codice传递，不再需要检查errno。 之后调用libbpf_set_print(libbpf_print_fn);将流程中一个自定义输出表达式设置为调试输出的回调函数，即运转minimal的这些输出全都时透过libbpf_print_fn输出的。
然后在minimal.c:24调用生成的minimal.skel.h中的预定义表达式minimal_bpfopen打开bpf流程，这里返回一个minimal_bpf类型的对象（c中采用结构体模拟对象）。 在31行将minimal_bpf对象的bss子对象的my_pid属性设置为当前进程pid 这里minimal_bpf对象和bss都由minimal.bpf.c代码编译而来。minimal.bpf.c经过clang 编译连接，生成minimal.bpf.o，这是一个elf文件，其中包含bss段，这个段内一般来说储存着minimal.bpf.c中所有经过初始化的变量。 skel->bss->my_pid = getpid();就是直接将minimal.bpf.o中的my_pid设置为minimal进程的pid。 之后在34行调用minimal_bpfload(skel);读取并验证ebpf流程。 41行调用minimal_bpfattach(skel);使ebpf流程附加到bpf源码中声明的追踪点上。 此时ebpf流程已经开始运转了。ebpf中透过bpf_printk输出的内容会写入linux debugFS中的trace_pipe中。能采用sudo cat /sys/kernel/debug/tracing/trace_pipe输出到终端里。 之后minimal流程会进入一个死循环，以维持ebpf流程的运转。当用户按下发送SIGINT信号后就会调用minimal_bpfdestroy(skel);卸载内核中的ebpf流程，之后退出。
接下来看minimal.bpf.c 这是ebpf流程的源码，是要读取到Mach中的ebpf软件包中运转的，由于在运转在Mach中，具有得天独厚的地理位置，能访问系统中所有资源，再配合上众多的tracepoint，就能充分发挥出强悍的追踪潜能。 下面是minimal.bpf.c的源码
// SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause/* Copyright (c) 2020 Facebook */#include <linux/bpf.h>#include <bpf/bpf_helpers.h>charLICENSE[] SEC(“license”) =“Dual BSD/GPL”;intmy_pid=0;SEC(“tp/syscalls/sys_enter_write”)inthandle_tp(void*ctx){intpid=bpf_get_current_pid_tgid() >>32;if (pid!=my_pid)return0;bpf_printk(“BPF triggered from PID %d.\n”, pid);return0;}
minimal.bpf.c会被clang 编译器编译为ebpf字节码，然后透过bpftool将其转换为minimal.skel.h头文件，以供minimal.c采用。 此代码中定义并初始化了一个全局变量my_pid，经过编译连接后此变量会进入elf文件的bss段中。 然后，代码中定义了一个表达式int handle_tp(void *ctx)，此表达式中透过调用bpf_g
然后比较pid与my_pid的值，如果相同则调用bpf_printk输出”BPF triggered from PID %d\n” 这里由于handle_tp表达式是透过SEC宏附加在write系统调用上，所以在调用write()时，handle_tp也会被调用，从而同时实现追踪系统调用的机能。 SEC宏在bpf流程中处于非常重要的地位。能参考此文档SEC宏能指定ebpf表达式附加的点，包括系统调用，静态tracepoint，动态的kprobe和uprobe，以及USDT等等。 Libbpf 期望 BPF 流程采用SEC()宏注释，其中传入的字符串模块SEC()确定 BPF 流程类型和可选的附加附加模块，例如 kprobe 流程要附加的Mach表达式名称或 cgroup 流程的挂钩类型。该SEC()定义最终被记录为 ELF section name。
透过llvm-objdump 能看到编译后的epbf流程文件包含一个以追踪点命名的section
ebpf字节码dump
ebpf流程能采用llvm-objdump -d dump 出ebpf字节码
bpftrace
bpftrace 提供了一种类似awk 的脚本语言，透过编写脚本，配合bpftrace支持的追踪点，能同时实现非常强悍的追踪机能
sudo apt-get update sudo apt-get install -y \ bison \ cmake \ flex \ g++ \ git \ libelf-dev \ zlib1g-dev \ libfl-dev \ systemtap-sdt-dev \ binutils-dev \ libcereal-dev \ llvm-12-dev \ llvm-12-runtime \ libclang-12-dev \ clang-12 \ libpcap-dev \ libgtest-dev \ libgmock-dev \ asciidoctor git clonehttps://github.com/iovisor/bpftracemkdir bpftrace/build; cd bpftrace/build; ../build-libs.sh cmake -DCMAKE_BUILD_TYPE=Release .. make -j8 sudo make install
bpftrace命令行模块
# bpftraceUSAGE: bpftrace [options] filenamebpftrace [options]-eprogramOPTIONS:-B MODE output buffering mode (line, full, or none)-d debug info dry run-dd verbose debug info dry run-eprogram execute this program-h show this help message-IDIR add the specified DIR to the search pathfor include files.–include FILE adds an implicit #include which is read before the source file is preprocessed.-l [search] list probes-p PID enable USDT probes on PID-cCMDrun CMD and enable USDT probes on resulting process-q keep messages quiet-v verbose messages-kemit a warning when a bpf helper returns an error (except read functions)-kk check all bpf helper functions–version bpftrace versionENVIRONMENT: BPFTRACE_STRLEN [default: 64] bytes on BPF stack per str()BPFTRACE_NO_CPP_DEMANGLE [default:0] disable C++ symbol demangling BPFTRACE_MAP_KEYS_MAX [default: 4096] max keys ina map BPFTRACE_MAX_PROBES [default: 512] max number of probes bpftrace can attach toBPFTRACE_MAX_BPF_PROGS [default:512] max number of generated BPF programsBPFTRACE_CACHE_USER_SYMBOLS [default: auto] enable user symbol cache BPFTRACE_VMLINUX [default: none] vmlinux path used for kernel symbol resolutionBPFTRACE_BTF [default: none] BTF fileEXAMPLES:bpftrace -l*sleep* list probes containing “sleep”bpftrace -ekprobe:do_nanosleep { printf(“PID %d sleeping…\n”, pid); } trace processes calling sleepbpftrace -etracepoint:raw_syscalls:sys_enter { @[comm] = count(); } count syscalls by process name
bpftrace流程语法规则
bpftrace语法由下列一个或多个action block结构组成，且语法关键字与c语言类似
probe[,probe]/predicate/ {action}
probe：探针，能采用bpftrace -l 来查看支持的所有tracepoint和kprobe探针
Predicate（可选）：在 / / 中指定 action 执行的条件。如果为True，就执行 action
action：在事件触发时运转的程序，每行语句必须以 ; 结尾，并且用{}包起来
//：单行注释
/**/：多行注释
->：访问c结构体成员，例如：bpftrace -e tracepoint:syscalls:sys_enter_openat { printf(“%s %s\n”, comm, str(args->filename)); }
struct：结构声明，在bpftrace脚本中能定义自己的结构
bpftrace 单行指令
bpftrace -e 选项能指定运转一个单行流程 1、追踪openat系统调用
bpftrace -e tracepoint:syscalls:sys_enter_openat { printf(“%s %s\n”, comm, str(args->filename)); }
bpftrace -e tracepoint:raw_syscalls:sys_enter { @[comm] = count(); }
bpftrace -e tracepoint:raw_syscalls:sys_enter { @ = count(); } interval:s:1 { print(@); clear(@); }
bpftrace脚本文件
还能将bpftrace流程作为一个脚本文件，并且采用shebang#!/usr/local/bin/bpftrace能使其独立运转 例如：
1#!/usr/local/bin/bpftrace23tracepoint:syscalls:sys_enter_nanosleep4 {5printf(“%s is sleeping.\n”, comm);6 }
bpftrace探针类型
kprobe- Mach表达式启动
kretprobe- Mach表达式返回
uprobe- 用户级机能启动
uretprobe- 用户级表达式返回
tracepoint- Mach静态追踪点
usdt- 用户级静态追踪点
profile- 定时采样
interval- 定时输出
software- Mach软件事件
hardware- 处置器级事件
