Merge pull request #271 from brendangregg/master

tcpaccept

README.md

@@ -72,7 +72,8 @@ Tools:
 - tools/[opensnoop](tools/opensnoop): Trace open() syscalls. [Examples](tools/opensnoop_example.txt).
 - tools/[pidpersec](tools/pidpersec): Count new processes (via fork). [Examples](tools/pidpersec_example.txt).
 - tools/[syncsnoop](tools/syncsnoop): Trace sync() syscall. [Examples](tools/syncsnoop_example.txt).
-- tools/[tcpv4connect](tools/tcpv4connect): Trace TCP IPv4 active connections. [Examples](tools/tcpv4connect_example.txt).
+- tools/[tcpaccept](tools/tcpaccept): Trace TCP passive connections (accept()). [Examples](tools/tcpaccept_example.txt).
+- tools/[tcpv4connect](tools/tcpv4connect): Trace TCP IPv4 active connections (connect()). [Examples](tools/tcpv4connect_example.txt).
 - tools/[vfscount](tools/vfscount) tools/[vfscount.c](tools/vfscount.c): Count VFS calls. [Examples](tools/vfscount_example.txt).
 - tools/[vfsstat](tools/vfsstat) tools/[vfsstat.c](tools/vfsstat.c): Count some VFS calls, with column output. [Examples](tools/vfsstat_example.txt).
 

man/man8/tcpaccept.8

+.TH tcpaccept 8  "2015-08-25" "USER COMMANDS"
+.SH NAME
+tcpaccept \- Trace TCP passive connections (accept()). Uses Linux eBPF/bcc.
+.SH SYNOPSIS
+.B tcpaccept [\-h] [\-t] [\-x] [\-p PID]
+.SH DESCRIPTION
+This tool traces passive TCP connections (eg, via an accept() syscall;
+connect() are active connections). This can be useful for general
+troubleshooting to see what new connections the local server is accepting.
+
+This uses dynamic tracing of the kernel inet_csk_accept() socket function (from
+tcp_prot.accept), and will need to be modified to match kernel changes.
+
+This tool only traces successful TCP accept()s. Connection attempts to closed
+ports will not be shown (those can be traced via other functions).
+
+Since this uses BPF, only the root user can use this tool.
+.SH REQUIREMENTS
+CONFIG_BPF and bcc.
+.SH OPTIONS
+.TP
+\-h
+Print usage message.
+.TP
+\-t
+Include a timestamp column.
+.TP
+\-p PID
+Trace this process ID only (filtered in-kernel).
+.SH EXAMPLES
+.TP
+Trace all passive TCP connections (accept()s):
+#
+.B tcpaccept
+.TP
+Trace all TCP accepts, and include timestamps:
+#
+.B tcpconnect \-t
+.TP
+Trace PID 181 only:
+#
+.B tcpconnect \-p 181
+.SH FIELDS
+.TP
+TIME(s)
+Time of the event, in seconds.
+.TP
+PID
+Process ID
+.TP
+COMM
+Process name
+.TP
+IP
+IP address family (4 or 6)
+.TP
+RADDR
+Remote IP address. IPv4 as a dotted quad, IPv6 shows "..." then the last 4
+bytes (check for newer versions of this tool for the full address).
+.TP
+LADDR
+Local IP address. IPv4 as a dotted quad, IPv6 shows "..." then the last 4
+bytes (check for newer versions of this tool for the full address).
+.TP
+LPORT
+Local port
+.SH OVERHEAD
+This traces the kernel inet_csk_accept function and prints output for each event.
+The rate of this depends on your server application. If it is a web or proxy server
+accepting many tens of thousands of connections per second, then the overhead
+of this tool may be measurable (although, still a lot better than tracing
+every packet). If it is less than a thousand a second, then the overhead is
+expected to be negligible. Test and understand this overhead before use.
+.SH SOURCE
+This is from bcc.
+.IP
+https://github.com/iovisor/bcc
+.PP
+Also look in the bcc distribution for a companion _examples.txt file containing
+example usage, output, and commentary for this tool.
+.SH OS
+Linux
+.SH STABILITY
+Unstable - in development.
+.SH AUTHOR
+Brendan Gregg
+.SH SEE ALSO
+tcpv4connect(8), funccount(8), tcpdump(8)

man/man8/tcpv4connect.8

 .TH tcpv4connect 8  "2015-08-25" "USER COMMANDS"
 .SH NAME
-tcpv4connect \- Trace TCP IPv4 active connections. Uses Linux eBPF/bcc.
+tcpv4connect \- Trace TCP IPv4 active connections (connect()). Uses Linux eBPF/bcc.
 .SH SYNOPSIS
 .B tcpv4connect [\-h] [\-t] [\-x] [\-p PID]
 .SH DESCRIPTION
@@ -76,4 +76,4 @@ Unstable - in development.
 .SH AUTHOR
 Brendan Gregg
 .SH SEE ALSO
-funccount(8), tcpdump(8)
+tcpaccept(8), funccount(8), tcpdump(8)

tools/tcpaccept

+#!/usr/bin/python
+#
+# tcpaccept	Trace TCP accept()s.
+#		For Linux, uses BCC, eBPF. Embedded C.
+#
+# USAGE: tcpaccept [-h] [-t] [-p PID]
+#
+# This uses dynamic tracing of the kernel inet_csk_accept() socket function
+# (from tcp_prot.accept), and will need to be modified to match kernel changes.
+#
+# IPv4 addresses are printed as dotted quads. For IPv6 addresses, the last four
+# bytes are printed after "..."; check for future versions with better IPv6
+# support.
+#
+# Copyright (c) 2015 Brendan Gregg.
+# Licensed under the Apache License, Version 2.0 (the "License")
+#
+# 13-Oct-2015	Brendan Gregg	Created this.
+
+from __future__ import print_function
+from bcc import BPF
+import argparse
+
+# arguments
+examples = """examples:
+    ./tcpaccept           # trace all TCP accept()s
+    ./tcpaccept -t        # include timestamps
+    ./tcpaccept -p 181    # only trace PID 181
+"""
+parser = argparse.ArgumentParser(
+	description="Trace TCP accepts",
+	formatter_class=argparse.RawDescriptionHelpFormatter,
+	epilog=examples)
+parser.add_argument("-t", "--timestamp", action="store_true",
+	help="include timestamp on output")
+parser.add_argument("-p", "--pid",
+	help="trace this PID only")
+args = parser.parse_args()
+debug = 0
+
+# define BPF program
+bpf_text = """
+#include <uapi/linux/ptrace.h>
+#include <net/sock.h>
+#include <bcc/proto.h>
+
+int kretprobe__inet_csk_accept(struct pt_regs *ctx)
+{
+	struct sock *newsk = (struct sock *)ctx->ax;
+	u32 pid = bpf_get_current_pid_tgid();
+
+	if (newsk == NULL)
+		return 0;
+
+	// check this is TCP
+	u8 protocol = 0;
+	// workaround for reading the sk_protocol bitfield:
+	bpf_probe_read(&protocol, 1, (void *)((long)&newsk->sk_wmem_queued) - 3);
+	if (protocol != IPPROTO_TCP)
+		return 0;
+
+	// pull in details
+	u16 family = 0, lport = 0;
+	u32 saddr = 0, daddr = 0;
+	bpf_probe_read(&family, sizeof(family), &newsk->__sk_common.skc_family);
+	bpf_probe_read(&lport, sizeof(lport), &newsk->__sk_common.skc_num);
+	if (family == AF_INET) {
+		bpf_probe_read(&saddr, sizeof(saddr),
+		    &newsk->__sk_common.skc_rcv_saddr);
+		bpf_probe_read(&daddr, sizeof(daddr),
+		    &newsk->__sk_common.skc_daddr);
+
+		// output
+		bpf_trace_printk("4 %x %x %d\\n", daddr, saddr, lport);
+	} else if (family == AF_INET6) {
+		// just grab the last 4 bytes for now
+		bpf_probe_read(&saddr, sizeof(saddr),
+		    &newsk->__sk_common.skc_v6_rcv_saddr.in6_u.u6_addr32[3]);
+		bpf_probe_read(&daddr, sizeof(daddr),
+		    &newsk->__sk_common.skc_v6_daddr.in6_u.u6_addr32[3]);
+
+		// output and flip byte order of addresses
+		bpf_trace_printk("6 %x %x %d\\n", bpf_ntohl(daddr),
+		    bpf_ntohl(saddr), lport);
+	}
+	// else drop
+
+	return 0;
+}
+"""
+
+# code substitutions
+if args.pid:
+	bpf_text = bpf_text.replace('FILTER',
+	    'if (pid != %s) { return 0; }' % args.pid)
+else:
+	bpf_text = bpf_text.replace('FILTER', '')
+if debug:
+	print(bpf_text)
+
+# initialize BPF
+b = BPF(text=bpf_text)
+
+# header
+if args.timestamp:
+	print("%-9s" % ("TIME(s)"), end="")
+print("%-6s %-12s %-2s %-16s %-16s %-4s" % ("PID", "COMM", "IP", "RADDR",
+    "LADDR", "LPORT"))
+
+start_ts = 0
+
+def inet_ntoa(addr):
+	dq = ''
+	for i in range(0, 4):
+		dq = dq + str(addr & 0xff)
+		if (i != 3):
+			dq = dq + '.'
+		addr = addr >> 8
+	return dq
+
+# format output
+while 1:
+	(task, pid, cpu, flags, ts, msg) = b.trace_fields()
+	(ip_s, raddr_hs, laddr_hs, lport_s) = msg.split(" ")
+
+	if args.timestamp:
+		if start_ts == 0:
+			start_ts = ts
+		print("%-9.3f" % (ts - start_ts), end="")
+	print("%-6d %-12.12s %-2s %-16s %-16s %-4s" % (pid, task, ip_s,
+	    inet_ntoa(int(raddr_hs, 16)) if ip_s == "4" else "..." + raddr_hs,
+	    inet_ntoa(int(laddr_hs, 16)) if ip_s == "4" else "..." + laddr_hs,
+	    lport_s))

tools/tcpaccept_example.txt

+Demonstrations of tcpaccept, the Linux eBPF/bcc version.
+
+
+This tool traces the kernel function accepting TCP socket connections (eg, a
+passive connection via accept(); not connect()). Some example output (IP
+addresses changed to protect the innocent):
+
+# ./tcpaccept
+PID    COMM         IP RADDR            LADDR            LPORT
+907    sshd         4  192.168.56.1     192.168.56.102   22  
+907    sshd         4  127.0.0.1        127.0.0.1        22  
+5389   perl         6  ...fec0ae21      ...fec0ae21      7001
+
+This output shows three connections, two to PID 907, an "sshd" process listening
+on port 22, and one to a "perl" process listening on port 7001.
+
+The sshd connections were IPv4, and the addresses are printed as dotted quads.
+The perl connection was IPv6, and the last 4 bytes of each address is printed
+(for now; check for updated versions).
+
+The overhead of this tool should be negligible, since it is only tracing the
+kernel function performing accept. It is not tracing every packet and then
+filtering.
+
+This tool only traces successful TCP accept()s. Connection attempts to closed
+ports will not be shown (those can be traced via other functions).
+
+
+The -t option prints a timestamp column:
+
+# ./tcpaccept -t
+TIME(s)  PID    COMM         IP RADDR            LADDR            LPORT
+0.000    907    sshd         4  127.0.0.1        127.0.0.1        22  
+0.992    907    sshd         4  127.0.0.1        127.0.0.1        22  
+1.984    907    sshd         4  127.0.0.1        127.0.0.1        22  
+
+
+USAGE message:
+
+# ./tcpaccept -h
+usage: tcpaccept [-h] [-t] [-p PID]
+
+Trace TCP accepts
+
+optional arguments:
+  -h, --help         show this help message and exit
+  -t, --timestamp    include timestamp on output
+  -p PID, --pid PID  trace this PID only
+
+examples:
+    ./tcpaccept           # trace all TCP accept()s
+    ./tcpaccept -t        # include timestamps
+    ./tcpaccept -p 181    # only trace PID 181

tools/tcpv4connect

@@ -16,7 +16,7 @@ import argparse
 
 # arguments
 examples = """examples:
-    ./tcpv4connect           # trace all open() syscalls
+    ./tcpv4connect           # trace all TCP IPv4 connect()s
     ./tcpv4connect -t        # include timestamps
     ./tcpv4connect -p 181    # only trace PID 181
 """

tools/tcpv4connect_example.txt

@@ -48,6 +48,6 @@ optional arguments:
   -p PID, --pid PID  trace this PID only
 
 examples:
-    ./tcpv4connect           # trace all open() syscalls
+    ./tcpv4connect           # trace all TCP IPv4 connect()s
     ./tcpv4connect -t        # include timestamps
     ./tcpv4connect -p 181    # only trace PID 181