How to matching tcpdump data?

[tnhtnh]

Hey

I'm trying to create a search app for tcpdump - a splunk version of mk-tcp-model.

I need to somehow associate the tcpdump packets with one another. For example the tcpdump ouptut im ingesting is:

2011-08-15 13:28:36.851862 IP 127.0.0.1.3306 > 127.0.0.1.52888: tcp 78
2011-08-15 13:28:36.853024 IP 127.0.0.1.52888 > 127.0.0.1.3306: tcp 64
2011-08-15 13:28:36.853138 IP 127.0.0.1.3306 > 127.0.0.1.52888: tcp 11
2011-08-15 13:28:36.853230 IP 127.0.0.1.52888 > 127.0.0.1.3306: tcp 37
2011-08-15 13:28:36.853321 IP 127.0.0.1.3306 > 127.0.0.1.52888: tcp 99
2011-08-15 13:28:40.862205 IP 127.0.0.1.52888 > 127.0.0.1.3306: tcp 22
2011-08-15 13:28:40.862334 IP 127.0.0.1.3306 > 127.0.0.1.52888: tcp 64
2011-08-15 13:28:40.862379 IP 127.0.0.1.52888 > 127.0.0.1.3306: tcp 9
2011-08-15 13:28:40.862438 IP 127.0.0.1.3306 > 127.0.0.1.52888: tcp 11
2011-08-15 13:28:40.863192 IP 127.0.0.1.52888 > 127.0.0.1.3306: tcp 19
2011-08-15 13:28:40.863448 IP 127.0.0.1.3306 > 127.0.0.1.52888: tcp 175
2011-08-15 13:28:40.863543 IP 127.0.0.1.52888 > 127.0.0.1.3306: tcp 16
2011-08-15 13:28:40.863646 IP 127.0.0.1.3306 > 127.0.0.1.52888: tcp 113
2011-08-15 13:28:41.590145 IP 127.0.0.1.52888 > 127.0.0.1.3306: tcp 5

The contains the query and response in sequence -eg:

2011-08-15 13:28:36.851862 IP 127.0.0.1.3306 > 127.0.0.1.52888: tcp 78
2011-08-15 13:28:36.853024 IP 127.0.0.1.52888 > 127.0.0.1.3306: tcp 64

A single query that took 13:28:36.853024 -13:28:36.851862 seconds to execute.

2011-08-15 13:28:36.853138 IP 127.0.0.1.3306 > 127.0.0.1.52888: tcp 11
2011-08-15 13:28:36.853230 IP 127.0.0.1.52888 > 127.0.0.1.3306: tcp 37

A single query that took 13:28:36.853230 -13:28:36.853138 seconds to execute.

and so on..

So I'm having trouble building a transaction within spunk for them. I'm not sure if I can or not.

It hink this is what will work:

source="/tmp/tcpdump.out"  sourcetype="out-too_small" | rename srcipport as ipport  | rename dstipport as ipport  | transaction ipport maxevents=2

Has anyone else done this before?

[scentoni_splunk]

Sorry this is probably too late to help you, but maybe it will help others trying to do this:

| eval channel=if( srcip . srcport < dstip . dstport,
    printf("%s:%s-%s:%s",srcip,srcport,dstip,dstport),
    printf("%s:%s-%s:%s",dstip,dstport,srcip,srcport))
| transaction channel

What you want to compare is not exactly a 4-tuple but a set of two 2-tuples. Since the directionality is not relevant, we arbitrarily sort the list of two (ip,port) 2-tuples {src, dst} so that the first one is less than the second one.

[dwaddle]

I think transaction is the right way to go, but you should probably consider a different set of field extractions and transaction fields.

A single TCP session is identifiable by a 4-tuple -- (source_ip,source_port,dest_ip,dest_port). You need to extract ALL of these and use them ALL as the grouping fields on your transaction command.

I think one issue here, though, is that the definition of source_ip and dest_ip change depending on which participant in the session is sending the packet. I see where you tried to fix that via rename -- but I'm not sure that will work in all cases.