How can I extract specific IP addresses from this multiline event?

[eFlea]

I'm trying to generate a list of all IP addresses from all events where the user "timg" has a login failure.

I have hundreds of multiline events that look generally like this single event:

Mar 17, 2010 14:36:32 PM: LOGIN SUCCESS - USER:bobb IP:153.168.134.131
... 56 lines omitted ...
Mar 17, 2010 16:44:13 PM: LOGIN SUCCESS - USER:edk IP:211.219.95.138
Mar 17, 2010 16:44:14 PM: Login failed, Invalid password; User:timg
Mar 17, 2010 17:11:14 PM: LOGIN FAILURE - USER:timg IP:149.117.157.158
Mar 17, 2010 17:14:27 PM: LOGIN SUCCESS - USER:johnc IP:167.182.193.115
Mar 17, 2010 17:18:56 PM: LOGIN SUCCESS - USER:carlk IP:221.140.227.28
Mar 17, 2010 17:27:43 PM: Login failed, Invalid password; User:timg
Mar 17, 2010 17:27:43 PM: LOGIN FAILURE - USER:timg IP:53.37.39.27
Mar 17, 2010 18:14:11 PM: Login failed, Invalid password; User:melb
Mar 17, 2010 18:14:11 PM: LOGIN FAILURE - USER:melb IP:127.197.143.245

For this event, the output I'm looking for would be the following two IP addresses:

149.117.157.158
53.37.39.27

This would be relatively easy if each line in the event was correctly parsed as a separate event, with fields like "Date" "Login Status" "User Name" and "IP Address". Unfortunately, I only have user privileges on this Splunk instance, so I cannot create any configuration files to pre-parse the event.

Can someone please provide some suggestions on how to accomplish this using only the search app?

[ftk · Accepted solution]

You can do this using rex and setting max_match to a high number like so:

your search terms | rex field=_raw max_match=9999 "LOGIN FAILURE - USER:timg IP:(?<ip>\b\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\b)

This should give you a field named "ip" with all of timg's failure audit IPs.

[gkanapathy]

BTW, your search will be far more efficient if you make sure to include as search terms "LOGIN FAILURE" "timg" in the base search, and not just after the rex command, because that will make use of the Splunk index to ignore irrelevant events, rather than using regexes or post-search piping to filter them out after you've processed them

[eFlea]

This worked great. My previous attempts at using rex only gave the first match. Thanks for the help!

[Ayn]

First of all you'll want to extract the required fields explicitly in your search using the rex command.

<yourbasesearch> | rex max_match=9999 field=_raw "LOGIN FAILURE - USER:timg IP:(?<ip_address>\S+)"

With your fields extracted and ready, you can achieve what you want by simply searching for your user where the login status equals "FAILURE" and passing on the results to stats to get your list.

<yourbasesearch>
| rex field=_raw max_match=9999 "LOGIN FAILURE - USER:timg IP:(?<ip_address>\S+)"
| search login_status="FAILURE"
| stats values(ip_address)

[Ayn]

Ah, I totally missed that your event was multiline 🙂

I edited my answer to reflect that. Originally the rex statement would just extract one instance of login status/user/IP per event, which is why other results would be left out.

[eFlea]

This was close, but didn't work quite correctly. I suspect that lines like "Login failed, Invalid password; User:timg" messed up the rex parsing, and left a lot of events as multiline events. This led to a number of IP addresses being left out of the results.

[jbsplunk]

I would do something like:

| rex "(?, \d{1,3}.\d{1,3}.\d{1,3}.\d{1,3})

http://www.splunk.com/base/Documentation/latest/SearchReference/rex