Splunk Search

How to extract the IP and customer_name field from my sample data?

Explorer

Hello,

I am trying to extract the IP address that is noted after START: and the customer name. A customer could have multiple IP addresses, so I want to list the all the IPs, customer name, and count for each.

How do I do that?

My log looks like this:

2016-05-09 20:18:05,403 57.0.2 [ttp-bio-942-exec-163] INFO LoggingFilter -- reqId=234c24a ******************> START: **10.182.50.95 /dsg/test-old/v1/testAll ss=hzufcnfo233rsm897qps&customer_name=abc&network=fa&view_network=fa&locale=en_US&qa=1

0 Karma

Splunk Employee
Splunk Employee

I duplicate your example and then changed the IP's and customer names. The below regex works unless your events change format. I ran a test with your example data as sometimes the rex does not work the same as regex101.com

You highlighted "abc" as the customer name. The Splunk auto extract of the field=value pair does not properly work with your data as demonstrated with the below search

index=main | rex "START\:\s(?((\d{1,3}\.){3}\d{1,3}))\s.*?customer_name\=(?[a-zA-Z\d\s:]+)" | table _time customer_name cust_name ipaddr
0 Karma

Explorer

Since the row has customer_name=abc, you can actually call for customer_name in the search, and it will include the value. Splunk is really smart that way.

As for the IP address, try this regex instead:

 | rex field=_raw "(?<ipaddress>(\d{1,3}\.){3}\d{1,3})" |stats COUNT by ipaddress, customer_name

The regex looks for 1-3 digits followed by a '.' 3 times, followed by 1-3 digits, and assigns it to a newly created variable called ipaddress.

0 Karma

Splunk Employee
Splunk Employee

in order to help you with the regex you should have provided more than 1 sample

but this might give you a guidance. it might need tweak in your environment

sourcetype=xyz |rex &customer_name=(?([\S]+))&view  |rex START:\s(?(\d+.\d+.\d+.\d+))

then you will find 2 interesting port on your left side (interesting fields) by names customer_name & IP

good luck

0 Karma

Explorer

Tried this, did not work, get the below error. (Just tried the rex for IP)

Error in 'rex' command: Encountered the following error while compiling the regex 'START:\s(?(\d+.\d+.\d+.\d+))': Regex: assertion expected after (?(

I think I just need the rex for extracting the list for IP, customer_name is an actual field so I could just use that directly.

0 Karma

Explorer

This seems to work, but still working on removing field extraction

 sourcetype=xyz |rex START:\s(?(\d+.\d+.\d+.\d+))| stats count by IP_add, app|sort -count

Thanks!

0 Karma

Splunk Employee
Splunk Employee

Hi @maddy1011,
Have you tried using the Field Extractor? Here is some documentation that might help:
http://docs.splunk.com/Documentation/Splunk/6.4.0/Knowledge/ExtractfieldsinteractivelywithIFX

We also have some scenario-based documentation that includes a step using the Field Extractor, in case it is helpful.

Full scenario docs are here:
http://docs.splunk.com/Documentation/Splunk/6.4.0/Scenarios/Goals

And an example of using the Field Extractor for an IP address and username is mentioned here:
http://docs.splunk.com/Documentation/Splunk/6.4.0/Scenarios/Extractfields

Hope this helps!

Explorer

I tried using extract field from the sample link. It did extract the IP as a field but just the sample one. Not in a way that would give me the list.
Also can you tell me how do I undo the extraction? all my search results are now defaulting to this extraction.

0 Karma

Splunk Employee
Splunk Employee

Hi @maddy1011,
This topic:
http://docs.splunk.com/Documentation/Splunk/6.4.1/Knowledge/Managesearch-timefieldextractions#Delete...

has information on deleting a field extraction.

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!