How to extract the IP and customer_name field from my sample data?

[maddy1011]

Hello,

I am trying to extract the IP address that is noted after START: and the customer name. A customer could have multiple IP addresses, so I want to list the all the IPs, customer name, and count for each.

How do I do that?

My log looks like this:

2016-05-09 20:18:05,403 57.0.2 [ttp-bio-942-exec-163] INFO LoggingFilter -- reqId=234c24a ******************> START: **10.182.50.95 /dsg/test-old/v1/testAll ss=hzufcnfo233rsm897qps&customer_name=abc&network=fa&view_network=fa&locale=en_US&qa=1

[kbrown_splunk]

I duplicate your example and then changed the IP's and customer names. The below regex works unless your events change format. I ran a test with your example data as sometimes the rex does not work the same as regex101.com

You highlighted "abc" as the customer name. The Splunk auto extract of the field=value pair does not properly work with your data as demonstrated with the below search

index=main | rex "START\:\s(?((\d{1,3}\.){3}\d{1,3}))\s.*?customer_name\=(?[a-zA-Z\d\s:]+)" | table _time customer_name cust_name ipaddr

[drinkingjimmy]

Since the row has customer_name=abc, you can actually call for customer_name in the search, and it will include the value. Splunk is really smart that way.

As for the IP address, try this regex instead:

 | rex field=_raw "(?<ipaddress>(\d{1,3}\.){3}\d{1,3})" |stats COUNT by ipaddress, customer_name

The regex looks for 1-3 digits followed by a '.' 3 times, followed by 1-3 digits, and assigns it to a newly created variable called ipaddress.

[mosman_splunk]

in order to help you with the regex you should have provided more than 1 sample

but this might give you a guidance. it might need tweak in your environment

sourcetype=xyz |rex &customer_name=(?([\S]+))&view  |rex START:\s(?(\d+.\d+.\d+.\d+))

then you will find 2 interesting port on your left side (interesting fields) by names customer_name & IP

good luck

[maddy1011]

Tried this, did not work, get the below error. (Just tried the rex for IP)

Error in 'rex' command: Encountered the following error while compiling the regex 'START:\s(?(\d+.\d+.\d+.\d+))': Regex: assertion expected after (?(

I think I just need the rex for extracting the list for IP, customer_name is an actual field so I could just use that directly.

[maddy1011]

This seems to work, but still working on removing field extraction

 sourcetype=xyz |rex START:\s(?(\d+.\d+.\d+.\d+))| stats count by IP_add, app|sort -count

Thanks!

[frobinson_splun]

Hi @maddy1011,
Have you tried using the Field Extractor? Here is some documentation that might help:
http://docs.splunk.com/Documentation/Splunk/6.4.0/Knowledge/ExtractfieldsinteractivelywithIFX

We also have some scenario-based documentation that includes a step using the Field Extractor, in case it is helpful.

Full scenario docs are here:
http://docs.splunk.com/Documentation/Splunk/6.4.0/Scenarios/Goals

And an example of using the Field Extractor for an IP address and username is mentioned here:
http://docs.splunk.com/Documentation/Splunk/6.4.0/Scenarios/Extractfields

Hope this helps!

[maddy1011]

I tried using extract field from the sample link. It did extract the IP as a field but just the sample one. Not in a way that would give me the list.
Also can you tell me how do I undo the extraction? all my search results are now defaulting to this extraction.

[frobinson_splun]

Hi @maddy1011,
This topic:
http://docs.splunk.com/Documentation/Splunk/6.4.1/Knowledge/Managesearch-timefieldextractions#Delete...

has information on deleting a field extraction.