Field Extraction issue

[rajgowd1]

HI Experts,
i am able to exact 4th and 5th fields from below log but i am able to exact get the value if the 4th or 5th filed is HOSTNAME but if it is IPaddress then i am not able retrieve.here is the sample log

tcp 0 0 10.40.88.178:7171 10.40.88.175:50326 ESTABLISHED
tcp 0 0 12b8-splfwd02.ndm.nsro:7171 10.40.88.170:50326 TIME_WAIT
tcp 0 0 10.40.88.178:7171 poc-card-luna2.nad.ns:50326 TIME_WAIT
tcp 0 0 12b8-splfwd02.ndm.nsro:7171 10.30.88.145:50326 ESTABLISHED
tcp 0 0 10.40.88.178:7171 poc-card-luna4.nad.ns:50326 SYNC_SENT
tcp 0 0 12b8-splfwd03.ndm.nsro:7171 poc-card-luna1.nad.ns:46756 TIME_WAIT
tcp 0 0 12b8-splfwd03.ndm.nsro:7171 10.30.88.117:46756 SYNC_SENT
tcp 0 0 10.40.88.178:7171 poc-card-luna2.nad.ns:46756 TIME_WAIT
tcp 0 0 12b8-splfwd03.ndm.nsro:7171 10.40.83.157:46756 TIME_WAIT
tcp 0 0 12b8-splfwd03.ndm.nsro:7171 poc-card-luna4.nad.ns:46756 ESTABLISHED
tcp 0 0 12b8-splfwd04.ndm.nsro:7171 poc-card-luna1.nad.ns:46756 TIME_WAIT
tcp 0 0 10.40.88.178:7171 poc-card-luna3.nad.ns:46756 SYNC_SENT
tcp 0 0 12b8-splfwd04.ndm.nsro:7171 poc-card-luna2.nad.ns:46756 SYNC_SENT
tcp 0 0 12b8-splfwd04.ndm.nsro:7171 poc-card-luna4.nad.ns:46756 ESTABLISHED
tcp 0 0 12b8-splfwd04.ndm.nsro:7171 poc-card-luna2.nad.ns:46756 ESTABLISHED

[richgalloway]

If you're using the Splunk UI you're probably using the automatic field extractor which is most likely producing a regex string that is too specific. Try this regex, instead. It skips the first three space-delimited fields and extracts the next two space-delimited fields as 'src' and 'dest', respectively.

(?:[^\s]+\s){3}(?P<src>[^\s]+)\s(?<dest>[^\s]+)

Perhaps you want the 4th and 5th fields to be the host name/IP address and port. That regex string is

(?:[^ ]+ ){3}(?P<src>[^:]+):(?<dest>\d+)

[govindsinghrawa · Accepted solution]

Below Regex can be used in field extractor in write your own regular expression syntax:

^(?< protocol>[\S]+)\s*?(?< field1>[\S]+)\s*?(?< field2>[\S]+)\s*?(?< hostOrIp1>[^:]+):(?< hostOrIp1Port>[\S]+)\s*?(?< hostOrIp2>[^:]+):(?< hostOrIp2Port>[\S]+)\s.*?(?< state>[\S]+)

During search time this regex can also be used with rex to extract fields if fields are not already extracted:

yourBaseSearch

|rex field=_raw "^(?< protocol>[\S]+)\s*?(?< field1>[\S]+)\s*?(?< field2>[\S]+)\s*?(?< hostOrIp1>[^:]+):(?< hostOrIp1Port>[\S]+)\s*?(?< hostOrIp2>[^:]+):(?< hostOrIp2Port>[\S]+)\s.*?(?< state>[\S]+)"

| table hostOrIp1, hostOrIp2

NOTE: Please remove the space in each of the tags, example "< protocol>", "< field1>" etc.

[rajgowd1]

Thank you govindsinghrawat.i am trying to create visualization by using all your experts answers.
is there way to create visualization by using similar search

both are giving same results

index=netstat|rex field=_raw "^(?[\S]+)\s*?(?[\S]+)\s*?(?[\S]+)\s*?(?[^:]+):(?[\S]+)\s*?(?[^:]+):(?[\S]+)\s.*?(?[\S]+)"
| eventstats count by state hostOrIp1 hostOrIp2 |eval CountConnStatus=count+" - "+state|chart values(CountConnStatus) by hostOrIp1 hostOrIp2

index=netstat |rex "(?[a-zA-Z0-9][^\s]+)\s(?[^\s]+)\s(?.*)" |eventstats count by CONN_STATUS CBOL_SERVER HSM |eval CountConnStatus=count+" - "+CONN_STATUS|chart values(CountConnStatus) by CBOL_SERVER HSM

[govindsinghrawa]

Hi @rajgowd

The above regex which I gave is to extract fields. Once the fields are extracted then you can use them i your commands.

Now coming to visulaizations. Visualizations can only be created depending on how you end your search, for example :

if you end your search with timechart command then you will immediately have visualization options for line, bar etch. Example:
yourBaseSearch | timechart count by yourField

If you have statistics being generated as aggregations like min, max, sum etc you can plot them using timechart, chart etc.

[rajgowd1]

Hi Rich,thanks for your response.
From logs,I am trying to extract using splunk UI.
I think I am able extract multiple fields.

[rajgowd1]

Hi Rich,thanks for your response.
From logs,I am trying to extract using splunk UI.
I think I am able extract multiple fields.

[sundareshr]

You have few options.

1) Edit props/transforms per instructions in the blog post. Your data appears to be space delimited, so you will specific " " in your transforms instead of ","

http://blogs.splunk.com/2013/03/11/quick-n-dirty-delimited-data-sourcetypes-and-you/

2) The other option is to use Interactive Field Extractor (IFX) to extract the fields as space delim. IFX will generate the similar props/transforms edits for the extractions.

3) Use regex in your SPL like this

base search | rex "(?<f1>[^\s]+)\s(?<f2>[^\s]+)\s(?<f3>[^\s]+)\s(?<f4>[^\s]+)\s(?<f5>[^\s]+)\s(?<f6>[^\s]+)" | table f*

[richgalloway]

How are you extracting the fields? Is this a search-time or index-time extraction?