topic Re: How to extract keywords and associated values with rex from raw logs and display data in a table? in Splunk Search

How to extract keywords and associated values with rex from raw logs and display data in a table?

satya2p — Thu, 19 Feb 2015 16:28:39 GMT

I have a request input output logged by various sourcetypes in XML and other similar below format. I tried multiple options to extract exact keywords and associated values to display in a table, but I have been unable to do so. Please help.

Sample Data:

Somecontent.ssn=XXXXXXXXX.Somecontent.SomeOthercontent
in XML: <SSN> XXXXXXXXX </SSN>

Options Tried:

| rex field=_raw "SSN=(?P\d\d\d-?\d\d-?\d\d\d\d)" | stats count by SSN
| rex field=_raw=\W+SSN "(?\w+)" | stats count by SSN
| rex field=_raw "SSN\=(?\d{9})+"| stats count by SSN

Re: How to extract keywords and associated values with rex from raw logs and display data in a table?

richgalloway — Thu, 19 Feb 2015 18:29:46 GMT

Your rex commands are not extracting fields. Therefore, there is nothing for downstream commands to work with. Try this:

... | rex "SSN\>\s*(?P<SSN>\d{9})" | stats count by SSN

The first "SSN" is an eyecatcher to help rex find the right data in your XML. The second "SSN" is a field name which can be used by the stats command.

Re: How to extract keywords and associated values with rex from raw logs and display data in a table?

satya2p — Thu, 19 Feb 2015 19:02:35 GMT

Thank you.. I tried, its just reading all SSN keyword(considering false positive) not extracting the values and listing out.

Re: How to extract keywords and associated values with rex from raw logs and display data in a table?

richgalloway — Thu, 19 Feb 2015 19:14:06 GMT

Let's try something simpler. This should display all the SSN values found.

... | rex "SSN\>\s*(?P<SSN>\d{9})" | table SSN

Re: How to extract keywords and associated values with rex from raw logs and display data in a table?

satya2p — Thu, 19 Feb 2015 19:20:52 GMT

I tried this option earlier, table getting generated for each event logged by keyword but data is not populating. seems its unable to extract values. I am using splunk 5.0.9, is xml filed extraction is available in this release.

Re: How to extract keywords and associated values with rex from raw logs and display data in a table?

richgalloway — Thu, 19 Feb 2015 19:23:52 GMT

Yes, field extraction using the rex command is the same in version 5. Is the sample data in your OP accurate? Regex strings can be very sensitive to differences in white space, case, etc.