topic Re: How to extract from a value from XML and include it in the search? in Splunk Search

How to extract from a value from XML and include it in the search?

friscos — Tue, 16 Aug 2016 19:33:40 GMT

Hi,

I would like to extract the XML field value from an XML string from the log and include it in the search. What is the best way to do that?

Currently, whenever a request is posted, I am searching with the id, but I want to create a dynamic search such that whenever a new employee is added, I can see it in the Splunk search.

I tried the field extractor regex (^(?:[^>\n]*>){4}(?P\w+)), but not sure how I can use this regex in the search box.

Sample XML:

<employee>
    <id>TEST001</id>
</employee>

Re: How to extract from a value from XML and include it in the search?

somesoni2 — Tue, 16 Aug 2016 19:54:57 GMT

You could do like this

your base search | rex "(^(?:[^>\n]*>){4}(?P<employeeId>\w+))"

your base search | rex "\<id\>(?P<EmployeeId>[^\<]+)"

You can also save this regex in props.conf/field extraction to extract this automatically.

Re: How to extract from a value from XML and include it in the search?

friscos — Tue, 16 Aug 2016 20:29:20 GMT

Thanks Somesoni for your response.

my base search is this:

host="myserverhostname" rex "(^(?:[^>\n]*>){4}(?P<employee>\w+))" - This returns 'no results found'

I want to search for TEST001 in all the log files, how do i dynamically pass TEST001 in the search box?

I also tried this: host="myserverhostname" rex "\<id\>(?P<employee>[^\<]+)"

If i search with host="myserverhostname" | rex "(^(?:[^>\n]*>){4}(?P<employee>\w+))" , this returns all the logging and doesn't filter by id.

Re: How to extract from a value from XML and include it in the search?

somesoni2 — Tue, 16 Aug 2016 20:48:16 GMT

The above regex is only for extracting the id from the raw xml into a field. To filter the data using your filter TEST001, you can try any of these method

host="myserverhostname" "TEST001"

host="myserverhostname" | regex _raw=".*\<id\>TEST001.*"

Re: How to extract from a value from XML and include it in the search?

friscos — Wed, 17 Aug 2016 13:29:57 GMT

So, there is no way to extract the value of id and include it in the search field?

I do not want to put TEST001 in the search as the value changes every time a new id is created. I want to monitor the logs based on the value of id that gets created and written to the log.

Re: How to extract from a value from XML and include it in the search?

pasokkum — Wed, 17 Aug 2016 13:57:51 GMT

Create props.conf in $SplunkHome$/etc/system/local of both search head and indexer with the following stanza
[sourcetype]
KV_MODE=xml
After making the changes restart search heads and indexers

Re: How to extract from a value from XML and include it in the search?

friscos — Wed, 17 Aug 2016 18:09:59 GMT

I added KV_MODE=xml in the prop.xml and restarted the server.

How do i now extract the of value of an xml from log file and include it in the search?

Thanks

Re: How to extract from a value from XML and include it in the search?

somesoni2 — Wed, 17 Aug 2016 18:26:20 GMT

Give this a shot

host="myserverhostname" [ search host="myserverhostname" | rex "(^(?:[^>\n]*>){4}(?P<employee>\w+))" | stats count by employee | table employee | rename employee as search ]

Re: How to extract from a value from XML and include it in the search?

pasokkum — Thu, 18 Aug 2016 06:32:15 GMT

your base search | fields employee.id

Re: How to extract from a value from XML and include it in the search?

friscos — Thu, 18 Aug 2016 18:07:16 GMT

no luck..doesn't fetch anything. thanks for your time.

Re: How to extract from a value from XML and include it in the search?

friscos — Thu, 18 Aug 2016 18:11:09 GMT

host=myhostname | fields employee.id returns no results unfortunately.

Re: How to extract from a value from XML and include it in the search?

pasokkum — Fri, 19 Aug 2016 05:54:01 GMT

on running host=myhostname sourcetype=mysourcetype.. are you able to see the interesting fields in the left side of the pane in search head?

Re: How to extract from a value from XML and include it in the search?

friscos — Fri, 19 Aug 2016 12:54:53 GMT

Yes, I see the selected fields and interesting fields. I did try extract the id field from the event xml (using regex) but not able to include that regex in the search.

Re: How to extract from a value from XML and include it in the search?

pasokkum — Mon, 22 Aug 2016 06:40:11 GMT

are you loading .xml files to indexer?