How to extract from a value from XML and include it in the search?

[friscos]

Hi,

I would like to extract the XML field value from an XML string from the log and include it in the search. What is the best way to do that?

Currently, whenever a request is posted, I am searching with the id, but I want to create a dynamic search such that whenever a new employee is added, I can see it in the Splunk search.

I tried the field extractor regex (^(?:[^>\n]*>){4}(?P\w+)), but not sure how I can use this regex in the search box.

Sample XML:

<employee>
    <id>TEST001</id>
</employee>

[pasokkum]

Create props.conf in $SplunkHome$/etc/system/local of both search head and indexer with the following stanza
[sourcetype]
KV_MODE=xml
After making the changes restart search heads and indexers

[friscos]

I added KV_MODE=xml in the prop.xml and restarted the server.

How do i now extract the of value of an xml from log file and include it in the search?

Thanks

[pasokkum]

your base search | fields employee.id

[friscos]

host=myhostname | fields employee.id returns no results unfortunately.

[pasokkum]

on running host=myhostname sourcetype=mysourcetype.. are you able to see the interesting fields in the left side of the pane in search head?

[friscos]

Yes, I see the selected fields and interesting fields. I did try extract the id field from the event xml (using regex) but not able to include that regex in the search.

[pasokkum]

are you loading .xml files to indexer?

[somesoni2]

You could do like this

your base search | rex "(^(?:[^>\n]*>){4}(?P<employeeId>\w+))"

OR

your base search | rex "\<id\>(?P<EmployeeId>[^\<]+)"

You can also save this regex in props.conf/field extraction to extract this automatically.

[friscos]

Thanks Somesoni for your response.

my base search is this:

host="myserverhostname" rex "(^(?:[^>\n]*>){4}(?P<employee>\w+))" - This returns 'no results found'

I want to search for TEST001 in all the log files, how do i dynamically pass TEST001 in the search box?

I also tried this: host="myserverhostname" rex "\<id\>(?P<employee>[^\<]+)"

If i search with host="myserverhostname" | rex "(^(?:[^>\n]*>){4}(?P<employee>\w+))" , this returns all the logging and doesn't filter by id.

[somesoni2]

The above regex is only for extracting the id from the raw xml into a field. To filter the data using your filter TEST001, you can try any of these method

host="myserverhostname" "TEST001"

OR

host="myserverhostname" | regex _raw=".*\<id\>TEST001.*"

[friscos]

So, there is no way to extract the value of id and include it in the search field?

I do not want to put TEST001 in the search as the value changes every time a new id is created. I want to monitor the logs based on the value of id that gets created and written to the log.

[somesoni2]

Give this a shot

host="myserverhostname" [ search host="myserverhostname" | rex "(^(?:[^>\n]*>){4}(?P<employee>\w+))" | stats count by employee | table employee | rename employee as search ] 

[friscos]

no luck..doesn't fetch anything. thanks for your time.