Splunk Search

How to extract from a value from XML and include it in the search?

friscos
Explorer

Hi,

I would like to extract the XML field value from an XML string from the log and include it in the search. What is the best way to do that?

Currently, whenever a request is posted, I am searching with the id, but I want to create a dynamic search such that whenever a new employee is added, I can see it in the Splunk search.

I tried the field extractor regex (^(?:[^>\n]*>){4}(?P\w+)), but not sure how I can use this regex in the search box.

Sample XML:

<employee>
    <id>TEST001</id>
</employee>
0 Karma

pasokkum
Path Finder

Create props.conf in $SplunkHome$/etc/system/local of both search head and indexer with the following stanza
[sourcetype]
KV_MODE=xml
After making the changes restart search heads and indexers

0 Karma

friscos
Explorer

I added KV_MODE=xml in the prop.xml and restarted the server.

How do i now extract the of value of an xml from log file and include it in the search?

Thanks

0 Karma

pasokkum
Path Finder

your base search | fields employee.id

0 Karma

friscos
Explorer

host=myhostname | fields employee.id returns no results unfortunately.

0 Karma

pasokkum
Path Finder

on running host=myhostname sourcetype=mysourcetype.. are you able to see the interesting fields in the left side of the pane in search head?

0 Karma

friscos
Explorer

Yes, I see the selected fields and interesting fields. I did try extract the id field from the event xml (using regex) but not able to include that regex in the search.

0 Karma

pasokkum
Path Finder

are you loading .xml files to indexer?

0 Karma

somesoni2
Revered Legend

You could do like this

your base search | rex "(^(?:[^>\n]*>){4}(?P<employeeId>\w+))"

OR

your base search | rex "\<id\>(?P<EmployeeId>[^\<]+)"

You can also save this regex in props.conf/field extraction to extract this automatically.

friscos
Explorer

Thanks Somesoni for your response.

my base search is this:

host="myserverhostname" rex "(^(?:[^>\n]*>){4}(?P<employee>\w+))" - This returns 'no results found'

I want to search for TEST001 in all the log files, how do i dynamically pass TEST001 in the search box?

I also tried this: host="myserverhostname" rex "\<id\>(?P<employee>[^\<]+)"

If i search with host="myserverhostname" | rex "(^(?:[^>\n]*>){4}(?P<employee>\w+))" , this returns all the logging and doesn't filter by id.

0 Karma

somesoni2
Revered Legend

The above regex is only for extracting the id from the raw xml into a field. To filter the data using your filter TEST001, you can try any of these method

host="myserverhostname" "TEST001"

OR

host="myserverhostname" | regex _raw=".*\<id\>TEST001.*"
0 Karma

friscos
Explorer

So, there is no way to extract the value of id and include it in the search field?

I do not want to put TEST001 in the search as the value changes every time a new id is created. I want to monitor the logs based on the value of id that gets created and written to the log.

0 Karma

somesoni2
Revered Legend

Give this a shot

host="myserverhostname" [ search host="myserverhostname" | rex "(^(?:[^>\n]*>){4}(?P<employee>\w+))" | stats count by employee | table employee | rename employee as search ] 
0 Karma

friscos
Explorer

no luck..doesn't fetch anything. thanks for your time.

0 Karma
Get Updates on the Splunk Community!

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud  In today’s fast-paced digital ...

Observability protocols to know about

Observability protocols define the specifications or formats for collecting, encoding, transporting, and ...

Take Your Breath Away with Splunk Risk-Based Alerting (RBA)

WATCH NOW!The Splunk Guide to Risk-Based Alerting is here to empower your SOC like never before. Join Haylee ...