How to read and filter (complete) XML hierarchy?

[bartzr]

We are currently in the evaluation phase of splunk.
Dependent on the results of a feasibility study licenses will be purchased or not.

There is one important use case that we are currently not able to get to work.
We are having a hierarchical XML-File.

Simplified example:

<hierarchy1>
  <param1>1</param1>
  <param2>2</param2>
  <hierarchy2>
    <param3>3</param3>
    <param4>4</param4>
      <hierarchy3>
        <param5>5</param5>
        <param6>6</param6>
      </hierarchy3>
      <hierarchy3>
        <param5>7</param5>
        <param6>8</param6>
      </hierarchy3>
      <hierarchy3>
        <param5>9</param5>
        <param6>10</param6>
      </hierarchy3>
  </hierarchy2>
</hierarchy1>

We need to read the complete hierarchy and be able to select/filter all params:

Example query: give me all params (param1, param2, param3, param4, param5, param6) for param1=1, param5=7 OR param5=5.

Ideally the query is supported by indexes so that the magic does not have to happen during query run time.
Can this be achieved with splunk?
If yes, how?

Thanks in advance
Ronny

[gregbo]

Did you ever get an answer to this? I'm having a similar problem.

[somesoni2]

You would need to setup the sourcetype definition to parse the data as XML. See this for more details

https://answers.splunk.com/answers/187195/how-to-add-and-parse-xml-data-in-splunk.html

For how it will look in Splunk upon ingestion, run this search from evaluation splunk instance.

| gentimes start=-1 | eval _raw=" <hierarchy1>
   <param1>1</param1>
   <param2>2</param2>
   <hierarchy2>
     <param3>3</param3>
     <param4>4</param4>
       <hierarchy3>
         <param5>5</param5>
         <param6>6</param6>
       </hierarchy3>
       <hierarchy3>
         <param5>7</param5>
         <param6>8</param6>
       </hierarchy3>
       <hierarchy3>
         <param5>9</param5>
         <param6>10</param6>
       </hierarchy3>
   </hierarchy2>
 </hierarchy1>" | table _raw | spath

[bartzr]

We followed up on your second proposal.
In order to better show what we need we extended the example slightly to:

| gentimes start=-1 | eval _raw=" <hierarchy1>
        <param1>1</param1>
        <param2>2</param2>
        <hierarchy2>
          <param3>3</param3>
          <param4>4</param4>
            <hierarchy3>
              <param5>5</param5>
              <param6>6</param6>
            </hierarchy3>
            <hierarchy3>
              <param5>7</param5>
              <param6>8</param6>
            </hierarchy3>
            <hierarchy3>
              <param5>9</param5>
              <param6>10</param6>
            </hierarchy3>
        </hierarchy2>
        <hierarchy2>
          <param3>a</param3>
          <param4>b</param4>
            <hierarchy3>
              <param5>c</param5>
              <param6>d</param6>
            </hierarchy3>
            <hierarchy3>
              <param5>e</param5>
              <param6>f</param6>
            </hierarchy3>
            <hierarchy3>
              <param5>g</param5>
              <param6>h</param6>
            </hierarchy3>
        </hierarchy2>
      </hierarchy1>" | table _raw | spath

When invoking this search string we get a table that contains all the single values.
However the relation between the hierarchies seems to be lost.

It seems not to be possible to link
hierarchy1.hierarchy2.hierarchy3.param5 with value 'g' to hierarchy1.hierarchy2.param3 with value 'a'.

This actually is the issue that we are struggling with.
Maybe just a small issue.

How would a query look like that asks for hierarchy1.hierarchy2.hierarchy3.param5 = 'g' where the result includes all metadata (the information provided in the hierarchy levels above) and exlucing elements with hierarchy1.hierarchy2.hierarchy3.param5 != 'g'

Thanks
Ronny

[bartzr]

Thanks for your really fast response!

The example that you referred to was working for me except for the following portion:
How would a search string look like that provides me all entries with comment="Happy birthday" without showing the data for comment="Good pic!"?

result without filter
photo_id, title , format , owner_id , owner , comment
"123", "Birthday", "jpg", "1111", "Jason", "Good pic!"
"123", "Birthday", "jpg", "1111", "Jason", "Happy birthday"

result with the filter applied as described above
photo_id, title , format , owner_id , owner , comment
"123", "Birthday", "jpg", "1111", "Jason", "Happy birthday"

--> one line less, all "meta-data" available

When I understand correctly by applying the sourcetype configuration as referred by your link I get one "photo-event" per photo tag.
Splunk then allows me to filter on event base.
So when I filter on comment = "Happy birthday" I will get all event data with that comment. Unfortunately this includes the comment "Good pic!" as well.
When I create smaller chunks, lets say one event per comment I have the comments separated, can properly filter on comment content but loose all meta data (like photo_id, owner and so on).

Did I make my point comprehensible?
If not let me know and I'll try to rephrase.

Thanks
Ronny