Search in XML file

[adityaanand]

Hi,
I am new in splunk world.
I have an XML file that contains following data.

 <TargetMachines>
         <TargetMachine Name="win7a2" IPAddress="10.167.177.30" Status="Running" >
              <Tasks>
                      <Task TaskSer="137" PackageName="Client Applications" PackageVersion="V13.5 (P1007499-002)"  Status="Fail">
                          <Steps>
                                 <Step ID="f2b56177-" Name="ARIA PM" Status="Pass" StepSer="4738" />
                                 <Step ID="46b4db06" Name="ARIA RO" Status="Fail" StepSer="4739">
                                 <Step ID="3de785d5-c6" Name="Deluxe Reports" Status="NotStarted" StepSer="4742" />
                                  ...........
                                  ...........
                           <Steps>
                      <Task>
                 <Tasks>
      </TargetMachine>
      <TargetMachine Name="win7a3" IPAddress="10.167.177.31" Status="Running" >
              <Tasks>
                      <Task TaskSer="138" PackageName="Client Applications" PackageVersion="V13.5 (P1007499-002)"  Status="Fail">
                          <Steps>
                                 <Step ID="f2b56174-" Name="ARIA PM" Status="Pass" StepSer="4656" />
                                 <Step ID="46b4db03" Name="ARIA RO" Status="Fail" StepSer="4657">
                                 <Step ID="3de785d5" Name="Deluxe Reports" Status="NotStarted" StepSer="4658" />
                                  ..................
                                 ..................
                           <Steps>
                      <Task>
                 <Tasks>
      </TargetMachine>
 </TargetMachines>

The file contains huge data as above .
I have broken the events using regular expression making changes in props.conf
BREAK_ONLY_BEFORE = > <Task Task

I am trying to find fail count of step by Step Name of a particular package, so that i can understand which step is being failed most.
Here package name is Client Applications
My search query is :

source="RSDReport.xml" host="PU4D9W0ND02" index="test" sourcetype="RSD_Log" | spath | search "Task{@PackageName}"="Client Applications" "Task.Steps.Step{@Status}"="Fail" |stats count(Task.Steps.Step{@StepSer}) by Task.Steps.Step{@Name}| where "Task.Steps.Step{@Status}"=="Fail"

Result: No results found.
But It is showing 9 events.

When i run following search query :

source="RSDReport.xml" host="PU4D9W0ND02" index="test" sourcetype="RSD_Log" | spath | search "Task{@PackageName}"="Client Applications" "Task.Steps.Step{@Status}"="Fail" |stats count(Task.Steps.Step{@StepSer})

It gives result - count(Task.Steps.Step{@StepSer}) =351

But when i run following search query :

source="RSDReport.xml" host="PU4D9W0ND02" index="test" sourcetype="RSD_Log" | spath | search "Task{@PackageName}"="Client Applications" "Task.Steps.Step{@Status}"="Fail" |stats count(Task.Steps.Step{@StepSer}) by Task.Steps.Step{@Name}

It gives result

   Task.Steps.Step{@Name}          count(Task.Steps.Step{@StepSer})
   ARIA Import Export NLS            351
    ARIA PM                            351
    ARIA PM NLS                        351
    ARIA RO                            351
   ARIA RO NLS                         351
   Application Frame NLS               351
   Application Framework              351
  Biological Optimization            351  
  .................
 .................

and more step name and its count

Please help me.

Thanks,
Aditya

[woodcock · Accepted solution]

This will do it:

source="RSDReport.xml" host="PU4D9W0ND02" index="test" sourcetype="RSD_Log" | spath | search "Task{@Status}"="Fail" | rex "Step ID=\"(?<FailedStepID>[^\"]*)\" Name=\"(?<FailedStepName>[^\"]*)\" Status=\"Fail\" StepSer=\"(?<FailedStepSer>[^\"]*)\"" | stats count by "Task{@PackageName}",FailedStepName

[adityaanand]

Thanks a lot for giving your precious time. Now it is working as expected.
There is one request, can you explain what have you done in regular expression. It will help me a lot.

rex "Step ID=\"(?<FailedStepID>[^\"]*)\" Name=\"(?<FailedStepName>[^\"]*)\" Status=\"Fail\" StepSer=\"(?<FailedStepSer>[^\"]*)\""

Thanks once again!!!

[woodcock]

The rex command uses standard PCRE with named capturing groups to create ad-hoc fields that are associated only with the search that you run. You can learn about PCRE in hundreds of places around the web.

[adityaanand]

Is regex is dependent on how do i break events?
Actually Earlier i broke the events on <Task> tag.
But now i broke the events on <TargetMachine> tag and used the same query as above. But this time output is not correct.
I found that there is two <Task> in <TargetMachine>...</TargetMachine> and both are failed. So which ever task found earlier, that included into result and other one is excluded .
It is my thinking that might be if splunk found a particular match in an event then it ignores the rest of part of that event.
Am I right ?

[woodcock]

Yes, rex will only run once against your event but you can create a field extraction with the same RegEx that will run more than once; read about it here (and search for mv_add😞

http://docs.splunk.com/Documentation/Splunk/latest/admin/Transformsconf