Splunk Dev

Output XML via a custom search command

mw
Splunk Employee
Splunk Employee

I'm working on a custom search command which will take the results of a search and create an XML output file. As a very simplified example, the search might look like this:

source=a OR source=b | fields host, source, some_field | outputxml

Within my search command, I read the results and aggregate all of the stuff into Python dicts (e.g. source[type]['total'] += 1, source[type][value] += 1, etc), and then attempt to write the results to a randomly named output file, where the XML would look something like:

<xml>
  <source type="syslog" total="2">
    <some_field value="1" count="1"/>
    <some_field value="0" count="1"/>
  </source>
  <source type="dhcp" total="1">
    <some_field value="1" count="1"/>
  <source>
</xml>

However, I suppose due to map/reduce maybe, multiple output files are created with the results being spread among them. At least, I suppose that it would make sense to be a function of map/reduce, and actually rather cool to see in action.

Is my analysis correct? If so, what is the best practice for handling this merging of results into a single, highly structured output file where order matters?

Tags (1)
0 Karma
1 Solution

gkanapathy
Splunk Employee
Splunk Employee

I would suggest that it might be easier to get what you want by calling the Splunk API:

wget --no-check-certificate --user=admin --password=changeme -O - --post-data='search sourcetype%3Dmysourcetype | head 2&exec_mode=oneshot' https://localhost:8089/services/search/jobs

There is also generally no need for you to worry about map-reduce. Splunk will take care of that. (It's possible to write map-reduceable search commands if you specify them as streaming, but converting CSV to XML and attempting to merge them in the reduce step is not an operation that will gain from what Splunk already does with the results.) So you can just worry about convert the CSV input to XML on a single node.

View solution in original post

gkanapathy
Splunk Employee
Splunk Employee

I would suggest that it might be easier to get what you want by calling the Splunk API:

wget --no-check-certificate --user=admin --password=changeme -O - --post-data='search sourcetype%3Dmysourcetype | head 2&exec_mode=oneshot' https://localhost:8089/services/search/jobs

There is also generally no need for you to worry about map-reduce. Splunk will take care of that. (It's possible to write map-reduceable search commands if you specify them as streaming, but converting CSV to XML and attempting to merge them in the reduce step is not an operation that will gain from what Splunk already does with the results.) So you can just worry about convert the CSV input to XML on a single node.

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...