Getting Data In

Can I use props.conf to parse logfile with some XML entries instead of manually?



i've been looking around a bit but it appears my google-fu isnt up to snuff for this problem.

i'm wondering how one can parse non-pure xml logs. as in we have a ".txt" file that timestamps, debug messages and all that jazz. but it also writes entire XML responses in the log file. i dont know if the source log is able to separate it out (doubt it) but the question goes as follows.

i'm aware there is a XML parser for the props.conf. but does that work on intermittent xml formatted log entries?

<date>,source,debug: event happened
<date>,source,debug: function executed
<date>,source,debug: send data to URI
<date>,source,debug: multiline XML response
<date>,source,log message

the xml response can sometimes be longer than 1000 lines which is where the issue arrives, as we get an error due to the log entry being more than 1000 lines long. it's all the xml tags <meta> </meta> <link> </link> and the like.

does props support something in the line of

  XML do XML parsing 
  do normal props things

or do I have to parse ell the XML response segments manually?

Labels (2)
0 Karma



I don't know if this is feasible for you but you should be able to rename a sourcetype based on regex (as described here

With this you could use the default xml sourcetype and the original sourcetype with the rest of your data.

I am not sure about the long xml thought.

0 Karma
Get Updates on the Splunk Community!

Build Scalable Security While Moving to Cloud - Guide From Clayton Homes

 Clayton Homes faced the increased challenge of strengthening their security posture as they went through ...

Mission Control | Explore the latest release of Splunk Mission Control (2.3)

We’re happy to announce the release of Mission Control 2.3 which includes several new and exciting features ...

Cloud Platform | Migrating your Splunk Cloud deployment to Python 3.7

Python 2.7, the last release of Python 2, reached End of Life back on January 1, 2020. As part of our larger ...