We have data as XML documents. How can we index each XML document as one Splunk event?
A sample -
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<mlcpMetricsModel xmlns="http://xxxxxxx">
<canonical>xxxxxx</canonical>
<duration>PT41M3.401S</duration>
<env>PROD</env>
<ecmProcDateTime>20170727_M</ecmProcDateTime>
<outputRecords>4948262</outputRecords>
<outputRecordsCommitted>4948262</outputRecordsCommitted>
<outputRecordsFailed>0</outputRecordsFailed>
<reportDate>2017-08-02T10:49:31</reportDate>
<source>CDB</source>
<startTime>2017-08-02T10:08:18.512</startTime>
</mlcpMetricsModel>
Try this
[yoursourcetype]
LINE_BREAKER = ([\r\n]+)(?=\<mlcpMetricsModel )
SHOULD_LINEMERGE = false
TIME_PREFIX = \<reportDate\>
TIME_FORMAT = %Y-%m-%dT%H:%M:%S
MAX_TIMESTAMP_LOOKAHEAD = 19
Try this
[yoursourcetype]
LINE_BREAKER = ([\r\n]+)(?=\<mlcpMetricsModel )
SHOULD_LINEMERGE = false
TIME_PREFIX = \<reportDate\>
TIME_FORMAT = %Y-%m-%dT%H:%M:%S
MAX_TIMESTAMP_LOOKAHEAD = 19
Now that I'm preparing for the admin certification, I wonder why @somesoni2 set MAX_TIMESTAMP_LOOKAHEAD = 19
, which obviously works but based on the data it appears that the value should be much higher.
The timestamp is extracted from <reportDate>
tag which is 2017-08-02T10:49:31
, 19 character long value.
got it ; -) so, if TIME_PREFIX
exists it starts from there, otherwise, from the beginning of the line.
(thumbs up)
Just in case it's still confusing for anyone, it's the length of timestamp string represented by TIME_FORMAT string. (%Y-%m-%dT%H:%M:%S => %Y(4)-(1)%m(2)-(1)%d(2)T(1)%H(2):(1)%M(2):(1)%S(2) => 4+1+2+1+2+1+2+1+2+1+2 =19
)
@somesoni2, I think \<startTime\>
is a better candidate for TimeStamp. However, @ddrillic must confirm.
Thank you both - let me check...
Gorgeous - thank you !! we ended up extracting the XML fields using a series such as - spath mlcpMetricsModel.env
...