Hi,
I'm trying to perform some field extractions in a log containing availability reports of segments in our system similar to the one below:
<SSEL><ET>2011-07-02T12:11:29.676Z</ET><LSID>12000</LSID><SEID>1007</SEID><S>OUT-OF-SERVICE</S><LSID>12000</LSID><SEID>1010</SEID><S>OUT-OF-SERVICE</S></SSEL>
etc. followed by from 10 up to 200 combinations of <LSID><SEID><S> tags and ending in </SSEL>
The syntax of the logline is like:
<DateTime><Logical ID><Segment ID><Status>....<Logical ID><Segment ID><Status>
In human language: per logline we have 1 datetime and multiple status reports for parts of the system identified by <Logical ID><Segment ID> .
I'm looking to extract the combinations of LSID, SEID and S and have each combination extracted with the EventTime ( ) that is present at the start
of the logline. The goal is to create an overview of the status (S) for combinations of LSID,SEID during the day.
I've tried multi-value extraction using props.conf and transforms.conf like below but that fails for 2 reasons:
- the time is only found once in the logline
- the relation between LSID, SEID and S is broken as Splunk extracts all LSID's in the logline individually. Same for all SEID's and all S's.
My props.conf and transforms.conf:
[OL]
pulldown_type = 1
REPORT-r1 = ol_logfile
[ol_logfile]
REGEX=(\d{4}-\d{2}-\d{2})T(\d{2}:\d{2}:\d{2}\.\d{3})(.*?)<LSID>(\d*?)\</LSID>.*?<SID>(\d*?)\</SID>.*?<S>(.*?)\</S>
FORMAT=date::$1,time::$2,garbage::$3,LSID::$4,SEID::$5,S::$6
MV_ADD=true
I'm thinking now of writing a Python preprocessor to deliver the logline to Splunk like this:
<ET>2011-07-02T12:11:29.676Z</ET><LSID>12000</LSID><SEID>1007</SEID><S>OUT-OF-SERVICE</S>
<ET>2011-07-02T12:11:29.676Z</ET><LSID>12000</LSID><SEID>1010</SEID><S>OUT-OF-SERVICE</S>
....
But I'd rather use built-in Splunk routines as the preprocessor adds complexity and maintenance.
What would be the most Splunkish way to import the data in Splunk for my report? I'm not happy with the preprocessor, but I also don't see how to extract
the multivalue part.
Any hints are greatly appreciated, thanks!
... View more
