Hi,
I have a feed where it appears that multiple events are being sent on the same line, and I need to break them out. Can someone help me out? The break looks like it should be after the Timestamp, these are all appearing on the same line.
Mar 16 18:44:29 myhostname NTNAME=-SM-XNlwpU2OKY6Z5O9%2fdn3dUlqVlktrStAYeZaWtlO2dtNaD%2f45RYB9%2bb&TARGET=-SM-http%3a%2f%2fshare1%2eapp%2ey%2ecom%2fshare%2fdb%2fGet%2fDocument--2008231%2f s-port=443 cs-username=- c-ip=x.x.x.x cs(User-Agent)=Mozilla/5.0+(Windows+NT+6.1;+Win64;+x64;+rv:58.0)+Gecko/20100101+Firefox/58.0 sc-status=200 sc-substatus=0 sc-win32-status=0 time-taken=62 <13>Mar 16 18:44:29 myhostname AgentDevice=MSIIS AgentLogFile=u_ex180316 AgentLogFormat=W3C AgentLogProtocol=W3C date=2018-03-16 time=18:44:23 s-ip=x.x.x.x cs-method=GET cs-uri-stem=/siteminder/ntlm/creds.ntc cs-uri-query=CHALLENGE=&SMAGENTNAME=-SM-mqLlVsTdZPBzRsxcdlQAR2H4%2bN7cc%2ffYvhe2fzHoB1PFt9kRHccU3f3f0&TARGET=-SM-HTTP%3a%2f%2frtn%2eray%2ezebm%2f s-port=443 cs-username=- c-ipx.x.x.x cs(User-Agent)=Mozilla/5.0+(Windows+NT+6.1;+WOW64;+Trident/7.0;+rv:11.0)+like+Gecko sc-status=401 sc-substatus=2 sc-win32-status=5 time-taken=31 <13>Mar 16 18:44:29 myhostname AgentDevice=MSIIS AgentLogFile=u_ex180316 AgentLogFormat=W3C AgentLogProtocol=W3C date=2018-03-16 time=18:44:23 s-ip=x.x.x.x cs-method=GET cs-uri-stem=/sntnder/ntlm/creds.ntc cs-uri-query=CHALLENGE=&SMAGENTNAME=-SM-mqLlVsTdZPBzRKcdlQAR2H4%2bNqhz7cc%2ffYvhe2fzHoB1PFt9kRHccU3f3f0&TARGET=-SM-HTTP%3a%2f%2fozztn%2fzaay%2gcom%2f s-port=443 cs-username=- c-ip=x.x.x.x cs(User-Agent)=Mozilla/5.0+(Windows+NT+6.1;+WOW64;+Trident/7.0;+rv:11.0)+like+Gecko sc-status=401 sc-substatus=1 sc-win32-status=2148074254 time-taken=31
What's that <13>
all about? Is that actually in the data? If so, could this be as easy as breaking there with your LINE_BREAKER
?
<13> looks like a syslog priority tag to me (encoding user.notice).
How is this data coming in to your forwarder?
Anyway, I guess you could indeed break on that <\d+>, or to make it a little safer for <...> stuff to occur elsewhere in the event, include part of the timestamp format as well.
Give this a try
[yourSourceType]
SHOULD_LINEMERGE = false
LINE_BREAKER = (\<\d+\>)
...other timestamp settings...