topic Re: how to discard data before a timestamp? in Getting Data In

how to discard data before a timestamp?

wojtek_swiatek — Thu, 03 Jul 2014 15:28:57 GMT

Hello

I get some data (TCP syslog) from a Juniper VPN server. Unfortunately they are incorrectly parsed by splunk because some data is prepended before the timestamp. A sample of the data (this is all in one block) is below, I have replaced by INFOINFOINFO the actual data from systlog (which varies):

197 <134>Juniper: 2014-07-03 17:11:02 - INFOINFONFO234 <134>Juniper: 2014-07-03 17:11:02 - INFOINFOINFO194 <134>Juniper: 2014-07-03 17:11:02 - INFOINFOINFO

How can I tell splunk to discard (first example) the string 197 <134>Juniper: as the actual syslog data is afterwards?

Thank you!

Re: how to discard data before a timestamp?

richgalloway — Thu, 03 Jul 2014 15:51:42 GMT

Modify your props.conf file and add a 'TIME_PREFIX = Juniper:' statement to the appropriate stanza.

Re: how to discard data before a timestamp?

wojtek_swiatek — Fri, 04 Jul 2014 06:26:51 GMT

Thank you - I added
[source::tcp:25890] TIME_PREFIX = Juniper:
(two lines) to /opt/splunk/etc/system/local/props.conf but I still get the same chunks (I tested this by sending the sting in my question via netcat splunk.example.com 25890). Is there a way to debug or would you see another place I should add something?

Re: how to discard data before a timestamp?

wojtek_swiatek — Fri, 04 Jul 2014 06:37:11 GMT

After some more checks I see that:
- a single line starting with "Juniper:" is indexed properly (the Juniper: prefix is not removed, though. But the indexed times are OK)
- when several lines are concatenated they are not broken into single ones (all start with Juniper:)