https://community.splunk.com/t5/Getting-Data-In/How-to-merge-messages-into-a-single-event-using-one-timestamp/m-p/231922#M45186 <P>Hi, </P> <P>Relative newbie here. I have a host that is splitting large messages into multiple syslog messages. The beginning of the message contains two timestamps, but the trailing messages only contain one. The messages arrive like this:</P> <PRE><CODE>May 5 08:12:16 myhost.company.com 05/05/16 08:12:16.392 [part1] May 5 08:12:16 myhost.company.com [part2] May 5 08:12:16 myhost.company.com [part3] </CODE></PRE> <P>Miraculously, I somehow managed to get the messages combined into a single by editing the props.conf:</P> <PRE><CODE>NO_BINARY_CHECK = true MAX_EVENTS=50000 SHOULD_LINEMERGE = false MAX_TIMESTAMP_LOOKAHEAD = 25 TIME_PREFIX = ^ TRUNCATE = 0 LINE_BREAKER = ([\r\n]+)\w+\s+\d+\s+\d+:\d+:\d+\s+\w+\.\w+\.\w+\.\w+\s\d+\/\d+\/\d+\s+\d+:\d+:\d+\.\d+ </CODE></PRE> <P>And this seems to work fine, but what I noticed is that the timestamps from the two trailing messages are also included in the combined event, kind of like this:</P> <PRE><CODE>May 5 08:12:16 myhost.company.com 05/05/16 08:12:16.392 [part1]May 5 08:12:16 myhost.company.com [part2]May 5 08:12:16 myhost.company.com [part3] </CODE></PRE> <P>Is there a way to break the events based on the second timestamp in the first message, but strip out the timestamps from the following two messages?</P> Thu, 05 May 2016 13:35:22 GMT _smp_ 2016-05-05T13:35:22Z https://community.splunk.com/t5/Getting-Data-In/How-to-merge-messages-into-a-single-event-using-one-timestamp/m-p/231922#M45186 <P>Hi, </P> <P>Relative newbie here. I have a host that is splitting large messages into multiple syslog messages. The beginning of the message contains two timestamps, but the trailing messages only contain one. The messages arrive like this:</P> <PRE><CODE>May 5 08:12:16 myhost.company.com 05/05/16 08:12:16.392 [part1] May 5 08:12:16 myhost.company.com [part2] May 5 08:12:16 myhost.company.com [part3] </CODE></PRE> <P>Miraculously, I somehow managed to get the messages combined into a single by editing the props.conf:</P> <PRE><CODE>NO_BINARY_CHECK = true MAX_EVENTS=50000 SHOULD_LINEMERGE = false MAX_TIMESTAMP_LOOKAHEAD = 25 TIME_PREFIX = ^ TRUNCATE = 0 LINE_BREAKER = ([\r\n]+)\w+\s+\d+\s+\d+:\d+:\d+\s+\w+\.\w+\.\w+\.\w+\s\d+\/\d+\/\d+\s+\d+:\d+:\d+\.\d+ </CODE></PRE> <P>And this seems to work fine, but what I noticed is that the timestamps from the two trailing messages are also included in the combined event, kind of like this:</P> <PRE><CODE>May 5 08:12:16 myhost.company.com 05/05/16 08:12:16.392 [part1]May 5 08:12:16 myhost.company.com [part2]May 5 08:12:16 myhost.company.com [part3] </CODE></PRE> <P>Is there a way to break the events based on the second timestamp in the first message, but strip out the timestamps from the following two messages?</P> Thu, 05 May 2016 13:35:22 GMT https://community.splunk.com/t5/Getting-Data-In/How-to-merge-messages-into-a-single-event-using-one-timestamp/m-p/231922#M45186 _smp_ 2016-05-05T13:35:22Z https://community.splunk.com/t5/Getting-Data-In/How-to-merge-messages-into-a-single-event-using-one-timestamp/m-p/231923#M45187 <P>You should be able to use this:</P> <PRE><CODE>SEDCMD-strip_last_two_timestamps = s/\s+\d+\/\d+\/\d+\s+\d+:\d+:\d+\.\d+//g </CODE></PRE> <P>Put this on your indexers and restart Splunk and it should strip them out.</P> Thu, 05 May 2016 16:50:04 GMT https://community.splunk.com/t5/Getting-Data-In/How-to-merge-messages-into-a-single-event-using-one-timestamp/m-p/231923#M45187 woodcock 2016-05-05T16:50:04Z