Index reads in multiple lines of logs instead of line-by-line

[wanling]

Hi,

I encountered a problem with the splunk indexing.

I developed a script to invoke tshark to generate HTTP traffic logs and configured splunk to monitor these log files.e.g.

1301619606.572769 172.20.180.4 -> 216.184.2.32 HTTP GET /size-women-us.htm HTTP/1.1   http.host == "www.wonderquest.com"

1301619607.031847 172.20.180.4 -> 216.184.2.32 HTTP GET /wq.css HTTP/1.1   http.host == "www.wonderquest.com"

Initially the log was indexed correctly, line by line. However, i recently noticed splunk takes in multiple lines as one event. It also no longer recognised the timestamp defined for the log file (through field extraction). It seems to take in the file creation date/time as the timestamp instead. All the related reports are affected.

The problem was observed even when splunk runs in stand-alone mode. And i tried some of the methods mentioned in the forum, but they did not help.

TIME_PREFIX = ,(?=\d+/\d+/\d{4} \d\d:\d\d) 

SHOULD_LINEMERGE = False 

MUST_BREAK_AFTER = <\n>

The splunk version is v4.1.4 build 82143. Hope someone can help here.

[Ayn · Accepted solution]

Was it working up until March 13th? I'm not seeing how your TIME_PREFIX applies to the log events, but if the parsing of your log events using epoch timestamps was working and recently stopped working, it seems likely it's due to the epoch bug fixed in 4.2.1: http://answers.splunk.com/questions/12621/since-march-13th-2011-gmt-splunk-no-longer-properly-parses...

[wanling]

Thanks Ayn! the timestamp can now be recognised correctly after applying the datetime format patch.