Splunk Search

Index reads in multiple lines of logs instead of line-by-line

wanling
Path Finder

Hi,

I encountered a problem with the splunk indexing.

I developed a script to invoke tshark to generate HTTP traffic logs and configured splunk to monitor these log files.e.g.

1301619606.572769 172.20.180.4 -> 216.184.2.32 HTTP GET /size-women-us.htm HTTP/1.1   http.host == "www.wonderquest.com"

1301619607.031847 172.20.180.4 -> 216.184.2.32 HTTP GET /wq.css HTTP/1.1   http.host == "www.wonderquest.com"

Initially the log was indexed correctly, line by line. However, i recently noticed splunk takes in multiple lines as one event. It also no longer recognised the timestamp defined for the log file (through field extraction). It seems to take in the file creation date/time as the timestamp instead. All the related reports are affected.

The problem was observed even when splunk runs in stand-alone mode. And i tried some of the methods mentioned in the forum, but they did not help.

TIME_PREFIX = ,(?=\d+/\d+/\d{4} \d\d:\d\d) 

SHOULD_LINEMERGE = False 

MUST_BREAK_AFTER = <\n>

The splunk version is v4.1.4 build 82143. Hope someone can help here.

Tags (1)
0 Karma
1 Solution

Ayn
Legend

Was it working up until March 13th? I'm not seeing how your TIME_PREFIX applies to the log events, but if the parsing of your log events using epoch timestamps was working and recently stopped working, it seems likely it's due to the epoch bug fixed in 4.2.1: http://answers.splunk.com/questions/12621/since-march-13th-2011-gmt-splunk-no-longer-properly-parses...

View solution in original post

0 Karma

Ayn
Legend

Was it working up until March 13th? I'm not seeing how your TIME_PREFIX applies to the log events, but if the parsing of your log events using epoch timestamps was working and recently stopped working, it seems likely it's due to the epoch bug fixed in 4.2.1: http://answers.splunk.com/questions/12621/since-march-13th-2011-gmt-splunk-no-longer-properly-parses...

0 Karma

wanling
Path Finder

Thanks Ayn! the timestamp can now be recognised correctly after applying the datetime format patch.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...