I posted something about this here: http://splunk-base.splunk.com/answers/75118/line-break-involving-line-of My last question I don't think really addressed the issue because I was using index preview and had uploaded a stand alone, inactive log. I have a log that is the output of a script that runs and checks some components of some software. The log has a Check Start time and a Check End time. In between Start and End are other lines indicating when parts of the script have passed....or even failed. Because the results are live and not every line has a timestamp, I'm running into issues indexing this log correctly. The lines of the output of the script might come in a few lines a second, wait....then wait some more, then a few more lines....then finally finish with Check End: Splunk is taking each little "chunk" it posts and making a new event for this. I don't want that. I want splunk to post one event from Start until End. I'm not sure if you can have splunk "wait" for all the data before posting to the index? So far I have tried the following props.conf entries on my splunk indexer, all at separate times of course. I'm not sure but do I need a combination of line breaker and must break after? [myscript] BREAK_ONLY_BEFORE = \bCheck start: \w+ \w+ \d+ \d+:\d+:\d+ \w+ \d+\b MAX_EVENTS = 19 [myscript] LINE_BREAKER = (Check start:)(\[r\n]+)(Check end:) [myscript] LINE_BREAKER = Check start:\[r\n]+|Check end: [myscript] LINE_BREAKER=([\r\n]+)(=+[\r\n]+Check start) [myscript] LINE_BREAKER = (Check start:([rn]+)Check end:) SHOULD_LINEMERGE = false I want the below text to show up as 1 log entry. Perhaps the timestamp it could give for the entry is when the Check starts. =========================================== Check start: Tue Feb 5 23:28:10 EST 2013 =========================================== Accessing ATOMPUB_URL: http://BLAHBLAH:810/wackadoo/lobster with userid admin@AWESOMEHOST Grabbing heartbeat folder... Reading test file for upload... Checking if file already exists... Creating test upload... Document successfully created. Verifying that document was successfully uploaded... Verifying that rendition was successfully created... Deleting uploaded document... Deletion successful. Test successfully completed. Heartbeat check returned in 4001ms. Heartbeat check status:0 =========================================== Check end: Tue Feb 5 23:28:14 EST 2013 =========================================== Here is an example of what Splunk is doing. 1 2/13/13 3:54:34.000 PM Deleting uploaded document... Deletion successful. Test successfully completed. Heartbeat check returned in 4001ms. Heartbeat check status:0 =========================================== Check end: Wed Feb 13 15:54:34 EST 2013 =========================================== 2 2/13/13 3:54:33.000 PM Grabbing heartbeat folder... Reading test file for upload... Checking if file already exists... Creating test upload... Document successfully created. Verifying that document was successfully uploaded... Verifying that rendition was successfully created... 3 2/13/13 3:54:30.000 PM =========================================== Check start: Wed Feb 13 15:54:30 EST 2013 =========================================== Accessing ATOMPUB_URL: http://BLAHBLAH:810/wackadoo/lobster with userid admin@awesomehost ... View more

I'm having trouble with a line breaking issue. I'm using the preview feature with splunk and I'm trying to get the line breaks to work properly. Currently splunk is breaking every time there is a timestamp and I don't want that. I need my log entries to look like this and be one event for each chunk of data: =========================================== Check start: Tue Feb 5 23:28:10 EST 2013 =========================================== Accessing ATOMPUB_URL: http://BLAHBLAH:810/wackadoo/lobster with userid admin@AWESOMEHOST Grabbing heartbeat folder... Reading test file for upload... Checking if file already exists... Creating test upload... Document successfully created. Verifying that document was successfully uploaded... Verifying that rendition was successfully created... Deleting uploaded document... Deletion successful. Test successfully completed. Heartbeat check returned in 4001ms. Heartbeat check status:0 =========================================== Check end: Tue Feb 5 23:28:14 EST 2013 =========================================== Splunk is currently breaking up this one event into two events that look like this: Event 1 Check start: Tue Feb 5 23:28:10 EST 2013 =========================================== Accessing ATOMPUB_URL: http://BLAHBLAH:810/wackadoo/lobster with userid admin@AWESOMEHOST Grabbing heartbeat folder... Reading test file for upload... Checking if file already exists... Creating test upload... Document successfully created. Verifying that document was successfully uploaded... Verifying that rendition was successfully created... Deleting uploaded document... Deletion successful. Test successfully completed. Heartbeat check returned in 4001ms. Heartbeat check status:0 =========================================== Event 2 Check end: Tue Feb 5 23:26:20 EST 2013 =========================================== =========================================== I need to make it look like first entry I posted and have it be 1 event. So far I have Timestamp is always prefaced by a pattern and this regex: \bCheck \w+: \w+ \w+ \d+ \d+:\d+:\d+ \w+ \d+\b I have this for my props.conf: MUST_BREAK_AFTER = \bCheck end: \w+ \w+ \d+ \d+:\d+:\d+ \w+ \d+\b It's almost there, but I still am having issues with a line of =======================. Splunk now see's all of this as one event with these settings: =========================================== =========================================== Check start: Tue Feb 5 23:31:59 EST 2013 =========================================== Accessing ATOMPUB_URL: http://blahblah:810/booberry/lobster with userid admin@AWESOME Grabbing heartbeat folder... Reading test file for upload... Checking if file already exists... Creating test upload... Document successfully created. Verifying that document was successfully uploaded... Verifying that rendition was successfully created... Deleting uploaded document... Deletion successful. Test successfully completed. Heartbeat check returned in 3501ms. Heartbeat check status:0 =========================================== Check end: Tue Feb 5 23:32:03 EST 2013 I also tried specifying the timestamp format with %a %b %H:%M:%S %Z %Y for the timestamp. I kept getting the message that it "failed to parse timestamp" and was defaulting to the file modtime. This didn't do much to get rid of this error. Any ideas how to get that last ================= under Check end? If not I'll probably just do without but was curious if there was something I missed. ... View more

Ok I have been trying to get /var/log/messages and /var/log/cron to be indexed as the "splunk" user for a while and I'm pretty frustrated. I read this question here: http://splunk-base.splunk.com/answers/60388/recommended-permissions-on-varlog-for-splunk_ta_nix I followed some of the advice here and set permissions for /var/log directory using ACLs. I also tried making the splunk user a member of the adm group. I EVEN got to the point of changing permissions on the file itself with chmod and the file still did not show up as being indexed. I made sure I restarted splunk each time on the forwarder when I made these changes. I'm running Centos 6.2 Has anyone else had this issue? I could use UDP for port 514 but this way seemed a bit more sense and less config changes. The error i found in splunkd.log for this is: 02-07-2013 13:41:29.441 -0500 WARN FilesystemChangeWatcher - error getting attributes of path "/var/log/messages": Permission denied Obviously these changes aren't making a difference. Anyone else have this problem? Someone mentioned to me today "selinux" could be an issue. I also thought about giving sudo all access for the splunk user. Anyone else try this? ... View more