Rails Log LINEMERGE not working

[krussell101]

I'm not successfully getting line merging to work in my environment. The problem is with virtually all of my rail logs.

In /opt/splunk/etc/system/local/props.conf on my indexer I have the following:

[source::.../home/myapp/log/production.log]
BREAK_ONLY_BEFORE = Started
SHOULD_LINEMERGE = True

I don't have this on the HWF that are creating the logs. That's right is it not?

I had the above two lines reversed and changed due to a comment in another post. But that didn't fix it.

Also of note, I'm running tail -f production.log in a terminal window and watching that while seeing the real-time events come in on splunk. Pretty sure this is NOT related to log buffering. The events are flowing out of "tail" quite nicely. Certainly no second long delays.

Here are some examples of what is and is not working:

did not get split

Started GET "/merchant/en" for xx.xxx.xxx.xxx at 2012-07-25 19:07:21 +0000

Processing by Partner::TransfersController#index as */*

Parameters: {"locale"=>"en"}

Rendered partner/transfers/_form.html.erb (23.9ms)

Rendered partner/transfers/_merchant_info.erb (1.3ms)

Rendered partner/transfers/_transactions_table.html.erb (0.5ms)

Rendered partner/transfers/_filter_form.html.erb (2.9ms)

Rendered partner/transfers/_total_filter.html.erb (0.7ms)

Rendered partner/transfers/_filter_table.html.erb (1.1ms)

Rendered partner/transfers/index.html.erb within layouts/application (33.3ms)

Rendered layouts/public/_header.html.erb (60.7ms)

Completed 200 OK in 121ms (Views: 95.7ms | ActiveRecord: 8.9ms)

split this entry

Started GET "/merchant/en" for xx.xxx.xxx.xxx at 2012-07-25 19:02:23 +0000

Processing by Partner::TransfersController#index as */*

Parameters: {"locale"=>"en"}

Rendered partner/transfers/_form.html.erb (25.4ms)

Rendered partner/transfers/_merchant_info.erb (0.8ms)

Rendered partner/transfers/_transactions_table.html.erb (0.5ms)

split was here

Rendered partner/transfers/_filter_form.html.erb (67.5ms)

Rendered partner/transfers/_total_filter.html.erb (0.8ms)

Rendered partner/transfers/_filter_table.html.erb (1.4ms)

Rendered partner/transfers/index.html.erb within layouts/application (99.5ms)

Rendered layouts/public/_header.html.erb (1.1ms)

Completed 200 OK in 127ms (Views: 102.0ms | ActiveRecord: 8.8ms)

Split this entry

Started POST "/merchant/en/merchant_employees/sign_in" for xx.xxx.xxx.xxx at 2012-07-25 19:12:23 +0000

Processing by SessionsController#create as */*

Parameters: {"utf8"=>"✓", "authenticity_token"=>"+/lDricYw5cogAGwWZKMfbRFi1UNIip3bBVLXBUB1OM=", "merchant_employee"=>{"login_or_email"=>"healthcheck", "password"=>"[FILTERED]"}, "locale"=>"en"}

split was here

Redirected to https://www.geopay.kg/merchant/en

Completed 302 Found in 95ms (ActiveRecord: 0.0ms)

did not get split

Started POST "/merchant/en/merchant_employees/sign_in" for XX.XXX.XXX.XXX at 2012-07-25 19:17:20 +0000

Processing by SessionsController#create as */*

Parameters: {"utf8"=>"✓", "authenticity_token"=>"8HkKQSzw5TP/Rur0U+mqOm9xkbthOaVAAsAoBZ5ziFE=", "merchant_employee"=>{"login_or_email"=>"healthcheck", "password"=>"[FILTERED]"}, "locale"=>"en"}

Redirected to https://www.geopay.kg/merchant/en

Completed 302 Found in 159ms (ActiveRecord: 0.0ms)

Help Appreciated!

[dwaddle]

I see your comment "I don't have this on the HWF that are creating the logs. That's right is it not?"

If you are using the Heavy forwarder, then you need the linebreaking / linemerging configurations set up there. The Heavy forwarder does do event parsing, which is what makes it "heavy".

For a light or universal forwarder, having those configurations only on the indexer is correct.

What I might do to change your configuration is to update it as follows:

BREAK_ONLY_BEFORE = ^Started

Just to make sure that "Started" that appears in the middle of a line is ignored.