Getting Data In

Another "Linebreak not working" question

Explorer

My developers have an unorthodox format for their logs. The only timestamp on a multi-line log entry is at the very end of the event, in a "summary" line. Naturally, Splunk incorrectly treats the summary as the first line for the following log entry. Nearly all of the non-summary lines are indented with 2 spaces. The summary line which I am trying to reformat as the last line in the event always ends with a string like this: 99M +0k

With that in mind, I tried these two solutions. The first works about 90% but inserts event breaks at some truly random spots the other 10% of the time. The second attempt does not work at all, in spite of my testing the REGEX via egrep on the splunkforder client directly.
First attempt:

[helios]
SHOULD_LINEMERGE = true
BREAK_ONLY_BEFORE_DATE = false
MUST_BREAK_AFTER = [0-9]+[MmKk] \+[0-9]+[MmKk]$
MUST_NOT_BREAK_AFTER = ^\s

SECOND ATTEMPT:

[helios]
SHOULD_LINEMERGE = false
LINE_BREAKER=([\r\n]+)127\.0\.0\.1 .* [0-9]+[MmKk] \+[0-9]+[MmKk]$

NOTE: Splunk is not having a problem with interpreting the datestamp, before or after I tried to rework the linebreaks.

Here is some sample data:

  (0.00) SELECT * FROM `movie_external_ids` WHERE ((`external_id` = 'pyczhoNDosSpEu7XWHB2BmUzVn-iHJpp') AND (`provider_id` = 32729)) LIMIT 1
  (0.00) SELECT * FROM `movies` WHERE (`embed_code` = 'pyczhoNDosSpEu7XWHB2BmUzVn-iHJpp') LIMIT 1
  (0.00) SELECT * FROM `ad_sets` WHERE (`ad_set_code` = '751eb5aa7dcd4fbb9f5fb0571cb39766') LIMIT 1
  (0.00) UPDATE `movies` SET `status` = 'live', `preview` = 'v2:a', `custom_promo_rev` = 158896606, `uploaded_by` = 0, `live_stream_smil_url` = NULL, `time` = 94528, `rate` = NULL, `uploaded_at` = '2012-04-17 01:56:40', `episode` = NULL, `description` = 'Featherman and Knott face off', `language` = 'en', `preview_image_index` = -158896606, `orig_movie_id` = 0, `temp_popularity` = 0, `processing_end` = '2012-04-17 02:18:44', `single_stream_video_descriptor_version` = NULL, `live_stream_id` = 0, `mix_base_id` = NULL, `user_specified_id` = 'pyczhoNDosSpEu7XWHB2BmUzVn-iHJpp', `reconstituted_source_file_available` = 1, `full_movie_available` = 0, `nuplayer_tweaks` = NULL, `has_recurring_flight_time` = 0, `price` = NULL, `approved_domains` = NULL, `items` = NULL, `ip` = NULL, `age_rating` = 'None', `source_filename_scheme_version` = 2, `genre` = 'Others', `promo` = '640x360 pyczhoNDosSpEu7XWHB2BmUzVn-iHJpp 0:promo158896606', `flight_start_time` = '2012-04-17 01:56:36', `processing_start` = '2012-04-17 02:01:27', `deleted_at` = NULL, `size` = 30774172, `embed_code` = 'pyczhoNDosSpEu7XWHB2BmUzVn-iHJpp', `has_outlines` = 0, `serving_url` = NULL, `is_part_of_series` = 0, `provider_id` = 32729, `name` = 'Coughin vs Tunkhannock Baseball', `updated_at` = '2012-05-23 23:22:44', `postprocess_status` = 'live', `admin_flag` = '', `flight_end_time` = NULL, `parent_ids` = '', `tweaks` = NULL, `player_id` = 56936, `iphone_enabled` = 0, `created_at` = '2012-04-17 01:56:36', `processing_progress` = 1.0, `content_type` = 'Video', `error_text` = NULL, `synd_group_id` = 42898, `ad_set_id` = 17192, `season` = NULL, `overrides_synd_flight_times` = 0 WHERE (`id` = 6742182) LIMIT 1
  (0.00) SELECT * FROM `providers` WHERE (`providers`.`id` = 32729) LIMIT 1
Helios request
127.0.0.1 - - [23/May/2012 23:22:44] "PUT /assets/pyczhoNDosSpEu7XWHB2BmUzVn-iHJpp/ad_set/751eb5aa7dcd4fbb9f5fb0571cb39766 HTTP/1.1" 200 - 0.2131 99M +0k
  (0.00) SELECT * FROM `users` WHERE (`api_key` = 'w1cHE6To2EhAeIt2mx2p9196-TtN.IYlbW') LIMIT 1
Helios request
127.0.0.1 - - [23/May/2012 23:22:44] "GET /apis/authentication_info/w1cHE6To2EhAeIt2mx2p9196-TtN.IYlbW HTTP/1.1" 200 89 0.0142 99M +0k
  (0.00) SELECT * FROM `users` WHERE (`api_key` = 'w1cHE6To2EhAeIt2mx2p9196-TtN.IYlbW') LIMIT 1
  (0.00) SELECT * FROM `providers` WHERE (`providers`.`id` = 32730) LIMIT 1
  (0.00) SELECT * FROM `movie_external_ids` WHERE ((`external_id` = '0yaTBxNDp4oeW11cZJPRYAyme8oyodjW') AND (`provider_id` = 32730)) LIMIT 1
  (0.00) SELECT * FROM `movies` WHERE (`embed_code` = '0yaTBxNDp4oeW11cZJPRYAyme8oyodjW') LIMIT 1
  (0.00) SELECT * FROM `ad_sets` WHERE (`ad_set_code` = '78cda6ff1b2a4a8b9a70f232ab534bec') LIMIT 1
  (0.00) UPDATE `movies` SET `status` = 'live', `preview` = 'v2:a', `custom_promo_rev` = 161644216, `uploaded_by` = 0, `live_stream_smil_url` = NULL, `time` = 94899, `rate` = NULL, `uploaded_at` = '2012-05-18 21:10:11', `episode` = NULL, `description` = NULL, `language` = 'en', `preview_image_index` = -161644216, `orig_movie_id` = 0, `temp_popularity` = 0, `processing_end` = '2012-05-18 21:12:51', `single_stream_video_descriptor_version` = NULL, `live_stream_id` = 0, `mix_base_id` = NULL, `user_specified_id` = '0yaTBxNDp4oeW11cZJPRYAyme8oyodjW', `reconstituted_source_file_available` = 1, `full_movie_available` = 0, `nuplayer_tweaks` = NULL, `has_recurring_flight_time` = 0, `price` = NULL, `approved_domains` = NULL, `items` = NULL, `ip` = NULL, `age_rating` = 'None', `source_filename_scheme_version` = 2, `genre` = 'Others', `promo` = '640x360 0yaTBxNDp4oeW11cZJPRYAyme8oyodjW 0:promo161644216', `flight_start_time` = '2012-05-18 21:10:05', `processing_start` = '2012-05-18 21:10:22', `deleted_at` = NULL, `size` = 30618465, `embed_code` = '0yaTBxNDp4oeW11cZJPRYAyme8oyodjW', `has_outlines` = 0, `serving_url` = NULL, `is_part_of_series` = 0, `provider_id` = 32730, `name` = 'Thunder Get Ready for Game 3 in LA', `updated_at` = '2012-05-23 23:22:44', `postprocess_status` = 'live', `admin_flag` = '', `flight_end_time` = NULL, `parent_ids` = '', `tweaks` = NULL, `player_id` = 56937, `iphone_enabled` = 0, `created_at` = '2012-05-18 21:10:05', `processing_progress` = 1.0, `content_type` = 'Video', `error_text` = NULL, `synd_group_id` = 42899, `ad_set_id` = 17370, `season` = NULL, `overrides_synd_flight_times` = 0 WHERE (`id` = 7117882) LIMIT 1
  (0.00) SELECT * FROM `providers` WHERE (`providers`.`id` = 32730) LIMIT 1
127.0.0.1 - - [23/May/2012 23:22:44] "PUT /assets/0yaTBxNDp4oeW11cZJPRYAyme8oyodjW/ad_set/78cda6ff1b2a4a8b9a70f232ab534bec HTTP/1.1" 200 - 0.0463 95M +0k
  (0.09) SELECT * FROM `users` WHERE (`api_key` = 'A4cHE6JQ_qhd2K2c2rc9e1M6u_py.Jp7JE') LIMIT 1
Helios request
127.0.0.1 - - [23/May/2012 23:22:44] "GET /apis/authentication_info/A4cHE6JQ_qhd2K2c2rc9e1M6u_py.Jp7JE HTTP/1.1" 200 89 0.1024 59M +0k
Tags (1)
0 Karma

Explorer

Thanks for help with formatting. I have edited it further to eliminate erroneous newlines and confirmed that line wrapping is occurring as expected.

0 Karma

Splunk Employee
Splunk Employee

I have formatted your data and config so it's readable. However, please clarify whether the sample data you have included is exactly as you have pasted. In particular, you have some blank lines, and you also have some newlines within what appear to be single logical lines. Do those newlines actually occur in the data? Please edit your data so it looks exactly like what we might find in the file. Note that long lines will cause a horizontal scrollbar to appear.

0 Karma