My developers have an unorthodox format for their logs. The only timestamp on a multi-line log entry is at the very end of the event, in a "summary" line. Naturally, Splunk incorrectly treats the summary as the first line for the following log entry. Nearly all of the non-summary lines are indented with 2 spaces. The summary line which I am trying to reformat as the last line in the event always ends with a string like this: 99M +0k
With that in mind, I tried these two solutions. The first works about 90% but inserts event breaks at some truly random spots the other 10% of the time. The second attempt does not work at all, in spite of my testing the REGEX via egrep on the splunkforder client directly.
First attempt:
[helios]
SHOULD_LINEMERGE = true
BREAK_ONLY_BEFORE_DATE = false
MUST_BREAK_AFTER = [0-9]+[MmKk] \+[0-9]+[MmKk]$
MUST_NOT_BREAK_AFTER = ^\s
SECOND ATTEMPT:
[helios]
SHOULD_LINEMERGE = false
LINE_BREAKER=([\r\n]+)127\.0\.0\.1 .* [0-9]+[MmKk] \+[0-9]+[MmKk]$
NOTE: Splunk is not having a problem with interpreting the datestamp, before or after I tried to rework the linebreaks.
Here is some sample data:
(0.00) SELECT * FROM `movie_external_ids` WHERE ((`external_id` = 'pyczhoNDosSpEu7XWHB2BmUzVn-iHJpp') AND (`provider_id` = 32729)) LIMIT 1
(0.00) SELECT * FROM `movies` WHERE (`embed_code` = 'pyczhoNDosSpEu7XWHB2BmUzVn-iHJpp') LIMIT 1
(0.00) SELECT * FROM `ad_sets` WHERE (`ad_set_code` = '751eb5aa7dcd4fbb9f5fb0571cb39766') LIMIT 1
(0.00) UPDATE `movies` SET `status` = 'live', `preview` = 'v2:a', `custom_promo_rev` = 158896606, `uploaded_by` = 0, `live_stream_smil_url` = NULL, `time` = 94528, `rate` = NULL, `uploaded_at` = '2012-04-17 01:56:40', `episode` = NULL, `description` = 'Featherman and Knott face off', `language` = 'en', `preview_image_index` = -158896606, `orig_movie_id` = 0, `temp_popularity` = 0, `processing_end` = '2012-04-17 02:18:44', `single_stream_video_descriptor_version` = NULL, `live_stream_id` = 0, `mix_base_id` = NULL, `user_specified_id` = 'pyczhoNDosSpEu7XWHB2BmUzVn-iHJpp', `reconstituted_source_file_available` = 1, `full_movie_available` = 0, `nuplayer_tweaks` = NULL, `has_recurring_flight_time` = 0, `price` = NULL, `approved_domains` = NULL, `items` = NULL, `ip` = NULL, `age_rating` = 'None', `source_filename_scheme_version` = 2, `genre` = 'Others', `promo` = '640x360 pyczhoNDosSpEu7XWHB2BmUzVn-iHJpp 0:promo158896606', `flight_start_time` = '2012-04-17 01:56:36', `processing_start` = '2012-04-17 02:01:27', `deleted_at` = NULL, `size` = 30774172, `embed_code` = 'pyczhoNDosSpEu7XWHB2BmUzVn-iHJpp', `has_outlines` = 0, `serving_url` = NULL, `is_part_of_series` = 0, `provider_id` = 32729, `name` = 'Coughin vs Tunkhannock Baseball', `updated_at` = '2012-05-23 23:22:44', `postprocess_status` = 'live', `admin_flag` = '', `flight_end_time` = NULL, `parent_ids` = '', `tweaks` = NULL, `player_id` = 56936, `iphone_enabled` = 0, `created_at` = '2012-04-17 01:56:36', `processing_progress` = 1.0, `content_type` = 'Video', `error_text` = NULL, `synd_group_id` = 42898, `ad_set_id` = 17192, `season` = NULL, `overrides_synd_flight_times` = 0 WHERE (`id` = 6742182) LIMIT 1
(0.00) SELECT * FROM `providers` WHERE (`providers`.`id` = 32729) LIMIT 1
Helios request
127.0.0.1 - - [23/May/2012 23:22:44] "PUT /assets/pyczhoNDosSpEu7XWHB2BmUzVn-iHJpp/ad_set/751eb5aa7dcd4fbb9f5fb0571cb39766 HTTP/1.1" 200 - 0.2131 99M +0k
(0.00) SELECT * FROM `users` WHERE (`api_key` = 'w1cHE6To2EhAeIt2mx2p9196-TtN.IYlbW') LIMIT 1
Helios request
127.0.0.1 - - [23/May/2012 23:22:44] "GET /apis/authentication_info/w1cHE6To2EhAeIt2mx2p9196-TtN.IYlbW HTTP/1.1" 200 89 0.0142 99M +0k
(0.00) SELECT * FROM `users` WHERE (`api_key` = 'w1cHE6To2EhAeIt2mx2p9196-TtN.IYlbW') LIMIT 1
(0.00) SELECT * FROM `providers` WHERE (`providers`.`id` = 32730) LIMIT 1
(0.00) SELECT * FROM `movie_external_ids` WHERE ((`external_id` = '0yaTBxNDp4oeW11cZJPRYAyme8oyodjW') AND (`provider_id` = 32730)) LIMIT 1
(0.00) SELECT * FROM `movies` WHERE (`embed_code` = '0yaTBxNDp4oeW11cZJPRYAyme8oyodjW') LIMIT 1
(0.00) SELECT * FROM `ad_sets` WHERE (`ad_set_code` = '78cda6ff1b2a4a8b9a70f232ab534bec') LIMIT 1
(0.00) UPDATE `movies` SET `status` = 'live', `preview` = 'v2:a', `custom_promo_rev` = 161644216, `uploaded_by` = 0, `live_stream_smil_url` = NULL, `time` = 94899, `rate` = NULL, `uploaded_at` = '2012-05-18 21:10:11', `episode` = NULL, `description` = NULL, `language` = 'en', `preview_image_index` = -161644216, `orig_movie_id` = 0, `temp_popularity` = 0, `processing_end` = '2012-05-18 21:12:51', `single_stream_video_descriptor_version` = NULL, `live_stream_id` = 0, `mix_base_id` = NULL, `user_specified_id` = '0yaTBxNDp4oeW11cZJPRYAyme8oyodjW', `reconstituted_source_file_available` = 1, `full_movie_available` = 0, `nuplayer_tweaks` = NULL, `has_recurring_flight_time` = 0, `price` = NULL, `approved_domains` = NULL, `items` = NULL, `ip` = NULL, `age_rating` = 'None', `source_filename_scheme_version` = 2, `genre` = 'Others', `promo` = '640x360 0yaTBxNDp4oeW11cZJPRYAyme8oyodjW 0:promo161644216', `flight_start_time` = '2012-05-18 21:10:05', `processing_start` = '2012-05-18 21:10:22', `deleted_at` = NULL, `size` = 30618465, `embed_code` = '0yaTBxNDp4oeW11cZJPRYAyme8oyodjW', `has_outlines` = 0, `serving_url` = NULL, `is_part_of_series` = 0, `provider_id` = 32730, `name` = 'Thunder Get Ready for Game 3 in LA', `updated_at` = '2012-05-23 23:22:44', `postprocess_status` = 'live', `admin_flag` = '', `flight_end_time` = NULL, `parent_ids` = '', `tweaks` = NULL, `player_id` = 56937, `iphone_enabled` = 0, `created_at` = '2012-05-18 21:10:05', `processing_progress` = 1.0, `content_type` = 'Video', `error_text` = NULL, `synd_group_id` = 42899, `ad_set_id` = 17370, `season` = NULL, `overrides_synd_flight_times` = 0 WHERE (`id` = 7117882) LIMIT 1
(0.00) SELECT * FROM `providers` WHERE (`providers`.`id` = 32730) LIMIT 1
127.0.0.1 - - [23/May/2012 23:22:44] "PUT /assets/0yaTBxNDp4oeW11cZJPRYAyme8oyodjW/ad_set/78cda6ff1b2a4a8b9a70f232ab534bec HTTP/1.1" 200 - 0.0463 95M +0k
(0.09) SELECT * FROM `users` WHERE (`api_key` = 'A4cHE6JQ_qhd2K2c2rc9e1M6u_py.Jp7JE') LIMIT 1
Helios request
127.0.0.1 - - [23/May/2012 23:22:44] "GET /apis/authentication_info/A4cHE6JQ_qhd2K2c2rc9e1M6u_py.Jp7JE HTTP/1.1" 200 89 0.1024 59M +0k
... View more