What is the best practice for parsing custom multiline logs?

[gauravmishra15]

Dear Splunk experts,

I am working on parsing multiline custom application logs where log represents multiple lines to process some cases with invariable pattern including case type, processing StartvTime, SQL statements(multiple), respective SQL statement response time and processing end time.

Please suggest the best practice to parse these kind of logs for field extraction.

Thanks

[renjith_nair]

There are multiple ways to achieve this.

1. Easiest and best method is to merge events : http://docs.splunk.com/Documentation/Splunk/6.2.2/Data/Indexmulti-lineevents

2. If you have common fields, join the events with one of the commands (join,transaction,stats etc) . https://answers.splunk.com/answers/335149/how-do-get-the-combined-search-result.html

[s2_splunk · Accepted solution]

There isn't really a best practice; it all depends on what your logs actually look like. Please review the documentation on line breaking and the default (but configurable) limits for event sizes etc. to learn how this works.
If you can't get Splunk to do the right thing, your best chance of getting help is to provide a log sample of what you are dealing with and the configuration you have unsuccessfully tried.
Hope this helps.