How to configure Splunk preveing parsing multiline Imperva logs as separate events?

[charanyack16]

Imperva to Splunk - Unable to properly parse multiline events. Rawquery fields are appended with different timestamps for each newline.

EX:

Event 1

Jul 11 09:18:18 abc.xyz.com CEF: 0|Imperva Inc.|SecureSphere|10.5.0.13_0|XYZ|XYZ.DAM|Informative|dest_ip=qq.tyy.214.28 dest_port=0000 dbuser=xyz sou_ip=00.000.000.00 source=000 proto=TCP createtime=09 July 2000 16:18:10, cat=XYZ Configuration Changes servergroup=XY MSServer servicename =YZ QLServer applicationname=XYZ App event_id=00000 query=Query usergroup=Default autheticated=Yes applicationuser= application= osuser= hostname= dbname=xyzmanager schemaname= bindvariable= errorvalue= responsesize=0 responsetime=0 affectedrows=0 parsequery=if object_id(?) is not ? drop table #entitytransaction create table #entitytransaction (transactionid bigint) rawdata=#015

Event 2

Jul 11 09:18:18 abc.xyz.com                     CREATE TABLE #EntityTrans (TransId bigint)#015

Ideally they are a single event, but Splunk displays them as two different events

[hardikJsheth]

You need to configure two attributes (SHOULD_LINEMERGE and BREAK_ONLY_BEFORE) in your props.conf file for your source type.

For e.g

[imperva]
SHOULD_LINEMERGE =true
BREAK_ONLY_BEFORE = abc.xyz.com

Update source type and BREAK_ONLY_BEFORE to appropriate value as per your requirement. Also note, this configurations are required on indexer node in clustered environment.

[charanyack16]

The props.conf file is shown below. The last field name in the log (refer to the log above) "parsequery" has the value,
parsequery=if object_id(?) is not ? drop table #entitytransaction create table #entitytransaction (transactionid bigint) rawdata=#015----------space-------------- CREATE TABLE #EntityTrans (TransId bigint)#015

This is different from rest of the field values.You see lot of space between "rawdata=#015----------------space------------------------- CREATE TABLE #EntityTrans (TransId bigint)#015"

How do I configure attribute SHOULD_LINEMERGE and BREAK_ONLY_BEFORE ?

[imperva:dam:uswc]
SHOULD_LINEMERGE = true
NO_BINARY_CHECK=true
CHARSET=UTF-8
KV_MODE=none
disabled=false
pulldown_type=true
TZ=America/Los_Angeles
TIME_PREFIX = ^
MAX_TIMESTAMP_LOOKAHEAD = 15
REPORT-0basic_fields_uswc = extract_basic_info_uswc
REPORT-1app_fields_uswc = extract_dam_fields_uswc
BREAK_ONLY_BEFORE =\s+CEF

[hardikJsheth]

You can have following parameters.

SHOULD_LINEMERGE = true
MUST_BREAK_AFTER =(parsequery=.*)abc.xyz.com
BREAK_ONLY_BEFORE =(abc.xyz.com\sCEF:)

Please replace abc.xyz.com with appropriate host name as you are getting in your logs. Also ensure this configurations are implemented on indexer node in case of distributed architecture.