Symantec log LINE BREAKING and weird timestamp

[bernardoortega]

Hi Guys

We find some small subsets of the logs, around 0.1% to be multiline events.

As an example, the one below. When doing a search index=symantec linecount>1 is when we see those events and the very strange dates which on some events is 1979 and 1980 as well.

I configured this in props.conf on the symantec TA on the indexers for LINE BREAKING: ([\n\r]+)\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}, but still I see linecount>2 not more than 1

2016-05-18 01:23:30,Info,XXXXXXXXX,Category: 2,LiveUpdate Manager,A LiveUpdate session ran successfully.  No new updates were available.
2262-04-02 01:24:55,Info,XXXXXXXXX,Category: 2,Symantec AntiVirus,Symantec Endpoint Protection services failed to start.  (2000005F)
2262-04-02 01:24:55,Info,XXXXXXX,Category: 2,Symantec AntiVirus,Could not start Service Engine err=2000005F

[Zubbuz]

Hi bernardoortega,

What was done to fix this issue?

[jkat54]

Try this instead:

[sep12:system]
SHOULD_LINEMERGE=false
NO_BINARY_CHECK=true
BREAK_ONLY_BEFORE=\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}

[bernardoortega]

Hello

Well, first i tried with this, which it remove all event with lincecount>1 but lincecount>2 you may find still some. Is weird, becuase if you go to http://regexr.com/ and paly with the event and regex it seems to break all lines, but on https://regex101.com/ it only one one line and does not continue.

THis is the /local/props.conf I CONFIGURED on the SEP TA:

[sep12:system]

LINE_BREAKER = ([\n\r]+)\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}

[sep12:agent]

LINE_BREAKER = ([\n\r]+)\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}

[sep12:behavior]

LINE_BREAKER = ([\n\r]+)\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}

[sep12:traffic]

LINE_BREAKER = ([\n\r]+)\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}

[sep12:scan]

LINE_BREAKER = ([\n\r]+)\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}

[sep12:ids]

LINE_BREAKER = ([\n\r]+)\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}

[sep12:risk]

LINE_BREAKER = ([\n\r]+)\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}

[sep12:proactive]

LINE_BREAKER = ([\n\r]+)\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}

[sep12:admin]

LINE_BREAKER = ([\n\r]+)\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}

[sep12:policy]

LINE_BREAKER = ([\n\r]+)\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}

And the default one, which is working now is: (DEFAULT TO THE TA

[sep12:log]
TRANSFORMS-sub_sourcetype_sep12_file = force_sourcetype_sep12_agent,force_sourcetype_sep12_traffic,force_sourcetype_sep12_proactive,force_sourcetype_sep12_behavior,force_sourcetype_sep12_risk,force_sourcetype_sep12_admin,force_sourcetype_sep12_policy,force_sourcetype_sep12_agt_system,force_sourcetype_sep12_scan,force_sourcetype_sep12_security,force_sourcetype_sep12_scm_system

.tmp transforms are for file-based data

[sourcetype::...smc_admin.tmp]
sourcetype = sep12:admin
SHOULD_LINEMERGE = false
TIME_FORMAT = %Y-%m-%d%n%%H:%M%S

[sourcetype::...smc_agent_act.tmp]
sourcetype = sep12:agent
SHOULD_LINEMERGE = false
TIME_FORMAT = %Y-%m-%d%n%%H:%M%S

[sourcetype::...smc_policy.tmp]
sourcetype = sep12:policy
SHOULD_LINEMERGE = false
TIME_FORMAT = %Y-%m-%d%n%%H:%M%S

[sourcetype::...agt_behavior.tmp]
sourcetype = sep12:behavior
SHOULD_LINEMERGE = false
TIME_FORMAT = %Y-%m-%d%n%%H:%M%S

[sourcetype::...agt_proactive.tmp]
sourcetype = sep12:proactive
SHOULD_LINEMERGE = false
TIME_FORMAT = %Y-%m-%d%n%%H:%M%S

Thanks

[ppablo]

Hi @bernardoortega

When you say "symantec TA", which one specifically from https://splunkbase.splunk.com/ are you talking about?

[bernardoortega]

https://splunkbase.splunk.com/app/1365/#/documentation. The add on is embedded into the APP

is the one for version SEP12

[jkat54]

Can you post your entire stanza for this sourcetype in your props.conf please?

It should be similar to this:

[sourcetypeName]
SHOULD_LINEMERGE=true
NO_BINARY_CHECK=true
BREAK_ONLY_BEFORE=\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}

[bernardoortega]

I guess that When SHOULD_LINEMERGE=true is when you can configure BREAK_ONLY_BEFORE, but if it is false, it will not work

[jkat54]

yes, i got it mixed up. I've corrected my comment. Can you try with SHOULD_LINEMERGE=true and BREAK_ONLY_BEFORE=\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}?