All Apps and Add-ons

Symantec log LINE BREAKING and weird timestamp

bernardoortega
Path Finder

Hi Guys

We find some small subsets of the logs, around 0.1% to be multiline events.

As an example, the one below. When doing a search index=symantec linecount>1 is when we see those events and the very strange dates which on some events is 1979 and 1980 as well.

I configured this in props.conf on the symantec TA on the indexers for LINE BREAKING: ([\n\r]+)\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}, but still I see linecount>2 not more than 1

2016-05-18 01:23:30,Info,XXXXXXXXX,Category: 2,LiveUpdate Manager,A LiveUpdate session ran successfully.  No new updates were available.
2262-04-02 01:24:55,Info,XXXXXXXXX,Category: 2,Symantec AntiVirus,Symantec Endpoint Protection services failed to start.  (2000005F)
2262-04-02 01:24:55,Info,XXXXXXX,Category: 2,Symantec AntiVirus,Could not start Service Engine err=2000005F
0 Karma

Zubbuz
Loves-to-Learn

Hi bernardoortega,

 

What was done to fix this issue?

0 Karma

jkat54
SplunkTrust
SplunkTrust

Try this instead:

[sep12:system]
SHOULD_LINEMERGE=false
NO_BINARY_CHECK=true
BREAK_ONLY_BEFORE=\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}

0 Karma

bernardoortega
Path Finder

Hello

Well, first i tried with this, which it remove all event with lincecount>1 but lincecount>2 you may find still some. Is weird, becuase if you go to http://regexr.com/ and paly with the event and regex it seems to break all lines, but on https://regex101.com/ it only one one line and does not continue.

THis is the /local/props.conf I CONFIGURED on the SEP TA:

[sep12:system]

LINE_BREAKER = ([\n\r]+)\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}

[sep12:agent]

LINE_BREAKER = ([\n\r]+)\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}

[sep12:behavior]

LINE_BREAKER = ([\n\r]+)\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}

[sep12:traffic]

LINE_BREAKER = ([\n\r]+)\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}

[sep12:scan]

LINE_BREAKER = ([\n\r]+)\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}

[sep12:ids]

LINE_BREAKER = ([\n\r]+)\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}

[sep12:risk]

LINE_BREAKER = ([\n\r]+)\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}

[sep12:proactive]

LINE_BREAKER = ([\n\r]+)\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}

[sep12:admin]

LINE_BREAKER = ([\n\r]+)\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}

[sep12:policy]

LINE_BREAKER = ([\n\r]+)\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}

And the default one, which is working now is: (DEFAULT TO THE TA

[sep12:log]
TRANSFORMS-sub_sourcetype_sep12_file = force_sourcetype_sep12_agent,force_sourcetype_sep12_traffic,force_sourcetype_sep12_proactive,force_sourcetype_sep12_behavior,force_sourcetype_sep12_risk,force_sourcetype_sep12_admin,force_sourcetype_sep12_policy,force_sourcetype_sep12_agt_system,force_sourcetype_sep12_scan,force_sourcetype_sep12_security,force_sourcetype_sep12_scm_system

.tmp transforms are for file-based data

[sourcetype::...smc_admin.tmp]
sourcetype = sep12:admin
SHOULD_LINEMERGE = false
TIME_FORMAT = %Y-%m-%d%n%%H:%M%S

[sourcetype::...smc_agent_act.tmp]
sourcetype = sep12:agent
SHOULD_LINEMERGE = false
TIME_FORMAT = %Y-%m-%d%n%%H:%M%S

[sourcetype::...smc_policy.tmp]
sourcetype = sep12:policy
SHOULD_LINEMERGE = false
TIME_FORMAT = %Y-%m-%d%n%%H:%M%S

[sourcetype::...agt_behavior.tmp]
sourcetype = sep12:behavior
SHOULD_LINEMERGE = false
TIME_FORMAT = %Y-%m-%d%n%%H:%M%S

[sourcetype::...agt_proactive.tmp]
sourcetype = sep12:proactive
SHOULD_LINEMERGE = false
TIME_FORMAT = %Y-%m-%d%n%%H:%M%S

Thanks

0 Karma

ppablo
Retired

Hi @bernardoortega

When you say "symantec TA", which one specifically from https://splunkbase.splunk.com/ are you talking about?

0 Karma

bernardoortega
Path Finder

https://splunkbase.splunk.com/app/1365/#/documentation. The add on is embedded into the APP

is the one for version SEP12

0 Karma

jkat54
SplunkTrust
SplunkTrust

Can you post your entire stanza for this sourcetype in your props.conf please?

It should be similar to this:

[sourcetypeName]
SHOULD_LINEMERGE=true
NO_BINARY_CHECK=true
BREAK_ONLY_BEFORE=\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}

0 Karma

bernardoortega
Path Finder

I guess that When SHOULD_LINEMERGE=true is when you can configure BREAK_ONLY_BEFORE, but if it is false, it will not work

0 Karma

jkat54
SplunkTrust
SplunkTrust

yes, i got it mixed up. I've corrected my comment. Can you try with SHOULD_LINEMERGE=true and BREAK_ONLY_BEFORE=\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}?

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...