Cyberark Logs - Line Breaking doesn't work

[HIBE151]

Hello together,

today we tried to fix the line breaking and timestamp problems we have with logs from cyberark.:

<5>1 2019-02-02T10:14:37Z hostid CEF:0|Cyber-Ark|Vault|10.5.0000|51|Retrieve File|5|act=Retrieve File suser=PasswordManager fname=Root\Policies\Policy-TEST_MAIL_DualControl.ini dvc= shost=192.169.122.122 dhost= duser= externalId= app= reason= cs1Label="Affected User Name" cs1= cs2Label="Safe Name" cs2=PasswordManagerShared cs3Label="Device Type" cs3= cs4Label="Database" cs4= cs5Label="Other info" cs5= cn1Label="Request Id" cn1= cn2Label="Ticket Id" cn2= msg=<5>1 2019-02-02T10:14:37Z hostid CEF:0|Cyber-Ark|Vault|10.5.0000|51|Retrieve File|5|act=Retrieve File suser=PasswordManager fname=Root\Policies\Policy-SWIFT_InvestigationCenter-TEST.ini dvc= shost=192.168.122.222 dhost= duser= externalId= app= reason= cs1Label="Affected User Name" cs1= cs2Label="Safe Name" cs2=PasswordManagerShared cs3Label="Device Type" cs3= cs4Label="Database" cs4= cs5Label="Other info" cs5= cn1Label="Request Id" cn1= cn2Label="Ticket Id" cn2= msg=<5>1 2019-02-02T10:14:37Z hostid CEF:0|Cyber-Ark|Vault|10.5.0000|51|Retrieve File|5|act=Retrieve File suser=PasswordManager fname=Root\Policies\Policy-BLUB_InvestigationCenter.ini dvc= shost=192.168.122.221 dhost= duser= externalId= app= reason= cs1Label="Affected User Name" cs1= cs2Label="Safe Name" cs2=PasswordManagerShared cs3Label="Device Type" cs3= cs4Label="Database" cs4= cs5Label="Other info" cs5= cn1Label="Request Id" cn1= cn2Label="Ticket Id" cn2= msg=<5>1 2019-02-02T10:14:37Z hostid CEF:0|Cyber-Ark|Vault|10.5.0000|51|Retrieve File|5|act=Retrieve File suser=PasswordManager fname=Root\Policies\Policy-SWIFT_GTX_Bottomline.ini dvc= shost=192.168.178.212 dhost= duser= externalId= app= reason= cs1Label="Affected User Name" cs1= cs2Label="Safe Name" cs2=PasswordManagerShared cs3Label="Device Type" cs3= cs4Label="Database" cs4= cs5Label="Other info" cs5= cn1Label="Request Id" cn1= cn2Label="Ticket Id" cn2= msg=<5>1 2019-02-02T10:14:37Z hostid CEF:0|Cyber-Ark|Vault|10.5.0000|51|Retrieve File|5|act=Retrieve File suser=PasswordManager fname=Root\Policies\Policy-SWIFT_GTX_Bottomline-TEST.ini dvc= shost=192.122.122.122 dhost= duser= externalId= app= reason= cs1Label="Affected User Name" cs1= cs2Label="Safe Name" cs2=PasswordManagerShared cs3Label="Device Type" cs3= cs4Label="Database" cs4= cs5Label="Other info" cs5= cn1Label="Request Id" cn1= cn2Label="Ticket Id" cn2= msg=

Timestamp problem fixed with following configuration:

MAX_TIMESTAMP_LOOKAHEAD = 30
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
TIME_FORMAT = %Y-%m-%dT%H:%M:%S
TIME_PREFIX = \<\d\>\d

But for any reason I can't change Line Breaks for this text. Actually I want to break the event everytime when one of those <5>1 appears. Does anyone has an idea why?

[pellegrini]

If syslog is sent with TCP or TLS then this is an known issue in CyberArk. If you are using SC4S (Splunk Connect for Syslog) the solution is to add a line feed https://cyberark-customers.force.com/s/article/00004289

[Esky73]

did you ever fix this?

[chrisyounger]

Hi @HIBE151

This should work LINE_BREAKER = ([\r\n]*)<\d+>1\s

One important thing to remember is that the LINE_BREAKER must specify a capturing group - even if it captures nothing (like it seems to in your case). Also don't forget the LINE_BREAKER happens on the first heavy forwarder or indexer that the data goes through.

I hope this help you

[whrg]

Hello @HIBE151,

You need to use LINE_BREAKER. It specifies how the log text is broken into individual events. By default, LINE_BREAKER is set to ([\r\n]+). That is, each line break starts a new event. In your case, there is no line break, so there is only one long event.

I suggest you set LINE_BREAKER to:

LINE_BREAKER = (<5>1\s*)

Have a look at this screenshot:

Now you don't need TIME_PREFIX because now the time stamp is at the beginning of each event.

What does the "Z" in "2019-02-02T10:14:37Z" stand for? If that is the time zone, then add %Z to your TIME_FORMAT.

Overall, props.conf should look something like this:

[name]
category = Custom
pulldown_type = true
NO_BINARY_CHECK = true
LINE_BREAKER = (<5>1\s*)
SHOULD_LINEMERGE = false
TIME_FORMAT = %Y-%m-%dT%H:%M:%S%Z

[HIBE151]

thanks for your answer. Finally the breaking is working. Anyway there is a second problem with the log.
<5>1 2019-02-02T10:14:37Z hostid CEF:0|Cyber-Ark|Vault|10.5.0000|51|Retrieve File|5|act=Retrieve File suser=PasswordManager fname=Root\Policies\Policy-TEST_MAIL_DualControl.ini dvc= shost=192.169.122.122 dhost= duser= externalId= app= reason= cs1Label="Affected User Name" cs1= cs2Label="Safe Name" cs2=PasswordManagerShared cs3Label="Device Type" cs3= cs4Label="Database" cs4= cs5Label="Other info" cs5= cn1Label="Request Id" cn1= cn2Label="Ticket Id" cn2= msg=CEF:0|Cyber-Ark|VaultMonitor|10.1.0000|45|1420|3|40|1|20|162|23|20|222|133|0

This Event is getting breaked into following events:
EVENT1:
<5>1 2019-02-02T10:14:37Z hostid CEF:0|Cyber-Ark|Vault|10.5.0000|51|Retrieve File|5|act=Retrieve File suser=PasswordManager fname=Root\Policies\Policy-TEST_MAIL_DualControl.ini dvc= shost=192.169.122.122 dhost= duser= externalId= app= reason= cs1Label="Affected User Name" cs1= cs2Label="Safe Name" cs2=PasswordManagerShared cs3Label="Device Type" cs3= cs4Label="Database" cs4= cs5Label="Other info" cs5= cn1Label="Request Id" cn1= cn2Label="Ticket Id" cn2= msg=

EVENT2:
CEF:0|Cyber-Ark|VaultMonitor|10.1.0000|45|1420|3|40|1|20|162|23|20|222|133|0

Why is Splunk splitting this event in two events? Or is Splunk doing everything correct and this are two different events? I actually thought that EVENT2 should be part of the key msg. Anyone here who has a deep understanding of cyberark logs?

Thanks

[woodcock]

Try this in props.conf:

LINE_BREAKER = ([\r\n]+)<\d+>1
SHOULD_LINEMERGE = false

Apply this to the first full version of Splunk that handles the data (HFs or Indexers) and restart all Splunk instances there. When testing, use _index_earliest=-5m with All time timepicker to make sure that you really are looking at newly indexed events that should be effected (older events will stay wrong). Also, if you have done a sourctype-override, you must apply this to the original sourcetype or it will not work.