Getting Data In

Event Breaking events Zscaler

CarolinaHB
Explorer

Hello, 

I need to event break the following events, but they have a different date format. At the beginning, only at the end, it ends with the 'keyprotectiontype' field, which sometimes has 'NA'. Additionally, it must always have the 'reason' field at the beginning.

 

Apr 2 22:18:08 04-02 22: 17:39#011reason=Allowed#011event_id=7353490211603742721#011protocol=HTTP#011action=Allowed#011transactionsize=345241#011responsesize=344806#011requestsize=435#011urlcategory=Operating System and Software Updates#011serverip=92.123.121.156#011requestmethod=GET#011refererURL=None#011useragent=Microsoft BITS/7.8#011product=NSS#011location=Road Warrior#011ClientIP=12.2.11.10#011status=206#011user=lvtorrea@lula.com.es#011url=2.tlu.dl.delivery.mp.microsoft.com/filestreamingservice/files/20c818db-67ad-44d4-8409-4d9dd7986af1?P1=1712128627&P2=404&P3=2&P4=OEkaO+U5XHKvf+lM41oEFDeIKRAD9S6SWgch3BSzA/yxusk1LA44YVdjNg94soDh+D8bYKjPHLpS4296pI6Tcw==#011vendor=Zscaler#011hostname=dkdkdk #011clientpublicIP=1.111.120.11#011threatcategory=None#011threatname=None#011filetype=None#011appname=General Browsing#011pagerisk=0#011threatseverity=None#011department=XXXXX (1422)#011urlsupercategory=Information Technology#011appclass=General Browsing#011dlpengine=None#011urlclass=Business Use#011threatclass=None#011dlpdictionaries=None#011fileclass=None#011bwthrottle=NO#011contenttype=application/octet_stream#011unscannabletype=None#011devicehostname=MAA#011deviceowner=lvtorrea#011keyprotectiontype= Software Protection#0122024-04-02 22:17:39#011reason=Allowed#011event_id=7353490211788947457#011protocol=SSL#011action=Allowed#011transactionsize=9568#011responsesize=4934#011requestsize=4634#011urlcategory=Microsoft_WVD_URL#011serverip=20.189.173.26#011requestmethod=NA#011refererURL=None#011useragent=Unknown#011product=NSS#011location=Road Warrior#011ClientIP=192.168.0.147#011status=NA#011user=jlvaldezo@lula.com.es#011url=us-v10c.events.data.microsoft.com#011vendor=Zscaler#011hostname=dkdkdk#011clientpublicIP=1.19.72.10#011threatcategory=None#011threatname=None#011filetype=None#011appname=General Browsing#011pagerisk=0#011threatseverity=None#011department=xxxxxxx MANAGEMENT#011urlsupercategory=User-defined#011appclass=General Browsing#011dlpengine=None#011urlclass=Bandwidth Loss#011threatclass=None#011dlpdictionaries=None#011fileclass=None#011bwthrottle=NO#011contenttype=Other#011unscannabletype=None#011devicehostname=KDKD#011deviceowner=jlvaldezo#011keyprotectiontype=N/A#012202

 

 

Can you help me?

Labels (1)
Tags (1)
0 Karma
1 Solution

ashurack_qmulos
Explorer

I'm by no means an rsyslog guru but ran into it recently.  There may be a better way to solve this but the quick fix was to turn off both supportOctetCountedFraming (input) and escapeControlCharacterTab (global).

 

$EscapeControlCharacterTab off

[...other config...]

input(type="imtcp" port="<port>" name="<name>" ruleset="<ruleset>" supportOctetCountedFraming="off")

 

 

View solution in original post

scelikok
SplunkTrust
SplunkTrust

Hi @CarolinaHB,

I noticed that "#012" exists in your event as end of event marker.

You can use below as a line breaker;

LINE_BREAKER=#012()

 

 

If this reply helps you an upvote and "Accept as Solution" is appreciated.
0 Karma

saranvishva
Explorer

It will work but extract with "#012 ".

0 Karma

PickleRick
SplunkTrust
SplunkTrust

#012 here is Line Feed character (\n) escaped by rsyslog (as well as #011 is an escaped \t).

Question is why it's escaped. It would be easiest if the events were broken by rsyslog.

ashurack_qmulos
Explorer

I'm by no means an rsyslog guru but ran into it recently.  There may be a better way to solve this but the quick fix was to turn off both supportOctetCountedFraming (input) and escapeControlCharacterTab (global).

 

$EscapeControlCharacterTab off

[...other config...]

input(type="imtcp" port="<port>" name="<name>" ruleset="<ruleset>" supportOctetCountedFraming="off")

 

 

PickleRick
SplunkTrust
SplunkTrust

I don't know what is the original rsyslog configuration (and even where that rsyslog is :-)).

But your option will only make the tab character (un)escaped.

The general option for escaping characters is parser.escapeControlCharactersOnReceive

0 Karma

ashurack_qmulos
Explorer

Setting supportOctetCountedFraming="off" on the input fixes newlines being encoded to #012.

 

0 Karma

PickleRick
SplunkTrust
SplunkTrust

It does this effect but it works a bit differently. With octet counted option rsyslog split the input connection (because it works with tcp input only) based on the length of the event which should be given at the beginning of the event if I remember correctly. So the main problem is not that the new lines are encoded as #012 but that the events are not split at newline characters as they should be. If you turn of the octet counted option, the incoming tcp stream is broken into separate events on newline character so there is nothing to encode as #012 anymore.

Get Updates on the Splunk Community!

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

 (view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...