zScaler logs via Syslog causing problems with line breaks at rsyslog layer

asridhara — Fri, 17 Sep 2021 04:51:35 GMT

We have configured zScaler logs to send logs to a syslog server, where rsyslog intercepts the feed and writes it to a file. HF is deployed to forward logs from file to Indexers. The setup works fine. However, rsyslog upon receiving the logs does some funny things such as

2021-09-1704:12:27 reason=Allowed event_id=7008750744672403548 pr
2021-09-17T14:12:52.976915+10:00 10.24.12.5 otocol=HTTP_PROXY action=Allowed transactionsize=130 responsesize=65 requestsize=65 urlcategory=Corporate Marketing serverip=52.13.15.12 clienttranstime=0 requestmethod=CONNECTrefererURL="None" useragent=Unknown product=NSS location=

As you can see the feed is broken in to two lines (log length is not causing the break)

Is there an rsyslog config I can use to remediate this issue

The zScaler format we have used is below

%d{yy}-%02d{mth}-%02d{dd}%02d{hh}:%02d{mm}:%02d{ss}\treason=%s{reason}\tevent_id=%d{recordid}\tprotocol=%s{proto}\taction=%s{action}\ttransactionsize=%d{totalsize}\tresponsesize=%d{respsize}\trequestsize=%d{reqsize}\turlcategory=%s{urlcat}\tserverip=%s{sip}\tclienttranstime=%d{ctime}\trequestmethod=%s{reqmethod}\trefererURL="%s{ereferer}"\tuseragent=%s{ua}\tproduct=NSS\tlocation=%s{location}\tClientIP=%s{cip}\tstatus=%s{respcode}\tuser=%s{login}\turl="%s{eurl}"\tvendor=Zscaler\thostname=%s{ehost}\tclientpublicIP=%s{cintip}\tthreatcategory=%s{malwarecat}\tthreatname=%s{threatname}\tfiletype=%s{filetype}\tappname=%s{appname}\tpagerisk=%d{riskscore}\tdepartment=%s{dept}\turlsupercategory=%s{urlsupercat}\tappclass=%s{appclass}\tdlpengine=%s{dlpeng}\turlclass=%s{urlclass}\tthreatclass=%s{malwareclass}\tdlpdictionaries=%s{dlpdict}\tfileclass=%s{fileclass}\tbwthrottle=%s{bwthrottle}\tservertranstime=%d{stime}\tmd5=%s{bamd5}\tcontenttype=%s{contenttype}\ttrafficredirectmethod=%s{trafficredirectmethod}\trulelabel=%s{rulelabel}\truletype=%s{ruletype}\tmobappname=%s{mobappname}\tmobappcat=%s{mobappcat}\tmobdevtype=%s{mobdevtype}\tbwclassname=%s{bwclassname}\tbwrulename=%s{bwrulename}\tthrottlereqsize=%d{throttlereqsize}\tthrottlerespsize=%d{throttlerespsize}\tdeviceappversion=%s{deviceappversion}\tdevicemodel=%s{devicemodel}\tdevicemodel=%s{devicemodel}\tdevicename=%s{devicename}\tdevicename=%s{devicename}\tdeviceostype=%s{deviceostype}\tdeviceostype=%s{deviceostype}\tdeviceosversion=%s{deviceosversion}\tdeviceplatform=%s{deviceplatform}\tclientsslcipher=%s{clientsslcipher}\tclientsslsessreuse=%s{clientsslsessreuse}\tclienttlsversion=%s{clienttlsversion}\tserversslsessreuse=%s{serversslsessreuse}\tservertranstime=%d{stime}\tsrvcertchainvalpass=%s{srvcertchainvalpass}\tsrvcertvalidationtype=%s{srvcertvalidationtype}\tsrvcertvalidityperiod=%s{srvcertvalidityperiod}\tsrvocspresult=%s{srvocspresult}\tsrvsslcipher=%s{srvsslcipher}\tsrvtlsversion=%s{srvtlsversion}\tsrvwildcardcert=%s{srvwildcardcert}\tserversslsessreuse="%s{serversslsessreuse}"\tdlpidentifier="%d{dlpidentifier}"\tdlpmd5="%s{dlpmd5}"\tepochtime="%d{epochtime}"\tfilename="%s{filename}"\tfilesubtype="%s{filesubtype}"\tmodule="%s{module}"\tproductversion="%s{productversion}"\treqdatasize="%d{reqdatasize}"\treqhdrsize="%d{reqhdrsize}"\trespdatasize="%d{respdatasize}"\tresphdrsize="%d{resphdrsize}"\trespsize="%d{respsize}"\trespversion="%s{respversion}"\ttz="%s{tz}"\n

Thanks

Re: zScaler logs via Syslog causing problems with line breaks at rsyslog layer

da — Wed, 20 Apr 2022 23:37:13 GMT

Hello,

I was curious if you received any feedback on the issue or if you found a solution? Thank you.

Re: zScaler logs via Syslog causing problems with line breaks at rsyslog layer

asridhara — Thu, 21 Apr 2022 00:24:23 GMT

Hi,

Unfortunately we still have not found a solution or a workaround for these logs

Re: zScaler logs via Syslog causing problems with line breaks at rsyslog layer

PickleRick — Thu, 21 Apr 2022 04:51:29 GMT

Firstly, we don't know what syslog daemon you use and what is its confiiguration.

Secondly, I'd start with dumping the incoming traffic to see how the zscaler sends those logs.

Re: zScaler logs via Syslog causing problems with line breaks at rsyslog layer

joshuasolman — Fri, 27 Jan 2023 19:33:32 GMT

Did you get a response? We are having the same issue

topic zScaler logs via Syslog causing problems with line breaks at rsyslog layer in Getting Data In

zScaler logs via Syslog causing problems with line breaks at rsyslog layer

Re: zScaler logs via Syslog causing problems with line breaks at rsyslog layer

Re: zScaler logs via Syslog causing problems with line breaks at rsyslog layer

Re: zScaler logs via Syslog causing problems with line breaks at rsyslog layer

Re: zScaler logs via Syslog causing problems with line breaks at rsyslog layer