Netscreen Firewall Syslog Input not line breaking

Dragonnet — Mon, 25 Oct 2010 23:43:12 GMT

I know this problem has already been addressed but I cannot resolve the problem using the directions in 'Juniper Netscreen TCP Syslog messages not breaking properly'

I have added the entries in the two conf files as listed there

You also might need to set a line breaker defined in your sourcetype as

follows $SPLUNK_HOME/etc/system/local/inputs.conf

*[tcp://9999]
sourcetype = juniper_syslog_stuff
And In your $SPLUNK_HOME/etc/system/local/props.conf
[junpiper_syslog_stuff]
LINE_BREAKER=(\x00)<\d+>
SHOULD_LINEMERGE=False*

And changed the tcp to 1468 which is the port I am using. This does not work and I still get the lines added together. Looking at the actual log output in splunk I can see that the line break is different in my system \x00<133> but I have tried every possible permutation of that in the LINE_BREAKER expression and I cannot get it to work

I am sure I am just being a muppet but some help would be appreciated

25/10/2010 19:55:00.000

zone=Untrust dst zone=Trust action=Permit sent=887 rcvd=529 src=93.189.29.26 dst=212.21.101.220 src_port=52584 dst_port=80 src-xlated ip=93.189.29.26 port=52584 dst-xlated ip=212.21.101.220 port=80 session_id=1824 reason=Close - TCP FIN\x00<133>ssg5-serial: NetScreen device_id=0162102007000604 [Root]system-notification-00257(traffic): start_time="2010-10-25 17:19:56" duration=1 policy_id=6 service=http proto=6 src zone=Untrust dst zone=Trust action=Permit sent=1312 rcvd=12852 src=93.189.29.26 dst=212.21.101.220 src_port=52588 dst_port=80 src-xlated ip=93.189.29.26 port=52588 dst-xlated ip=212.21.101.220 port=80 session_id=3203 reason=Close - TCP FIN\x00<133>ssg5-serial: NetScreen device_id=0162102007000604 [Root]system-notification-00257(traffic): start_time="2010-10-25 17:19:53" duration=4 policy_id=9 service=dns proto=17 src zone=Untrust dst zone=Trust action=Permit sent=83 rcvd=215 src=195.96.0.4 dst=212.21.101.193 src_port=47092 dst_port=53 src-xlated ip=195.96.0.4 port=47092 dst-xlated ip=212.21.101.193 port=53 session_id=3973 reason=Close - RESP\x00<133>ssg5-serial: NetScreen device_id=0162102007000604 [Root]system-notification-00257(traffic): start_time="2010-10-25 17:19:55" duration=2 policy_id=6 service=http proto=6 src zone=Untrust dst zone=Trust action=Permit sent=3196 rcvd=897 src=88.97.218.190 dst=212.21.101.217 src_port=64015 dst_port=80 src-xlated ip=88.97.218.190 port=64015 dst-xlated ip=212.21.101.217 port=80 session_id=3983 reason=Close - TCP FIN\x00<133>ssg5-serial: NetScreen device_id=0162102007000604 [Root]system-notification-00257(traffic): start_time="2010-10-25 17:19:55" duration=2 policy_id=9 service=dns proto=17 src zone=Untrust dst zone=Trust action=Permit sent=97 rcvd=226 src=195.252.72.67 dst=212.21.101.193 src_port=32768 dst_port=53 src-xlated ip=195.252.72.67 port=32768 dst-xlated ip=212.21.101.193 port=53 session_id=2524 reason=Close - RESP\x00<133>ssg5-serial: NetScreen device_id=0162102007000604 [Root]system-notification-00257(traffic): start_time="2010-10-25 17:19:53" duration=4 policy_id=6 service=http proto=6 src zone=Untrust dst zone=Trust action=Permit sent=7156 rcvd=40647 src=79.173.154.37 dst=212.46.132.46 src_port=53796 dst_port=80 src-xlated ip=79.173.154.37 port=53796 dst-xlated ip=212.46.132.46 port=80 session_id=2075 reason=Close - TCP FIN\x00<133>ssg5-serial: NetScreen device_id=0162102007000604 [Root]system-notification-00257(traffic): start_time="2010-10-25 17:19:49" duration=8 policy_id=6 service=http proto=6 src zone=Untrust dst zone=Trust action=Permit sent=3679 rcvd=62167 src=79.173.154.37 dst=212.46.132.46 src_port=53792 dst_port=80 src-xlated ip=79.173.154.37 port=53792 dst-xlated ip=212.46.132.46 port=80 session_id=3941 reason=Close - TCP FIN\x00<133>ssg5-serial: NetScreen device_id=0162102007000604 [Root]system-notification-00257(traffic): start_time="2010-10-25 17:19:55" duration=2 policy_id=9 service=dns proto=17 src zone=Untrust dst zone=Trust action=Permit sent=97 rcvd=178 src=203.135.190.6 dst=212.21.101.193 src_port=5413 dst_port=53 src-xlated ip=203.135.190.6 port=5413 dst-xlated ip=212.21.101.193 port=53 session_id=2896 reason=Close - RESP\x00<133>ssg5-serial: NetScreen device_id=0162102007000604 [Root]system-notification-00257(traffic): start_time="2010-10-25 17:19:53" duration=4 policy_id=6 service=https proto=6 src zone=Untrust dst zone=Trust action=Permit sent=194 rcvd=130 src=217.147.95.3 dst=212.46.132.46 src_port=34187 dst_port=443 src-xlated ip=217.147.95.3 port=34187 dst-xlated ip=212.46.132.46 port=443 session_id=3287 reason=Close - AGE OUT\x00<133>ssg5-serial: NetScreen device_id=0162102007000604 [Root]system-notification-00257(traffic): start_time="2010-10-25 17:19:53" duration=4 policy_id=6 service=https proto=6 src zone=Untrust dst zone=Trust action=Permit sent=194 rcvd=130 src=217.147.95.3 dst=212.46.132.46 src_port=34184 dst_port=443 src-xlated ip=217.147.95.3 port=34184 dst-xlated ip=212.46.132.46 port=443 session_id=2261 reason=Close - AGE OUT\x00<133>ssg5-serial: NetScreen device_id=0162102007000604 [Root]system-notification-00257(traffic): start_time="2010-10-25 17:19:53" duration=4 policy_id=6 service=https proto=6 src zone=Untrust dst zone=Trust action=Permit sent=194 rcvd=130 src=217.147.95.3 dst=212.46.132.46 src_port=34179 dst_port=443 src-xlated ip=217.147.95.3 port=34179 dst-xlated ip=212.46.132.46 port=443 session_id=3654 reason=Close - AGE OUT\x00<133>ssg5-serial: NetScreen device_id=0162102007000604 [Root]system-notification-00257(traffic): start_time="2010-10-25 17:19:53" duration=4 policy_id=6 service=https proto=6 src zone=Untrust dst zone=Trust action=Permit sent=194 rcvd=130 src=217.147.95.3 dst=212.46.132.46 src_port=34178 dst_port=443 src-xlated ip=217.147.95.3 port=34178 dst-xlated ip=212.46.132.46 port=443 session_id=3466 reason=Close - AGE OUT\x00<133>ssg5-serial: NetScreen device_id=0162102007000604 [Root]system-notification-00257(traffic): start_time="2010-10-25 17:19:53" duration=4 policy_id=6 service=https proto=6 src zone=Untrust dst zone=Trust action=Permit sent=194 rcvd=130 src=217.147.95.3 dst=212.46.132.46 src_port=34173 dst_port=443 src-xlated ip=217.147.95.3 port=34173 dst-xlated ip=212.46.132.46 port=443 session_id=2486 reason=Close - AGE OUT\x00<133>ssg5-serial: NetScreen device_id=0162102007000604 [Root]system-notification-00257(traffic): start_time="2010-10-25 17:19:53" duration=4 policy_id=6 service=https proto=6 src zone=Untrust dst zone=Trust action=Permit sent=194 rcvd=130 src=217.147.95.3 dst=212.46.132.46 src_port=34165 dst_port=443 src-xlated ip=217.147.95.3 port=34165 dst-xlated ip=212.46.132.46 port=443 session_id=2716 reason=Close - AGE OUT\x00<133>ssg5-serial: NetScreen device_id=0162102007000604 [Root]system-notification-00257(traffic): start_time="2010-10-25 17:19:55" duration=2 policy_id=6 service=https proto=6 src zone=Untrust dst zone=Trust action=Permit sent=194 rcvd=130 src=217.147.95.3 dst=212.46.132.46 src_port=34254 dst_port=443 src-xlated ip=217.147.95.3 port=34254 dst-xlated ip=212.46.132.46 port=443 session_id=4043 reason=Close - AGE OUT\x00<133>ssg5-serial: NetScreen device_id=0162102007000604 [Root]system-notification-00257(traffic): start_time="2010-10-25 17:19:54" duration=3 policy_id=6 service=https proto=6 src zone=Untrust dst zone=Trust action=Permit sent=194 rcvd=130 src=217.147.95.3 dst=212.46.132.46 src_port=34249 dst_port=443 src-xlated ip=217.147.95.3 port=34249 dst-xlated ip=212.46.132.46 port=443 session_id=2318 reason=Close - AGE OUT\x00<133>ssg5-serial: NetScreen device_id=0162102007000604 [Root]system-notification-00257(traffic): start_time="2010-10-25 17:19:54" duration=3 policy_id=6 service=https proto=6 src zone=Untrust dst zone=Trust action=Permit sent=194 rcvd=130 src=217.147.95.3 dst=212.46.132.46 src_port=34248 dst_port=443 src-xlated ip=217.147.95.3 port=34248 dst-xlated ip=212.46.132.46 port=443 session_id=2625 reason=Close - AGE OUT\x00<133>ssg5-serial: NetScreen device_id=0162102007000604 [Root]system-notification-00257(traffic): start_time="2010-10-25 17:19:54" duration=3 policy_id=6 service=https proto=6 src zone=Untrust dst zone=Trust action=Permit sent=194 rcvd=130 src=217.147.95.3 dst=212.46.132.46 src_port=34241 dst_port=443 src-xlated ip=217.147.95.3 port=34241 dst-xlated ip=212.46.132.46 port=443 session_id=2467 reason=Close - AGE OUT\x00<133>ssg5-serial: NetScreen device_id=0162102007000604 [Root]system-notification-00257(traffic): start_time="2010-10-25 17:19:54" duration=3 policy_id=6 service=https proto=6 src zone=Untrust dst zone=Trust action=Permit sent=194 rcvd=130 src=217.147.95.3 dst=212.46.132.46 src_port=34226 dst_port=443 src-xlated ip=217.147.95.3 port=34226 dst-xlated ip=212.46.132.46 port=443 session_id=3831 reason=Close - AGE OUT\x00<133>ssg5-serial: NetScreen device_id=0162102007000604 [Root]system-notification-00257(traffic): start_time="2010-10-25 17:19:54" duration=3 policy_id=6 service=https proto=6 src zone=Untrust dst zone=Trust action=Permit sent=194 rcvd=130 src=217.147.95.3 dst=212.46.132.46 src_port=34218 dst_port=443 src-xlated ip=217.147.95.3 port=34218 dst-xlated ip=212.46.132.46 port=443 session_id=2672 reason=Close - AGE OUT\x00<133>ssg5-serial: NetScreen device_id=0162102007000604 [Root]system-notification-00257(traffic): start_time="2010-10-25 17:19:54" duration=3 policy_id=6 service=https proto=6 src zone=Untrust dst zone=Trust action=Permit sent=194 rcvd=130 src=217.147.95.3 dst=212.46.132.46 src_port=34210 dst_port=443 src-xlated ip=217.147.95.3 port=34210 dst-xlated ip=212.46.132.46 port=443 session_id=2063 reason=Close - AGE OUT\x00<133>ssg5-serial: NetScreen device_id=0162102007000604 [Root]system-notification-00257(traffic): start_time="2010-10-25 17:19:53" duration=4 policy_id=6 service=https proto=6 src zone=Untrust dst zone=Trust action=Permit sent=194 rcvd=130 src=217.147.95.3 dst=212.46.132.46 src_port=34203 dst_port=443 src-xlated ip=217.147.95.3 port=34203 dst-xlated ip=212.46.132.46 port=443 session_id=3326 reason=Close - AGE OUT\x00<133>ssg5-serial: NetScreen device_id=0162102007000604 [Root]system-notification-00257(traffic): start_time="2010-10-25 17:19:53" duration=4 policy_id=6 service=https proto=6 src zone=Untrust dst zone=Trust action=Permit sent=194 rcvd=130 src=217.147.95.3 dst=212.46.132.46 src_port=34197 dst_port=443 src-xlated ip=217.147.95.3 port=34197 dst-xlated ip=212.46.132.46 port=443 session_id=3637 reason=Close - AGE OUT\x00<133>ssg5-serial: NetScreen device_id=0162102007000604 [Root]system-notification-00257(traffic): start_time="2010-10-25 17:19:53" duration=4 policy_id=6 service=https proto=6 src zone=Untrust dst zone=Trust action=Permit sent=194 rcvd=130 src=217.147.95.3 dst=212.46.132.46 src_port=34194 dst_port=443 src-xlated ip=217.147.95.3 port=34194 dst-xlated ip=212.46.132.46 port=443 session_id=3175 reason=Close - AGE OUT\x00<133>ssg5-serial: NetScreen device_id=0162102007000604 [Root]system-notification-00257(traffic): start_time=

Re: Netscreen Firewall Syslog Input not line breaking

Dragonnet — Tue, 26 Oct 2010 16:56:59 GMT

Sorted.

LINE_BREAKER=(<133>)

Works just fine, It does leave an entry of \x100 behind every reason= field and one day I will work out how to get rid of that.

If anyone can tell me how to kill that I would be grateful

Re: Netscreen Firewall Syslog Input not line breaking

dwaddle — Wed, 27 Oct 2010 20:54:51 GMT

The <133> is the syslog "facility" and "level" encoded into the message. The "\x00" is probably a null termination at the end of each message. One of my questions would be is the "\x00" actually a 0x00 byte, or the bytes 0x5C 0x78 0x30 0x30? A wireshark capture would tell you for sure.

Once you know, you could update the LINE_BREAKER to eat it as well, or use a SEDCMD in props.conf to filter it out.

Also, the <133> may not always have the digits "133" in it. Generalizing your regexp to 1 to 3 digits inside the <>'s would make it work if the Netscreen sends a different syslog level for some reason.

LINE_BREAKER=(<\d{1,3}>)

topic Netscreen Firewall Syslog Input not line breaking in Getting Data In

Netscreen Firewall Syslog Input not line breaking

Re: Netscreen Firewall Syslog Input not line breaking

Re: Netscreen Firewall Syslog Input not line breaking