Getting Data In

Problem reading syslog events

Path Finder

My firewall is using syslog-ng to send logs to my log server over TCP on port 514. In Splunk>>Manager>>Data inputs>>TCP I have one entry, for port 514, which says source=tcp:514x and host=Firewall.

If I set Sourcetype=syslog, one particular log appears with host=2011 instead of host=Firewall.

If instead I set Sourcetype=syslog-ng, most of the time a few events get combined into one.

What should I do?

Tags (1)
0 Karma

Path Finder

Excellent. Thanks for your help.

With the proviso that I don't know how to trigger host=2011, so I will wait for one of those events to happen naturally and see what happens.

...local\props.conf now says:

[syslog-ng]

TIME_FORMAT = %Y:%m:%d-%H:%M:%S

SHOULD_LINEMERGE=false

Is there anything else that should be done when changing the sourcetype from syslog to syslog-ng?

I presume, by the way, that the TCP 514 entry in Data Inputs applies before props.conf. Otherwise [syslog-ng] would not be recognised.

0 Karma

Path Finder

I cannot pretend to read that. But why is it doing it anyway? What is it hoping to achieve?

0 Karma

Legend

The reason you're getting host=2011 when using the "syslog" sourcetype is because Splunk has transforms for that particular sourcetype that sets the host based on log events. Here's the transform that does the job:

[syslog-host]
DEST_KEY = MetaData:Host
REGEX = :\d\d\s+(?:\d+\s+|(?:user|daemon|local.?)\.\w+\s+)*\[?(\w[\w\.\-]{2,})\]?\s
FORMAT = host::$1
0 Karma

Contributor

You might try adding the fllowing stanza to %SPLUNK_HOME\etc\system\local\props.conf

[syslog-ng]
SHOULD_LINEMERGE = False

Bounce splunk and check your events.

0 Karma

Path Finder

In response to JSapienza

Syslog only provides single-line events. All examples below are single lines.

inputs.conf has nothing relevant.

When the sourcetype is syslog, this event is picked up properly:-

<190>2011:10:19-16:45:13 reverseproxy: srcip="211.142.x.x" localip="66.207.x.x" size="0" user="-" host="211.142.x.x" method="HEAD" statuscode="200" time="8772" url="/" server="66.207.x.x" referer="-" cookie="-" set-cookie="-"

and this one gets host=2011:-

<190>2011:10:19-16:45:13 reverseproxy: [Wed Oct 19 16:45:13 2011] [warn] [client 211.142.x.x] proxy: no HTTP 0.9 request (with no host line) on incoming request and preserve host set forcing hostname to be 66.207.x.x for uri /

When the sourcetype is syslog-ng, the following two events get picked up as one:-

<30>2011:10:20-06:49:13 ulogd[4729]: id="2002" severity="info" sys="SecureNet" sub="packetfilter" name="Packet accepted" action="accept" fwrule="17" initf="eth1" outitf="eth2" srcmac="0:1e:79:1a:x.x" dstmac="0:1a:8c:11:x.x" srcip="69.165.x.x" dstip="192.168.x.x" proto="6" length="60" tos="0x00" prec="0x00" ttl="56" srcport="60634" dstport="8000" tcpflags="SYN"

<30>2011:10:20-06:49:14 ulogd[4729]: id="2002" severity="info" sys="SecureNet" sub="packetfilter" name="Packet accepted" action="accept" fwrule="10" initf="eth0" outitf="eth2" srcmac="0:21:9b:8e:x.x" dstmac="0:1a:8c:11:x.x" srcip="192.168.x.x" dstip="192.168.x.x" proto="6" length="48" tos="0x00" prec="0x00" ttl="127" srcport="63563" dstport="9997" tcpflags="SYN"

By the way, the local props.conf says:

[source::tcp:514]

TIME_FORMAT = %Y:%m:%d-%H:%M:S

host=Firewall-props

but I don't believe that is relevant.

0 Karma

Contributor

You might have a line format or line breaking issue. Are these multi-line events ? Paste in a few lines from the raw sylog so we can take a look.
What does the Stanza look like in your inputs.conf ? Check %SPLUNK_HOME%\etc\system\local\inputs.conf .

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!