Getting Data In

data sent repeatedly sent via syslog output

bmorgan
Explorer

Every second or so splunk sends all the qualifying events it has see since it started plus any new events. Note: This replication is not showing up in the index, and is reset on a restart of indexers and forwarders (our deployment system restarts both

Here are my configs:

----------- forwarder inputs excerpt -----------  
#Pick up the access logs as well as the Catalina out
[monitor::///somedir/tomcat5.5-logs]
_whitelist = \.(out|txt)$
disabled = false
followTail = 0

[monitor::///somedir/tomcat6.0/logs]
_whitelist = \.(out|txt|log)$
disabled = false
followTail = 0

----------- forwarder props excerpt ----------- 
#define sourcetype for both log types
[source::.../catalina.out]
sourcetype = log4j

[source::.../catalina_log.*.txt]  
# the 5.5 and 6.0 use different extension (I just checked GRC and Native)
sourcetype = catalina_log

[source::.../catalina.*.log]
sourcetype = catalina_log

----------- server props excerpt ----------- 
#send log4j and catalina_log to named SyslogOut Transforms
[log4j]
\# … skipping lines
TRANSFORMS-routing=SyslogOut_log4j

[catalina_log]
TRANSFORMS-routing=SyslogOut_catalina_log

----------- server transforms excerpt -----------  
#send SyslogOut Transforms to the corresponding syslog outputs
[SyslogOut_log4j]
REGEX=.*
DEST_KEY=_SYSLOG_ROUTING
FORMAT=SyslogOutAccess

[SyslogOut_catalina_log]
REGEX=.*
DEST_KEY=_SYSLOG_ROUTING
FORMAT=SyslogOutEvent

----------- server output except -----------  
#send SyslogOut data via syslog Access to one host and log4j to another
[syslog]
disabled = false
indexAndForward = true
defaultGroup=SyslogOutEvent,SyslogOutAccess

[syslog:SyslogOutEvent]
type = tcp
server = MyHost02:516

[syslog:SyslogOutAccess]
type = tcp
server = MyHost01:516



I used the following search to graph the issue:

latest=-2m@m earliest=-32m@m index="_internal" syslog Host* sourcetype="splunkd" _tcp_Kprocessed destIp | timechart span=1m per_minute(tcp_Kprocessed) by destIp

Here is the resulting graph:
alt text

?

Tags (2)
1 Solution

yannK
Splunk Employee
Splunk Employee

The search on metrics.log has to be more precise.
When using metrics, always specify the "group", or apples and oranges may be counted together.

Please try :

index=_internal source=metrics.log group=syslog_connections | eval KBps=tcp_Bps/1024 | timechart avg(KBps) as KBps by destIp span=1h | eval MB=KBps_span/1000 | fields - KBps

View solution in original post

yannK
Splunk Employee
Splunk Employee

The search on metrics.log has to be more precise.
When using metrics, always specify the "group", or apples and oranges may be counted together.

Please try :

index=_internal source=metrics.log group=syslog_connections | eval KBps=tcp_Bps/1024 | timechart avg(KBps) as KBps by destIp span=1h | eval MB=KBps_span/1000 | fields - KBps

View solution in original post

Register for .conf21 Now! Go Vegas or Go Virtual!

How will you .conf21? You decide! Go in-person in Las Vegas, 10/18-10/21, or go online with .conf21 Virtual, 10/19-10/20.