Hello fellow splunker,
i have a problem with syslog messages from a juniper SRX firewall in a different timezone. I will give a lot of details below "like always" but at first let me give you an short overview of the situation.
The raw Syslog Message from a Firewall in Tokyo when send to my syslog-ng (intermediate forwarder) in germany looks like this:
<14>May 8 11:44:44 juniperfw01 RT_FLOW: RT_FLOW_SESSION_CREATE: session created 10.8.5.63/16744->10.0.22.222/30013 0x0 None 10.8.5.63/16744->10.0.22.222/30013 0x0 N/A N/A N/A N/A 6 KAPFW-Tmp-KP01 FDMZ WAN 76060 N/A(N/A) reth0.0 UNKNOWN UNKNOWN UNKNOWN N/A N/A
You will see there is a timezone information missing per default. Juniper..
Syslog-ng now manipulates the message header (Host, Timestamp) depending on the config. After that the syslog message gets written to the disk where a universal forwarder collects the logfile and sends it to my splunk indexer.
The Message will appear in Splunk like this (different flow but its the header which is important here):
May 11 10:29:00 juniperfw01.contoso.com junos-alg: RT_ALG_WRN_CFG_NEED: MSRPC ALG detected packet from 10.46.0.11/53461 which need extra policy config with UUID:12345678-1234-abcd-ef00-01234567cffb or 'junos-ts-apc-any' to let it pass-through on ASL session
What i really whant is what the syslog collecter integrated in splunk is doing per default (Adding a second header):
May 8 11:44:40 juniperfw01.contoso.com May 8 18:44:40 juniperfw01 RT_FLOW: RT_FLOW_SESSION_DENY: session denied 10.46.30.21/54085->61.205.120.130/123 junos-ntp 17(0) DEFAULT-DENY-LOG(global) FRONT WAN UNKNOWN UNKNOWN N/A(N/A) reth3.0 UNKNOWN policy deny
A Splunk Syslog collector has a few options which are very nice:
1. If an event timestamp does not include a recognizable timezone, the Splunk platform uses the time zone of the host that indexes the event.
2. For example, if an event with field timestamp="2016-01-26 21:58:38.107" is indexed by a Splunk platform instance in GMT-8, it will subtract 8 hours from the timestamp and set the _time of event to: 2016-01-26T13:58:38.000-08:00
3. The Splunk platform normally looks at the text of the event for the timestamp, and by default will select the leftmost recognizable timestamp.
I'm sure there are multiple ways to accomplish this. But my solution needs a few necessary things:
1. I need the original timestamp included (like done by splunk) - this is necessary because the index timestamp and the event time are slightly different. This would make a Flow Tracing to be impossibile.
2. The traffic must be collected from the intermediate forwarder first.
I would love to accomplish this by modify the syslog-ng config which would add another header but i did not find anything about that.
Details about my config:
+-----------------+ +-------------------------------+ +-----------------------+
|Syslog Originator|---->----|intermediate forwarder udp:5517|---->----|Splunk Indexer tcp:9997|
+-----------------+ +-------------------------------+ +-----------------------+
Intermediate Forwarder:
syslog-ng conf:
options {
log_fifo_size(1024);
time_reopen (10);
chain_hostnames(no);
keep_hostname(yes);
use_dns(yes);
use_fqdn(yes);
create_dirs (yes);
group(syslog-ng);
dir_perm(0750);
perm(0640);
};
source s_udp5517 {
udp(
port(5517)
keep-hostname(no)
use-dns(yes)
);
};
destination d_juniper { file( "/data/syslog/forwarder/u5517/$HOST/$YEAR$MONTH$DAY-juniper.log" ); };
# Log
log { source(s_udp5517); destination(d_juniper); };
And the splunk inputs.conf from my universal forwarder running on this intermediate forwarder:
# Juniper
[monitor:///data/syslog/forwarder/u5517/*/*juniper.log]
sourcetype = juniper
index = juniper
disabled = 0
host_segment = 5
... View more