topic Re: timestamp issue in Security

timestamp issue

mcbradford — Thu, 21 Dec 2017 17:37:00 GMT

I have firewall events coming to my syslog-ng server. The firewall events are in Central European Timezone, so when the events are indexed, they are showing up in this time (as expected), so about 5 hours in the future when searching.

I am trying to figure out where to adjust this. I know the props.conf needs adjusted but where? I tried to add a props.conf to local within the app on the UF, but this made no change.

I then tried to add props.conf within local under the firewall app on the indexer, and I stopped getting events all together???

Re: timestamp issue

somesoni2 — Thu, 21 Dec 2017 18:58:21 GMT

So you've UF installed on your syslog-ng server which are sending data directory to Indexer (no intermediate heavy forwarders in between)? If yes, then the props.conf should be updated on Indexers. Can you confirm if you're adding TZ information correctly and within correct sourcetype stanza in your props.conf?

Re: timestamp issue

BenTan — Tue, 29 Sep 2020 17:25:24 GMT

Hi,

In order to understand which props.conf to be configured, it is important to understand the data pipeline, please refer to this link for more information:
https://docs.splunk.com/Documentation/Splunk/7.0.1/Admin/Configurationparametersandthedatapipeline

And to answer your question, if your Splunk architecture only has a Splunk UF installed on your syslog-ng and forward logs directly to Splunk indexer, you will need to configure your timestamp configuration in the indexer's props.conf. In certain situations, if you apply the INDEXED_EXTRACTIONS in your Universal Forwarder's props.conf, you will need to configure timestamp extractions on the same props.conf on UF as well.

If your UF is forwarding data to a Heavy Forwarder before forwarding to the indexer, you will need to configure timestamp configurations on the HF's props.conf.

Lastly, please review your timestamp configurations for the firewall sourcetype. These are the configurations used for timestamp extractions:
- TIME_PREFIX
- TIME_FORMAT
- MAX_TIMESTAMP_LOOKAHEAD
- TZ
- DATETIME_CONFIG

Hope it clears your doubts!

Regards,
Benjamin

Re: timestamp issue

mcbradford — Tue, 29 Sep 2020 17:28:02 GMT

BenTan,

Thanks for taking the time to answer my question. I have 8 firewalls that are in 4 different time zones.

I will only focus on one for now...

raw event

Jan 2 08:40:09 CSG2-MAIN-FW1 1,2018/01/02 08:40:08,011901000724,TRAFFIC,end,1,2018/01/02 08:40:08,10.3.0.63,8.8.8.8,216.85.221.10,8.8.8.8,Standard Outbound Apps,,,ping,vsys1,Trust,Untrust,ethernet1/1,ethernet1/4,Panorama,2018/01/02 08:40:08,149453,6,0,0,0,0,0x500019,icmp,allow,1176,588,588,12,2018/01/02 08:39:57,0,any,0,275058159,0x0,10.0.0.0-10.255.255.255,United States,0,6,6,aged-out,213,0,0,0,,CSG2-MAIN-FW1,from-policy,,,0,,0,,N/A

Since these are all going to a syslog server, I cannot use the host stanza, so I was going to use the source stanza.

The source is:
/opt/syslog-ng/palo_alto/CSG2-MAIN-FW1/2018-01-02/messages.txt

I tried:

[source::*CSG2-MAIN-FW1*]
TZ = PST

but it did not work

I tried:

[source::.../opt/syslog-ng/palo_alto/CSG2-MAIN-FW1/*]
TZ = PST

but it did not work

I am making the changes to props.conf located in:

/opt/splunk/etc/apps/Splunk_TA_paloalto/default

Re: timestamp issue

mcbradford — Tue, 02 Jan 2018 18:31:25 GMT

I figured it out...

[source::/opt/syslog-ng/palo_alto/CSG2-MAIN-FW1/*/messages.txt]
TZ = PST