HI at all I have a very strange thing:
I'm using Splunk 7.0.0 in all systems.
I have two Heavy Forwarders with a Load Balancer Netscaler in front of, that receive syslogs and send them to two Indexers.
There a Cisco ACS that sends syslogs to my HFs and it was running.
Some time ago there was an upgrade of Cisco ACS so from that moment I don't receive more events.
Checking Splunk logs I found that I have in _internal from the HFs the following logs:
11-22-2017 15:24:14.423 +0100 INFO Metrics - group=udpin_connections, xx.xx.xx.xx:514, sourcePort=514, _udp_bps=71.82, _udp_kbps=0.07, _udp_avg_thruput=0.08, _udp_kprocessed=27.53, _udp_eps=0.10
.
11-22-2017 15:24:14.420 +0100 INFO Metrics - group=per_host_thruput, series="xx.xx.xx.xx", kbps=0.0650822688668127, eps=0.06451524038685016, kb=2.017578125, ev=2, avg_age=31536011.5, max_age=31536023
Where xx.xx.xx.xx is the HFs IP address.
And this means that HFs are receiving logs, but they aren't indexed.
Anyone can help me to understand what's happening?
Bye.
Giuseppe
I found the problem: I don't know why Splunk didn't use the first timestamp in stead used the second interpreting last number of IP address as year, so timestamp was the highlighted
Nov 22 16:35:23 xx.xx.xx.16 Nov 22 16:35:31 CRM-ACS-A1 CSCOacs_Failed_Attempts 0000991147 2 0 2017-11-22 16:35:31.951 +01:00 ....
and event was indexed with timestamp
2016-11-22 16:53:31
This could be useful for others.
Bye.
Giuseppe
I found the problem: I don't know why Splunk didn't use the first timestamp in stead used the second interpreting last number of IP address as year, so timestamp was the highlighted
Nov 22 16:35:23 xx.xx.xx.16 Nov 22 16:35:31 CRM-ACS-A1 CSCOacs_Failed_Attempts 0000991147 2 0 2017-11-22 16:35:31.951 +01:00 ....
and event was indexed with timestamp
2016-11-22 16:53:31
This could be useful for others.
Bye.
Giuseppe
There an upgrade on my problem:
I found that events are indexed but with a wrong year date Nov 22 2016 !
This is the indexed log
Nov 22 16:35:23 xx.xx.xx.xx Nov 22 16:35:31 CRM-ACS-A1 CSCOacs_Failed_Attempts 0000991147 2 0 2017-11-22 16:35:31.951 +01:00 ....
But the problem is now: why Splunk read a wrong year?
Year isn't declared in logs but both Indexers and HFs system date are correct (2017).
Anyone has an idea where search the problem?
Bye.
Giuseppe