Deployment Architecture

Indexer not indexing data

afolabia
Path Finder

My Cisco Indexer just stopped indexing new data. Splunk is receiving data from the Syslog server but just not getting index and so nothing is showing in the Cisco Networks apps/addon. I do have an input/output file on my syslog servers through UF that is monitoring the folder with the logs, which is not the problem since i can see current&old logs in the SH. The output is pointing to my HF which forwards the data to the Idexer. I'm running 8.0.1 with 1 server each SH, IDX, DP, HF.
I know it's not indexing cuz my indexer haven't received data for at least a day and there are no errors in the the logs.

Tags (1)
0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @afolabia,
did you ever received logs from that source?
if you received and you don't receive since 12 days, it could be a timestamp problem: you have timestamp in dd/mm/yyyy format and Splunk read it as mm/dd/yyyy, in this way you should find your logs of 12/feb/2020 on 2 dec 2020.
You can solve this problem settinf the Timestamp format in props.conf (TIME_FORMAT=%d/%m/%Y....).

if instead you never received logs, start your debug from the source: the Cisco reach the HF? you can test it using telnet or network tools.

Ciao.
Giuseppe

0 Karma

afolabia
Path Finder

I don't think it's a timestamp since my SH is receiving data but not saving it to the indexer. Also, ports are open cuz I can ping on them from each servers.

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @afolabia,
why do you say that SU is receiving data? data is usually received by Indexers and SHs queryto Indexers for data.
Anyway, if you have an SH tata receives data (not good!), it's a good practice that all the Splunk servers (also SHs) send their logs (usually only internal logs but eventually also others) to Indexers to have only one data repository.

Ciao.
Giuseppe

0 Karma

nickhills
Ultra Champion

Do you have _internal logs from the HF? - Is it forwarding anything, or have both cisco and _internal logs stopped?

If my comment helps, please give it a thumbs up!
0 Karma
Get Updates on the Splunk Community!

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...

Tech Talk | Elevating Digital Service Excellence: The Synergy of Splunk RUM & APM

Elevating Digital Service Excellence: The Synergy of Real User Monitoring and Application Performance ...