Indicator "ingestion_latency_gap_multiplier" exceeded configured value. The observed value is 98344.
Is this normal? We have Splunk Universal Forwarder installed on all systems and forwarding Event logs. Is there any way to improve ingestion latency?
Ingestion latency is comparing the _time of the event vs the _indextime of the event.
Firstly you need to look at your data to understand what is producing that data and how it is generating the timestamps for that data.
You also need to understand what Splunk does re those timestamps in the data and whether it is getting the correct event timestamp into its _time field.
If you are having performance issues, then all events should be suffering the same issue, but if it's only some events/sourcetypes, then it will be a data issue or a forwarding issue.
Have you check that all your hosts have correct time source and all TZ definitions have correct values?
Ingestion latency is comparing the _time of the event vs the _indextime of the event.
Firstly you need to look at your data to understand what is producing that data and how it is generating the timestamps for that data.
You also need to understand what Splunk does re those timestamps in the data and whether it is getting the correct event timestamp into its _time field.
If you are having performance issues, then all events should be suffering the same issue, but if it's only some events/sourcetypes, then it will be a data issue or a forwarding issue.
Thank you for your reply. I will dig in further and see. I'm not the primary Splunk admin on our network, but I'll relay this information to him.