Anypoint studio to splunk logs missing?

[karthi2809]

Hi Team,

I am connecting any point studio with Splunk using HEC. The logs are forwarding but some of the logs are missing in Splunk but it is presented in Any point studio logs. How to troubleshoot the issue?

Thanks,

Karthi

[KendallW]

Hi @karthi2809 There could either be a network issue, some unknown routing rules, or there may be a timestamp parsing issue. 

Unless you have some external network monitoring or logging when the connection drops/fails to reach the endpoint, it can be tricky to pinpoint the cause of HEC issues. Do you have any such logging in the downstream Anypoint system?

So, to narrow down the issue, first check the Splunk _internal logs for any obvious timestamp parsing issues:

index=_internal splunk_server=* source=*splunkd.log* component=DateParserVerbose data_sourcetype=<your sourcetype>

If there are no obvious timestamp parsing issues returned, next check whether a single event from the missing time period has actually been indexed to Splunk. It could be that the timestamp extraction step has actually been bypassed, so the events have been indexed under a different timestamp, meaning they won't show up in your search. Search (using time period "All time") :

index=yourindex sourcetype=yoursourcetype _raw="full raw event copied from Anypoint logs"