Getting Data In

How do I debug syslog that dont get into splunk whereas I can see the log when I use tcpdump

ryanf
Engager

Hi everyone,

I’m currently sending vCenter logs via syslog to Splunk and have ensured that the syslog configuration and index name on Splunk is correct. However, the logs still aren’t appearing in the index.

I have tried to tcpdump and I can see the logs arriving at my Splunk instance.

below I attach the syslog configuration and tcpdump result on my splunk instance.

syslog tcpdump.jpg

WhatsApp Image 2024-09-05 at 14.10.09_4daa6736.jpg

What could be the cause of this issue, and what steps should I take to troubleshoot it?

Thanks for any insights!

Labels (1)
0 Karma

JohnEGones
Communicator

A few questions:

Do you have a TA for the logs you are ingesting and are they set-up on all the needed Splunk components, check your DOCs?

Looking at the _internal logs, do you see that Splunk has ingested them?

Can you do a search for a string that exists in your logs across all you indexes and find any responsive logs, in the time you verified that the data was ingested?

 

Also, for syslog data in general it is simpler and more durable to forward data to a syslog server and have a UF monitor relevant files and then you set-up monitoring stanzas per host/data source.

[monitor://var/log...whatever]
whitelist = regex
blacklist = regex
host_segment = as needed
crcSalt = <SOURCE> {as needed}
sourcetype = syslog {or whatever you want}
index = yourIndex

 

Consult also: How the Splunk platform handles syslog data over the UDP network protocol - Splunk Documentation

0 Karma
Get Updates on the Splunk Community!

Message Parsing in SOCK

Introduction This blog post is part of an ongoing series on SOCK enablement. In this blog post, I will write ...

Exploring the OpenTelemetry Collector’s Kubernetes annotation-based discovery

We’ve already explored a few topics around observability in a Kubernetes environment -- Common Failures in a ...

Use ‘em or lose ‘em | Splunk training units do expire

Whether it’s hummus, a ham sandwich, or a human, almost everything in this world has an expiration date. And, ...