Getting Data In

How to find the path for an unknown data source that is sending data to Splunk?

darknetone
Explorer

How can I tell where data is coming from? I have inherited an old Splunk 5.0.1 Enterprise Infrastructure. I can see data on the Splunk head for a specific (IP) server, however, this data is coming into _main. I got on the Windows box where this data is coming from and I could not see a universal forwarder or syslog implementation despite much searching. I do not know how the data is coming into Splunk, which is a problem since I need the data to go into a different index. This leaves me asking, how is the data coming in? Is there a way to trace events all the way back to the origination point AND know what the path that the data took? I there a way to know what process originated the data on the machine?

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

I'd check a few things:

  • timestamps on the events - maybe they're old and you're chasing ghosts?
  • host and source of the events
  • receiving enabled on the indexer
  • search index=_internal source=*metrics.log* group=tcpin_connections for info around incoming forwarder connections
  • inputs enabled on the indexer
  • if source and inputs don't line up, check for props.conf/transforms.conf rewrites (TRANSFORMS-foo in props.conf)
  • search index=_internal source=*metrics.log* thruput for clues where the indexer thinks it has throughput
0 Karma
Get Updates on the Splunk Community!

Splunk + ThousandEyes: Correlate frontend, app, and network data to troubleshoot ...

 Are you tired of troubleshooting delays caused by siloed frontend, application, and network data? We've got a ...

Splunk Observability for AI

Don’t miss out on an exciting Tech Talk on Splunk Observability for AI!Discover how Splunk’s agentic AI ...

🔐 Trust at Every Hop: How mTLS in Splunk Enterprise 10.0 Makes Security Simpler

From Idea to Implementation: Why Splunk Built mTLS into Splunk Enterprise 10.0  mTLS wasn’t just a checkbox ...