Splunk Search

Logs aren't coming in, forward servers are listed as inactive

happycaptain
Loves-to-Learn

I have very little experience with splunk, and am on a time crunch, so a bit of patience for my ignorance would be awesome. So today I was setting up an enterprise splunk solution for logs. I set up the universal forwarders on a few devices, and set up my indexer on a centos server. I set the receiving port (the default of 9997), set up a new index to sort my data out, and added from the indexer section, which seemed to work, except that I don't actually see any logs. When I get into those operating systems and run a list forward-server command (on linux) it comes back with inactive: ipaddress:port. I tried to see if there was something wrong with my firewall, but everything seems to be open for the 9997 port, I can ping back and forth between systems, I checked my outputs.conf file to make sure that there was the right server address there, and my inputs.conf seem right. I'm beyond clueless after reading all kinds of forums.

I also am having a bit of an issue with space on the system. Splunk tells me that my disk space is at the minimum under opt/splunk8 to deployment, but I don't know what is taking that space. Maybe it's the logs that were sent but never indexed? Where would those end up? (I made the mistake of not setting an index for the monitors that I set up earlier.)

Any help is appreciated, and again, I don't know a whole lot about splunk, so I'm just trying to get it to work... I had plans on integrating splunk into splunk phantom, but that's not happening until splunk works lol.

Thanks!

0 Karma

happycaptain
Loves-to-Learn

I set the minimum down to 500mb (I know its not suggested) and the same thing..

0 Karma

manjunathmeti
Champion

Indexing stops once minimum free disk space is reached for the directories where indexed data is stored. Check if you are storing your indexes in default splunk indexes path (/opt/splunk/var/lib/splunk). If yes, you need to change the path to some other path with sufficient disk space available.

And the default minimum free disk space required is 5000MB.

0 Karma
Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In December, the Splunk Threat Research Team had 1 release of new security content via the Enterprise Security ...

Why am I not seeing the finding in Splunk Enterprise Security Analyst Queue?

(This is the first of a series of 2 blogs). Splunk Enterprise Security is a fantastic tool that offers robust ...

Index This | What are the 12 Days of Splunk-mas?

December 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...