Getting Data In

Why has Splunk stopped indexing all log files?

cboard
Explorer

I've recently started using Splunk and it was working fine but at some point seems to have stopped indexing any logs.

I was trying to get a forwarder working so I'm guessing I've done something in trying to get the forwarder working that I've broken the main Splunk.

I've not been able to find anything relevant, everything I've come across (through Google searches) is more of a specific log not being indexed but, in my case, it seems to that every log has stopped.

From the search, I've done index=* and it only comes back with data from the 19th but I know the logs have been updated for today.

Where can I look for any problems?

Thanks

0 Karma

mthq
Engager

I seem to have a similar issue, running a standalone environment for 3 days - first two had events indexed but today I have "No results found." Monitoring single file - /var/log/mhn/mhn-splunk.log

This is college project and I seem to got stuck here. When checking splunkd.log I see:

0-21-2018 16:55:59.240 +0000 ERROR JsonLineBreaker - JSON StreamId:201389110879379108 had parsing error:Unexpected character: '-' - data_source="/var/log/mhn/mhn-splunk.log", data_host="ubuntu-s-2vcpu-4gb-ServerLondon-01", data_sourcetype="MHN"

The source file keeps getting populated.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

You should post a new question.

---
If this reply helps you, Karma would be appreciated.
0 Karma

skoelpin
SplunkTrust
SplunkTrust

You should first look in splunkd to see if there's any errors. You could do this by looking at the file directly under $SPLUNK_HOME/var/log/splunk/splunkd.log or from Splunk Web by putting this in the search bar index=_internal sourcetype=splunkd error

What does your Splunk environment look like? Are you in a distributed environment or standalone system? What changes did you make to the Splunk forwarder?

0 Karma

adonio
Ultra Champion
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

All Work and No Play? Not at .conf26! Unwind at These Evening Events

Between hands-on technical sessions, keynote reveals, and diving into live architectures, .conf26 is packed ...

Join the Hackathon at .conf26 and build a No-Code AI agent

Join us for the AI Agent Buildathon, an in-person, three-hour hands-on Hackathon where you’ll use Splunk Agent ...

Level Up Your Workflow: Mastering Splunk Cloud Management via Terraform

Tech Talk Recap   From Chaos to Control: Scaling Splunk Cloud with Infrastructure as Code Managing apps in ...