Getting Data In

On a Universal Forwarder I did a "splunk clean all", changed some things and started the forwarder, but one of my monitor stanzas is not forwarding...

neiljpeterson
Communicator

The forwarding from this directory was working previous to the clean. My understanding was this was supposed to clean out all indexes including the fishbucket, causing splunk to forget was had already been indexed and reindex it all. There have even been new events generated since clean, which I would certainly expect to be forwarded and indexed, but I am not seeing anything.

Other stanzas, from the same inputs.conf, are working, like performance data.

What am I doing wrong here?

For completeness sake, this is the stanza I am expecting to see data from.

[monitor://C:\Websites\logs\...\*]
disabled = false
sourcetype = app_logs
index = app_logs
0 Karma
1 Solution

neiljpeterson
Communicator

Turns out, this was a role permissions issue.

The user account that was performing the searches did not have "All internal indexes" selected under "Indexes searched by default" I was trying to validate the search by searching by host, not by index. If I search for this index specifically the data comes up.

After adding "All internal indexes" the data comes up in a by host search.

View solution in original post

0 Karma

neiljpeterson
Communicator

Turns out, this was a role permissions issue.

The user account that was performing the searches did not have "All internal indexes" selected under "Indexes searched by default" I was trying to validate the search by searching by host, not by index. If I search for this index specifically the data comes up.

After adding "All internal indexes" the data comes up in a by host search.

0 Karma

sunrise
Contributor

I don't know whether "splunk clean all"command delete fishbuckets or not.
But if you delete fishbucket manually and start UF instance, it will retransfer that monitoring data to Indexer.

cd $SPLUNK_HOME/var/lib/splunk/fishbucket
rm -fR *
0 Karma

neiljpeterson
Communicator

Unfortunately I am on windows... but I did another clean all and it did empty out the fishbucket dir.

0 Karma

MuS
Legend

Hi neiljpeterson,

On a forwarder you should remove the folder $SPLUNK_HOME/var/lib/splunk/fishbucket.
On an indexer splunk clean eventdata -index _fishbucket will do the magic.

cheers, MuS

0 Karma

neiljpeterson
Communicator

This is what I found

 04-01-2014 09:18:42.197 -0500 INFO  TailingProcessor - Parsing configuration stanza: monitor://C:\Websites\logs\...\*.
0 Karma

MuS
Legend

check your forwarders splunkd.log for anything related to tailingprocess regarding this input

0 Karma

neiljpeterson
Communicator

splunk clean all does this. I just did it again as a test.

0 Karma

MuS
Legend

On a forwarder you should remove the folder $SPLUNK_HOME/var/lib/splunk/fishbucket.

0 Karma

neiljpeterson
Communicator

one or the other or both?

0 Karma
Get Updates on the Splunk Community!

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud  In today’s fast-paced digital ...

Observability protocols to know about

Observability protocols define the specifications or formats for collecting, encoding, transporting, and ...

Take Your Breath Away with Splunk Risk-Based Alerting (RBA)

WATCH NOW!The Splunk Guide to Risk-Based Alerting is here to empower your SOC like never before. Join Haylee ...