Getting Data In

On a Universal Forwarder I did a "splunk clean all", changed some things and started the forwarder, but one of my monitor stanzas is not forwarding...

neiljpeterson
Communicator

The forwarding from this directory was working previous to the clean. My understanding was this was supposed to clean out all indexes including the fishbucket, causing splunk to forget was had already been indexed and reindex it all. There have even been new events generated since clean, which I would certainly expect to be forwarded and indexed, but I am not seeing anything.

Other stanzas, from the same inputs.conf, are working, like performance data.

What am I doing wrong here?

For completeness sake, this is the stanza I am expecting to see data from.

[monitor://C:\Websites\logs\...\*]
disabled = false
sourcetype = app_logs
index = app_logs
0 Karma
1 Solution

neiljpeterson
Communicator

Turns out, this was a role permissions issue.

The user account that was performing the searches did not have "All internal indexes" selected under "Indexes searched by default" I was trying to validate the search by searching by host, not by index. If I search for this index specifically the data comes up.

After adding "All internal indexes" the data comes up in a by host search.

View solution in original post

0 Karma

neiljpeterson
Communicator

Turns out, this was a role permissions issue.

The user account that was performing the searches did not have "All internal indexes" selected under "Indexes searched by default" I was trying to validate the search by searching by host, not by index. If I search for this index specifically the data comes up.

After adding "All internal indexes" the data comes up in a by host search.

0 Karma

sunrise
Contributor

I don't know whether "splunk clean all"command delete fishbuckets or not.
But if you delete fishbucket manually and start UF instance, it will retransfer that monitoring data to Indexer.

cd $SPLUNK_HOME/var/lib/splunk/fishbucket
rm -fR *
0 Karma

neiljpeterson
Communicator

Unfortunately I am on windows... but I did another clean all and it did empty out the fishbucket dir.

0 Karma

MuS
SplunkTrust
SplunkTrust

Hi neiljpeterson,

On a forwarder you should remove the folder $SPLUNK_HOME/var/lib/splunk/fishbucket.
On an indexer splunk clean eventdata -index _fishbucket will do the magic.

cheers, MuS

0 Karma

neiljpeterson
Communicator

This is what I found

 04-01-2014 09:18:42.197 -0500 INFO  TailingProcessor - Parsing configuration stanza: monitor://C:\Websites\logs\...\*.
0 Karma

MuS
SplunkTrust
SplunkTrust

check your forwarders splunkd.log for anything related to tailingprocess regarding this input

0 Karma

neiljpeterson
Communicator

splunk clean all does this. I just did it again as a test.

0 Karma

MuS
SplunkTrust
SplunkTrust

On a forwarder you should remove the folder $SPLUNK_HOME/var/lib/splunk/fishbucket.

0 Karma

neiljpeterson
Communicator

one or the other or both?

0 Karma
Get Updates on the Splunk Community!

Upcoming Webinar: Unmasking Insider Threats with Slunk Enterprise Security’s UEBA

Join us on Wed, Dec 10. at 10AM PST / 1PM EST for a live webinar and demo with Splunk experts! Discover how ...

.conf25 technical session recap of Observability for Gen AI: Monitoring LLM ...

If you’re unfamiliar, .conf is Splunk’s premier event where the Splunk community, customers, partners, and ...

A Season of Skills: New Splunk Courses to Light Up Your Learning Journey

There’s something special about this time of year—maybe it’s the glow of the holidays, maybe it’s the ...