topic Re: i feel dumb, but I see no data when I search in splunk. in Splunk Search

i feel dumb, but I see no data when I search in splunk.

lennyburns — Wed, 08 Feb 2012 00:29:46 GMT

I created 8 data inputs, each one is supposed to tail log files mathing a certain whitelist regex.
These inputs see the files (my preview worked and i see the # of files in the data inputs page.

The Application im using for the inputs is SEARCH.

When I go to the SEARCH app, I type a word I KNOW is in the logs, and I get nothing.
I type * and I get nothing.

I'm clearly missing something basic.

This wasn't this hard when i did this a few years ago.

Re: i feel dumb, but I see no data when I search in splunk.

mikelanghorst — Wed, 08 Feb 2012 00:33:52 GMT

Could you post the relevant stanza from inputs.conf? Are you sending the data to an index you're then not searching for?

Re: i feel dumb, but I see no data when I search in splunk.

lennyburns — Mon, 28 Sep 2020 10:24:17 GMT

Here is a sample of a few...

[monitor://\XXX-vdi-csa01\c$\Documents and Settings\All Users\Application Data\VMware\VDM\logs]
disabled = false
followTail = 1
host = XXX-VDI-CSA01_DEBUG-LOG
sourcetype = vmware_view_connection_broker_debug_log
index = vmware_view_brokers
blacklist = log-.txt
whitelist = debug-.txt

[monitor://\XXX-vdi-csb01\c$\Documents and Settings\All Users\Application Data\VMware\VDM\logs]
disabled = false
followTail = 1
host = XXX-VDI-CSB01_LOG
sourcetype = vmware_view_connection_server_log
index = vmware_view_brokers
blacklist = debug-.txt
whitelist = log-.txt

Re: i feel dumb, but I see no data when I search in splunk.

mikelanghorst — Mon, 28 Sep 2020 10:24:19 GMT

Here's the likely reason:
index = vmware_view_brokers

The summary app and by default your role will only search index=main by default. Simply add:
index=vmware_view_brokers

to your search and you should see the data just fine

You can change the default role that is searched under Manager> User Roles > your role.

Re: i feel dumb, but I see no data when I search in splunk.

lennyburns — Wed, 08 Feb 2012 01:05:34 GMT

how can i make the default index the brokers index? i plan to use splunk for nothign else but watching brokers.

Re: i feel dumb, but I see no data when I search in splunk.

lennyburns — Mon, 28 Sep 2020 11:21:32 GMT

so, i did that...
index="vmware_view_brokers" WARN

and i dont find any occurance of WARN

Re: i feel dumb, but I see no data when I search in splunk.

hexx — Wed, 08 Feb 2012 01:23:44 GMT

Usually, when I want to check that some data that I set to be indexed is present, I am as loose as possible with my search terms.

Typically, I will search over all time for :

index=* OR index=_* <TERM>

...where is a string (unique, if possible) naturally delimited in the source by delimiters such as white spaces, colons, slashes, etc.

Oh and there's no reason to feel dumb.

Re: i feel dumb, but I see no data when I search in splunk.

mikelanghorst — Wed, 08 Feb 2012 01:56:24 GMT

What about with just the index specified, no other strings to filter.

Re: i feel dumb, but I see no data when I search in splunk.

gkanapathy — Wed, 08 Feb 2012 03:56:46 GMT

also make sure the indexes are actually created.

Re: i feel dumb, but I see no data when I search in splunk.

lguinn2 — Wed, 08 Feb 2012 05:55:23 GMT

You can search with just the index specified. The search will return all events in the index (constrained by your time range selection).

Re: i feel dumb, but I see no data when I search in splunk.

lennyburns — Wed, 08 Feb 2012 14:39:31 GMT

I get no results. 😞

Re: i feel dumb, but I see no data when I search in splunk.

lennyburns — Mon, 28 Sep 2020 11:21:37 GMT

I think this might be the problem..

In the indexes section, the vmware_view_brokers index is only at 1MB and EVENT COUNT is 0, and the rest is N/A.

argh.

Re: i feel dumb, but I see no data when I search in splunk.

mikelanghorst — Mon, 28 Sep 2020 11:21:43 GMT

Hmm, this might take some more investigating. What's the output of:
./splunk bin splunk cmd btool indexes list --debug vmware_view_brokers

Alternatively, if you have access to the IRC channel you could pop in there, then we could look deeper and then update this Question with the outcome.

Re: i feel dumb, but I see no data when I search in splunk.

lennyburns — Wed, 08 Feb 2012 18:35:16 GMT

so i just run this command?

Re: i feel dumb, but I see no data when I search in splunk.

lennyburns — Mon, 28 Sep 2020 11:21:48 GMT

C:\Program Files\Splunk\bin>btool indexes list --debug vmware_view_brokers
search [vmware_view_brokers]
system assureUTF8 = false
system blockSignSize = 0
system blockSignatureDatabase = _blocksignature
search coldPath = $SPLUNK_DB\vmware_view_brokers\colddb
system coldToFrozenDir =
system coldToFrozenScript =
system compressRawdata = true
system defaultDatabase = main
system enableOnlineBucketRepair = true
system enableRealtimeSearch = true
system frozenTimePeriodInSecs = 188697600
search homePath = $SPLUNK_DB\vmware_view_brokers\db

Re: i feel dumb, but I see no data when I search in splunk.

lennyburns — Wed, 08 Feb 2012 18:46:26 GMT

system indexThreads = auto
system maxBloomBackfillBucketAge = 30d
system maxConcurrentOptimizes = 3
system maxDataSize = auto
system maxHotBuckets = 3
system maxHotIdleSecs = 0
system maxHotSpanSecs = 7776000
system maxMemMB = 5
system maxMetaEntries = 1000000
system maxRunningProcessGroups = 20
system maxRunningProcessGroupsLowPriority = 1
system maxTotalDataSizeMB = 500000
system maxWarmDBCount = 300
system memPoolMB = auto
system minRawFileSyncSecs = disable
system partialServiceMetaPeriod = 0

Re: i feel dumb, but I see no data when I search in splunk.

lennyburns — Mon, 28 Sep 2020 11:21:50 GMT

system quarantineFutureSecs = 2592000
system quarantinePastSecs = 77760000
system rawChunkSizeBytes = 131072
system rotatePeriodInSecs = 60
system serviceMetaPeriod = 25
system suppressBannerList =
system sync = 0
system syncMeta = true
search thawedPath = $SPLUNK_DB\vmware_view_brokers\thaweddb
system throttleCheckPeriod = 15

Re: i feel dumb, but I see no data when I search in splunk.

sgarvin55 — Mon, 28 Sep 2020 11:22:39 GMT

Ok, after trying to figure out why the wildcard "" wouldn't work in your inputs.conf, it was determined that it can't be used in the Splunk data preview or the stanza in inputs.conf.
EX:[monitor:///Users/username/Desktop/tmp/`debug-.txt`] Although the docs say differently. I tried and I got the same result.

With that said, I had to create these configurations to get it to work:

inputs.conf
[monitor:///Users/username/Desktop/tmp] <---location of debug.txt and log.txt logs
whitelist = (log-.*\.txt|debug-.*\.txt)

props.conf
[source::.../tmp/debug...txt]
sourcetype = debug
TRANSFORMS-index = choose_debug_index

[source::.../tmp/log...txt]
sourcetype = log
TRANSFORMS-index = choose_log_index

transforms.conf
[choose_debug_index]
SOURCE_KEY = _raw
REGEX = .*
DEST_KEY = _MetaData:Index
FORMAT = debugtest

[choose_log_index]
SOURCE_KEY = _raw
REGEX = .*
DEST_KEY = _MetaData:Index
FORMAT = logtest

NOTE: The names of the indexes and transforms stanzas above can be changed to what suits your needs.
Hope that helps.

Re: i feel dumb, but I see no data when I search in splunk.

gkanapathy — Thu, 09 Feb 2012 23:42:46 GMT

there is a way to show code in an Answers posting. The "code" button in the input editor (looks like "101-010" will make anything you type in literal. There should also be a help link somewhere to the formatting codes so you can type them directly (it is in Markdown), but some fail has removed it.

Re: i feel dumb, but I see no data when I search in splunk.

sgarvin55 — Fri, 10 Feb 2012 00:55:02 GMT

I learn something new everyday!
Thanks gkanapathy, I owe you a beer!