topic Re: Splunk cannot find files in Knowledge Management

Splunk cannot find files

fjp2485 — Mon, 29 Apr 2019 16:39:35 GMT

Hi Splunk,

We use Splunk Enterprise 7.2.3.

In our environment there are 49 XML files in subfolders which have to be found by splunk. (With file explorer the files are visible.)Name of 45 files look like this:
GDSNightlyStagingInstallation_*.xml and 4 of them look like this: GDSNightlyStagingSIT_*.xml. (With date and time info in place of '*'.) Each file's document element node has name: BundleResult and has an attribute named NetDurationMins.

We use this search string:
source="\GDS\*NightlyStaging" | search (BundleResult AND NetDurationMins="*") | dedup LogFile

This search string can find only the 45 GDSNightlyStagingInstallation_*.xml files but the remaining 4 GDSNightlyStagingSIT_*.xml files are not retrieved. Could you explain where is the error, please?

Thanks in advance,
Sandor Michaletzky

Re: Splunk cannot find files

richgalloway — Mon, 29 Apr 2019 16:54:10 GMT

What is the monitor statement (from inputs.conf) for the files?
If you run this query: | tstats count where index=* source=GDSNightlyStagingSIT_* do the four source files appear in the results?

Re: Splunk cannot find files

YoungN — Mon, 29 Apr 2019 17:19:00 GMT

I would also check permissions on those 4, make sure they didn't carry permissions from another device.

Re: Splunk cannot find files

fjp2485 — Tue, 30 Apr 2019 07:14:59 GMT

I tried to run the tstats, this was its result:

Reduced buckets were found in index='sambaresultindex'. Tstats searches are not supported on reduced buckets. Search results will be incorrect.

And of course result count was 0.

But the same was the result with GDSNightlyStagingInstallation*.

Re: Splunk cannot find files

fjp2485 — Tue, 30 Apr 2019 07:26:03 GMT

The two kinds of files (i.e. NightyStagingInstallation* and NightlyStagingSIT*) are produced by the same program in the same run of that. In addition the program does not give any special or different permissions to those files.

Re: Splunk cannot find files

fjp2485 — Tue, 30 Apr 2019 08:36:33 GMT

Hello,
I've found an index which works for our query and I've rerun the searches.

This search:
| tstats count where index=newsambalogindex source=\\bafs1\Builds\DashboardData\GDS\\GDSNightlyStaging
resulted 372983 events (so much because there was no filtering for content).
This search:
| tstats count where index=newsambalogindex source=\\bafs1\Builds\DashboardData\GDS\\GDSNightlyStagingInstallation
resulted the same number of events: 372983.
This search:
| tstats count where index=newsambalogindex source=\\bafs1\Builds\DashboardData\GDS\\GDSNightlyStagingSIT
resulted 0.

Please tell me if I should run other tests in order to continue investigation.
Regards,
fjp2485

Re: Splunk cannot find files

richgalloway — Tue, 30 Apr 2019 13:57:33 GMT

Try this query, instead. | metadata type=sources index=sambaresultindex | search source=GDSNightlyStagingSIT_*.

Re: Splunk cannot find files

fjp2485 — Tue, 30 Apr 2019 14:57:21 GMT

Hi nyoung,
Many thanks for dealing with my problem so much.

I ran the search string you've written:
" | metadata type=sources index=sambaresultindex | search source=*GDSNightlyStagingSIT_* "
but it didn't retrieve anything.

Then I removed the SIT_ suffix to check if installation results files are retrieved. They were also not retrieved.

Then I changed index to newsambalogindex and tried again with this search string (to check if installation log files are retrieved):
" | metadata type=sources index=newsambalogindex | search source=*GDSNightlyStaging* "
This search retrieved all installation log files but the test result files didn't appear again.

Then I tried to retrieve only the test result files:
" | metadata type=sources index=newsambalogindex | search source=*GDSNightlyStagingSIT_* "
and this also didn't retrieve anything - of course.

What do I do wrong?