Knowledge Management

Splunk cannot find files

fjp2485
Engager

Hi Splunk,

We use Splunk Enterprise 7.2.3.

In our environment there are 49 XML files in subfolders which have to be found by splunk. (With file explorer the files are visible.)Name of 45 files look like this:
GDSNightlyStagingInstallation_*.xml and 4 of them look like this: GDSNightlyStagingSIT_*.xml. (With date and time info in place of '*'.) Each file's document element node has name: BundleResult and has an attribute named NetDurationMins.

We use this search string:
source="\GDS\*NightlyStaging" | search (BundleResult AND NetDurationMins="*") | dedup LogFile

This search string can find only the 45 GDSNightlyStagingInstallation_*.xml files but the remaining 4 GDSNightlyStagingSIT_*.xml files are not retrieved. Could you explain where is the error, please?

Thanks in advance,
Sandor Michaletzky

0 Karma

fjp2485
Engager

Hi nyoung,
Many thanks for dealing with my problem so much.

I ran the search string you've written:
" | metadata type=sources index=sambaresultindex | search source=*GDSNightlyStagingSIT_* "
but it didn't retrieve anything.

Then I removed the SIT_ suffix to check if installation results files are retrieved. They were also not retrieved.

Then I changed index to newsambalogindex and tried again with this search string (to check if installation log files are retrieved):
" | metadata type=sources index=newsambalogindex | search source=*GDSNightlyStaging* "
This search retrieved all installation log files but the test result files didn't appear again.

Then I tried to retrieve only the test result files:
" | metadata type=sources index=newsambalogindex | search source=*GDSNightlyStagingSIT_* "
and this also didn't retrieve anything - of course.

What do I do wrong?

0 Karma

fjp2485
Engager

Hello,
I've found an index which works for our query and I've rerun the searches.

  • This search:
    | tstats count where index=newsambalogindex source=\\bafs1\Builds\DashboardData\GDS\\GDSNightlyStaging
    resulted 372983 events (so much because there was no filtering for content).

  • This search:
    | tstats count where index=newsambalogindex source=\\bafs1\Builds\DashboardData\GDS\\GDSNightlyStagingInstallation
    resulted the same number of events: 372983.

  • This search:
    | tstats count where index=newsambalogindex source=\\bafs1\Builds\DashboardData\GDS\\GDSNightlyStagingSIT
    resulted 0.

Please tell me if I should run other tests in order to continue investigation.
Regards,
fjp2485

0 Karma

fjp2485
Engager

The two kinds of files (i.e. NightyStagingInstallation* and NightlyStagingSIT*) are produced by the same program in the same run of that. In addition the program does not give any special or different permissions to those files.

0 Karma

nyoung_splunk
Splunk Employee
Splunk Employee

I would also check permissions on those 4, make sure they didn't carry permissions from another device.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

What is the monitor statement (from inputs.conf) for the files?
If you run this query: | tstats count where index=* source=GDSNightlyStagingSIT_* do the four source files appear in the results?

---
If this reply helps you, Karma would be appreciated.
0 Karma

fjp2485
Engager

I tried to run the tstats, this was its result:

Reduced buckets were found in index='sambaresultindex'. Tstats searches are not supported on reduced buckets. Search results will be incorrect.

And of course result count was 0.

But the same was the result with GDSNightlyStagingInstallation*.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Try this query, instead. | metadata type=sources index=sambaresultindex | search source=GDSNightlyStagingSIT_*.

---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

Splunk Training for All: Meet Aspiring Cybersecurity Analyst, Marc Alicea

Splunk Education believes in the value of training and certification in today’s rapidly-changing data-driven ...

The Splunk Success Framework: Your Guide to Successful Splunk Implementations

Splunk Lantern is a customer success center that provides advice from Splunk experts on valuable data ...

Investigate Security and Threat Detection with VirusTotal and Splunk Integration

As security threats and their complexities surge, security analysts deal with increased challenges and ...