Splunk Search

How to search multiple sources within my search?

andreacorrie
Explorer

How do I search multiple source files within my search? I want to do something like:

source="/foo/bar/2016/09/{08,15}/*.avro"

or

source="/foo/bar/2016/09/[08-30]/*.avro"

but neither syntax returns results.

Tags (3)
0 Karma

somesoni2
SplunkTrust
SplunkTrust

This should work

index=foo sourcetype=bar [| gentimes start=-1 | eval t=mvrange(8,30+1) | table t | mvexpand t | eval source="/foo/bar/2016/09/".substr("0".t,-2) | table source] | .... rest of the search

The subsearch is generating a series of sources from "/foo/bar/2016/09/08" to "/foo/bar/2016/09/30" in a giant OR statement ( source="/foo/bar/2016/09/08" OR source="/foo/bar/2016/09/09"...OR "/foo/bar/2016/09/30")

0 Karma

sundareshr
Legend

Try this

base search | regex source="\/\d{2}\/(0[8-9]|1[0-5])\/"
0 Karma

pasokkum
Path Finder

try this..

alt text

0 Karma

andreacorrie
Explorer

This will work but again, it doesn't scale for more than a few days. I'm looking for a solution that I can search 30 days, for example.

0 Karma

kschon_splunk
Splunk Employee
Splunk Employee

In the example you are using, I would suggest extracting the _time variable from your path, and then restricting your query by time (e.g. using the graphical time range picker). There is more information here:
https://docs.splunk.com/Documentation/Hunk/6.4.3/Hunk/Virtualindexconfigurationvariables

In particular, look at these properties:

vix.input.[N].et.regex
vix.input.[N].et.format
vix.input.[N].lt.regex
vix.input.[N].lt.format

If you want to query on something other than time, you can extract additional variables from the path as well, using the property "vix.input.1.path". You can see some examples here:
https://docs.splunk.com/Documentation/Hunk/6.4.3/Hunk/Setupavirtualindex

That should simplify your queries somewhat, since you won't have to simultaneously specify the values you're looking for, and where to find them in the path.

skoelpin
SplunkTrust
SplunkTrust

You could do source=* or you could do source="/foo/bar/2016/09/{08,09}/*.avro" OR source="/foo/bar/2016/09/[08-09]/*.avro"

0 Karma

andreacorrie
Explorer

In my example, I want to search the 8th or the 9th. I'm using syntax you would use in Apache Pig to specify multiple days. Yes, I could use source = source="/foo/bar/2016/09/08/*.avro OR source="/foo/bar/2016/09/09/*.avro but this doesn't scale if you want to search more than a couple days.

0 Karma

skoelpin
SplunkTrust
SplunkTrust

Ahh I see.. You want to have the ability to scale your searches relative to the date which is a dynamic value. Are the days in your source offset from the current day or do they match up to the current date?

I'm assuming with this, you want to search the data from that source that is from Sept 8 and Spet 9th?

source="/foo/bar/2016/09/{08,09}/*.avro"

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...