What are base search recommendations?

[glpadilla_sol]

Hello everyone,

I have noticed that some users in our Splunk environment are always using base searches and Post-process searches, because they was told that was a good practice to do that.

But there are some cases I have noticed that the use of the base search is not speeding up the dashboard instead spent more time.

For example there is a dashboard that uses a base search and they use something like this:

Base search

index=example sourcetype=testing | fields *

And then at the subsearch I can see that when Splunk uses that best search is doing something weird adding the | fields *  at search for example:

Example:

index=example sourcetype=testing | fields *
| eval Date=strftime(_time, "%m/%d/%Y")
| dedup s1, s1
| fields * | search something=tosearch
| fields * | eval _time = strptime(Date,"%m/%d/%Y")

When the post-search is

<query>
| eval Date=strftime(_time, "%m/%d/%Y")
| dedup s1, s1
| search something=tosearch
| eval _time = strptime(Date,"%m/%d/%Y")

</query>

So I would like to understand why Splunk does it.

And also I would like to know if there are some scenarios where use the base search is not recommended.

Thanks in advantage.

Best Regards,

[bowesmana]

Base searches are really intended to reuse aggregations rather than just bring a whole load of data into the dashboard like your example here.

Unless you use an aggregation there is a 500,000 event limit I believe and I have found some strange bad performance behaviour when trying to do this.

You should craft base searches carefully, considering how they can be used. Without seeing the other searches, it's hard to say whether your example makes sense as a base search.

As to why you are seeing those extra fields * statements, I can't say.

So, as a recommendation - using base searches just to try to 'cache' events, is bad practice, I've rarely found a good need for that

For example, if you have two searches, one needing count by host and the other needing count by sourcetype, then a sensible base search would be

index=x
| stats count by host sourcetype

and the appropriate post-process searches would be

| stats sum(count) as count by host

AND

| stats sum(count) as count by sourcetype

[gcusello]

Hi @glpadilla_sol,

base search should contain a streming command to reduce the number of results and then use results in panels.

Infact using fields at the end of the base search os a workaround when there isn't any streaming command.

Running a search like the one you shared, you don't have the advantages of a base search because you have all the results and all the fields.

And you don't reduce the CPU occupation becsuse you have a  search with all the results in each panel.

So I use base searches when I can group or filter results.

If your problem are performaces in dashboard running there are other methods to do this (accelerations, Summary Indexes, Data Models, etc...)

Ciao.

Giuseppe