Getting Data In

filter field that starts with certain alphabets

spark2310
Explorer

I have a large query that keeps failing/timing out because search head has no enough ram. I want to run the data in half instead (twice). running one query to filter only half the alphabets of values under FieldX, then run another one with rest of alphabets.

I can do something like FieldX=a* OR FieldX=b* ... but looking for something more practical to query 

0 Karma

thambisetty
SplunkTrust
SplunkTrust

The reason for consuming more memory, your query is collecting all matching events to search head.

why don't you apply transforming commands after your base filters, this will reduce the number of results fetched from indexers to search head.

https://docs.splunk.com/Splexicon:Transformingcommand

————————————
If this helps, give a like below.
0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @spark2310,

your problem isn't so clear for me: how do you can say that a search is slow for limited RAM available, have you any error message?

Are you respecting the minumum reference hardware for Search Head (16 CPUs and 12 GB RAM)? this would be the first question for you for the Splunk Support!

Usually the problem in searches is the availability of CPUs non RAM.

Aniway, coming back to your question: if you have many events, you have some methods to accelerate searches: Data Models, Summary Indexs, etc... that in few word seems to schedule a search that extracts the data for your searches and then you can run the search on the results, so you have a very quick search.

The best approach is to find a way to filter results in the main search, could you share you search that's give errors?

Anyway, you can filter your results using the "search" command with a free text (not so quick) or the "regex" command

| regex field=FieldX "^a|b.*)"

Ciao.

Giuseppe

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @spark2310,

in addition, after the main search (in which you should try to reduce the number of results), you could also reduce the number of extracted fields, taking only the ones you need for you search with the fields command.

Ciao.

Giuseppe

0 Karma

isoutamo
SplunkTrust
SplunkTrust

Hi

how about shortening the search period? That is the most efficient way to speed up the queries. 
r. Ismo

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...