Splunk Search

Can Splunk share memory data to different queries?

jenniferhao
Explorer

Hello, Splunk experts,

I have a very big raw data, and need to pass the different rules. For example: query1: index=abc, sourcetype=xyz data=raw|rule1,rule2...ruleN and another query2 is ndex=abc, sourcetype=xyz data=raw|ruleN+1,ruleN+2...ruleN+M....
the raw data is same, but rules are different . If I ran this 2 queries, how can I share same raw data in memory and don't need to load 2 times of the big data. Any solution for this?

Thanks.

0 Karma

jenniferhao
Explorer

Is it any way in Splunk to do the for loop nesting. Above requirement likes:

for raw in (raw1, raw2, raw3, raw4):
for rule in (rule1,rule2,rule3,rule4,rule5,rule6):
query(raw,rule)

0 Karma

richgalloway
SplunkTrust
SplunkTrust

You may be taking the wrong approach. What problem are you trying to solve that requires uploading the same block of data twice? Once you load your data into Splunk you can search it as many times as you like.
What do you mean by "rule"? That's not a Splunk term.

---
If this reply helps you, Karma would be appreciated.
0 Karma

jenniferhao
Explorer

Sorry, maybe I didn't explain clearly. I have a huge raw data, and huge rules which is Splunk term. Because the big numbers, I cannot run one query to pass everything due to Splunk limitation. So I split the raw data into some groups, like RAW1, RAW2.... and also split rules into some groups: RULE1, RULE2... Then I made a query like RAW1+RULE1, RAW1+RULE2....(all of them in Splunk term). The one rule example: |eval exception=if(exception="" AND Rep = "1" AND element = "abc" AND comments IN("abc1","abc2") AND band = "xxx" AND value != "-1", "special",exception). every RULE1 have a big numbers of this kind of rule. I just want to know if I can make RAW1 in the memoery, then I can just pass through different RULE groups to avoid load RAW1 many times.

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...