Splunk Search Language - Is there another command for Looping besides Map?

[potnuru]

Is there any command in Splunk for Looping other than Map command ?

Requirement is described as below:

I can't provide the data here but I can explain the scenario with an example.

Example:

Let's say we have Data Base Manager tool which is managing all the DB Connections/Sessions. Users should log on to DB Manger tool first using their personal account and then initiate a connection to Database using the Database account.

Each DB Manager Session Initiation log contains the following parameters:

1. User Account Name, 2. DB Account Name 3. DB Name 4. Session Initiation Time

Each DB Login event contain the following parameters:

1. DB Account name 2. DB Name 3. Session Start time

Hence, each login event on DB will have a corresponding session Initiation log on DB Manager tool. Let's assume the maximum Time difference between these two logs is 30 mins, it means for each DB Login Event we should have a log on DB Manager in the last 30 minutes.

Now the requirement is to create a Report for all the DB Login Events and add the parameter "User Account Name" from the DB Manger tool session initiation log corresponding to it.

[ITWhisperer]

The devil is in the detail!

Do you need to keep anything from Q1? If not, can you use the results from Q1 to filter Q2?

<search Q2> [<search Q1> | fields <common fields to filter with>]
...

If you do, then you can use stats to gather events from Q1 and Q2 assuming they have common field values

<search Q1> OR <search Q2>
| stats values(*) as * by <common fields>

[potnuru]

Thanks for your answer; but the problem here is that we don't know the earliest and latest time intervals for Q2; they should come from Q1 results/output.

Lets say Q1 is having 10 events in the output, each event will have the earliest and latest time parameters; We have to pass those parameters to Q2 and run it only in that time interval. In this way, we have to run Q2 for 10 times (as Q1 is having 10 events in the output).

[PickleRick]

Either use map command, which you mentioned earlier, or use a subsearch to generate set of possible timeranges. You can use multiple timeranges in a single simple search (as far as I remember you can't do that with tstats)

[potnuru]

Hi @PickleRick Thank you for your response.

Yes, I am able to get the desired results with Map command but the search performance is bad.

Can you elaborate more on the Sub Search approach please? Or give me an example.

[PickleRick]

You can use subsearch which returns set of (latest, earliest) pairs which will get rendered as set of conditions in form of ((latest=something earliest=something) OR (latest=something earliest=something) OR ...).

But it will most probably _not_ solve your performance pains.

If you have a search built like

<base search A>
| map search="<search B>"

If either search A or search B on its own is slow, changing from map to subsearch won't help much. Either build your searches better (can't tell you how since you don't share your searches) or just accept the fact that they're slow. Splunk still has to read the data from the buckets and process it and the only thing that can be slowing you additionaly in case of map command is possibly the limit on concurrent searches but still changing that to subsearch won't help much especially if those time ranges fall within different buckets.

[gcusello]

Hi @potnuru,

I'm not sure that's possible to have the result you like because SPL isn't a programming structured language, probably you have to structure your search with a different approach:

take the full time period and categorize events in a different way.

It's difficoult to describe this theory without having data for check!

Ciao.

Giuseppe

[potnuru]

Hello @gcusello 

Thank you for your response.

Currently I am able achive it by using MAP command in Splunk, but the performance is very very bad. I am looking for other approaches for solving this problem/logic 

I can't provide the data here but I can explain with an example about the scenario.

Example for the Scenario:

Let's say we have Data Base Manager tool which is managing all the DB Connections/Sessions. Users should log on to DB Manger first using their personal account and then initiate a connection to DB using the DB account.

Each DB Manager Session Initiation log contains the following parameters:

1. User Account Name, 2. DB Account Name 3. DB Name 4. Session Initiation Time

Each DB Login event contain the following parameters:

1. DB Account name 2. DB Name 3. Session Start time

In this process, each login event on DB will have a corresponding session Initiation log on DB Manager tool. Let's assume the max. Time difference between these two logs is 10 mins.. means for each DB Login Event we should have a log on DB Manager in last 10 minutes.

Now we have to create a Report for all the DB Login Events and add the parameter "User Account Name" on DB Manger corresponding to it.

[PickleRick]

There are some dirty tricks that can be used to do "looping" but more often than not the will to "loop" is just a non-splunkish way of thinking. The problem typically can be solved using other techniques.

[ITWhisperer]

What is the question?

[gcusello]

Hi @potnuru,

what's your question?

Ciao.

Giuseppe

[potnuru]

Hi @gcusello 

Please find my question below:

We have two different Search Queries, lets say Q1 and Q2; For each Event or Result in Q1, we have to run Q2 by passing the parameters from Q1 (for example: earliest and latest time, Username etc..). and add some parameters from the Q2 result (for example: Supervisor Id for the Username) to each event in Q1.

[gcusello]

HBi @potnuru,

you have to put both the searches terms in the main search correlating them using the OR operator.

then you have to find a common field to use as correlation key, if the field names are different, you can use "| eval key=coalesce(key1,key2)") to have the same field name.

Then you can use this key (eventually more than one) in a stats command to correlate the two data sets.

Then in the stats command you can put all the fields you need from one or the other search.

Then you can put some condition to validate data: e.g. if you want data present in both the searches find something different (e.g. index value) and in the stats command you can count the number of indexes.

Please share your searches, indicating the correlation fields.

Ciao.

Giuseppe