topic Re: Pass String Field from Outer Search into Inner Map Search in Splunk Search

Pass String Field from Outer Search into Inner Map Search

ErikaE — Thu, 31 Dec 2015 16:07:36 GMT

My search looks like this:

index=index_name source="Source A.csv" | eval Start2=strptime(Start, "%m/%d/%Y%H:%M") | eval End2=strptime(End, "%m/%d/%Y%H:%M") | map maxsearches=99999 search="search index=index_name earliest=$Start2$ latest=$End2$ source=\"Source B.csv\" | eval Problem2=\""$Problem$\"" | stats values($Problem2$) as Problem3, avg(Data) as Average, min(Data) as Min, max(Data) as Max, stdev(Data) as Stdev" |  table Average Min Max Stdev Problem3

Problem is a field in Source A of the form XX003 or X2999, a letter or two, then three or four numbers. I am using the Start and End fields from Source A to look in Source B's Data field and calculate stats for each Problem in Source A. I can't seem to get the Problem to pass through the map search. Help!

I have tried:

eval Problem=$Problem$ (like some other examples) 
eval Problem="$Problem$ 
eval Problem=\"$Problem\"

and the example in the code above. I have a nearly identical search with a numerical field i.e. OtherProblem=2.9 that works great.

Help!

Re: Pass String Field from Outer Search into Inner Map Search

jkat54 — Thu, 31 Dec 2015 16:26:50 GMT

If you just run this search does it produce a field called Problem?

  index=index_name source="Source A.csv" | eval Start2=strptime(Start, "%m/%d/%Y%H:%M") | eval End2=strptime(End, "%m/%d/%Y%H:%M") | table Problem

If not, then that's your ahem.. problem. Otherwise try this:
(use single quotes around your search, and simplify the search as much as possible):

index=index_name source="Source A.csv" 
 | eval Start2=strptime(Start, "%m/%d/%Y%H:%M") 
 | eval End2=strptime(End, "%m/%d/%Y%H:%M")
 | table Start2 End2 Problem
 | map maxsearches=99999 search='
  search index=index_name earliest=$Start2$ latest=$End2$ source="Source B.csv" | 
  | stats values($Problem$) as Problem, avg(Data) as Average, min(Data) as Min, max(Data) as Max, stdev(Data) as Stdev
 '
 | table Average Min Max Stdev Problem

There's no need to use eval in the map search to bring the variable in.

Re: Pass String Field from Outer Search into Inner Map Search

ErikaE — Mon, 04 Jan 2016 14:58:52 GMT

The outer search does produce a field called Problem. Sorry, I should have mentioned that I verified that already.

Single quotes causes the map search to fail/produce no result. The error given is: Unable to run query ''search'.

Re: Pass String Field from Outer Search into Inner Map Search

jplumsdaine22 — Mon, 04 Jan 2016 16:02:28 GMT

Map might not be the best command for this situation. I think you can run this in a single search by using streamstats and a pipelined search command.

index=index_name AND (source="Source A.csv" OR source="Source B.csv") 
| eval Start=strptime(Start, "%m/%d/%Y%H:%M") 
| eval End=strptime(End, "%m/%d/%Y%H:%M") 
| streamstats min(Start) as Start max(End) as End 
| search source="Source B.csv" AND (_time<End OR _time>Start) 
| stats avg(Data) as Average min(Data) as Min, max(Data) as Max, stdev(Data) as Stdev by Problem

Also I've changed your stats command to use Problem as a By clause, I think it will be more meaningful, but it might not be. Feel free to change it back to a values function. Also have a look at this splunk answer: https://answers.splunk.com/answers/129424/how-to-compare-fields-over-multiple-sourcetypes-without-join-append-or-use-of-subsearches.html

See http://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Streamstats
See http://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Stats

Re: Pass String Field from Outer Search into Inner Map Search

ErikaE — Tue, 05 Jan 2016 16:08:50 GMT

Your search returns no results, because Problem is not a field in source B. The 5th line of the search tosses out the Problem field when you search within source B. The sixth line returns results when you modify it to:

 | stats avg(Data) as Average min(Data) as Min, max(Data) as Max, stdev(Data) as Stdev by Start

The only field in common between the two sources is time. In the example you linked, there is a common field, user, which does not exist in my data set.

Re: Pass String Field from Outer Search into Inner Map Search

jplumsdaine22 — Wed, 06 Jan 2016 09:30:33 GMT

Can you possibly post a couple of example lines from each source? It would be useful to know what fields are available.

What you're probably going to need is a transaction command: http://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Transaction

Re: Pass String Field from Outer Search into Inner Map Search

jkat54 — Wed, 06 Jan 2016 12:46:40 GMT

Can you try converting it back to the format you had with many double quotes, but not using eval?