Search Json Field Using Dynamic Variable

[joemarty82]

Hello,

I have json data and I am trying to search a specific field using a dynamic variable. I can properly search if I have an exact static field but not dynamic field. As an example, the below works:

source="main.py"| spath "cve.CVE_data_meta.ID" | search "cve.CVE_data_meta.ID"="CVE-2018-XXXX" | table cve.description.description_data{}.value

However, I am trying to feed a dynamic variable (test) from a different search to extract the correct value such as the below (shortened to make it easier to read):

source="main.py"| eval test="CVE-2018-XXXX" | spath "cve.CVE_data_meta.ID" | search "cve.CVE_data_meta.ID"=test | table cve.description.description_data{}.value

In the above case as an example I just hard coded the test variable but that value will come from a different search. Anyhow, the above does not work. I tried many variations and nothing really seems to work. I think the problem is due to the fact splunk thinks I am working with multiple value data and I cannot properly search off that. Anyhow, I think there has to be an easy solution that I cannot seem to get on my own. Hopefully someone can push me to the finish line with this.

Thanks,

Marty

[bowesmana]

@joemarty82 

The 'search' command right hand side is a string. If you want to compare against the value of another field, use the 'where' command, where the right hand side is an eval statement, so, you could do

source="main.py"
| eval test="CVE-2018-XXXX" 
| spath "cve.CVE_data_meta.ID" 
| where match('cve.CVE_data_meta.ID', test)
| table cve.description.description_data{}.value

Of you could do

| where 'cve.CVE_data_meta.ID'=test

Note that in eval statements, fields with non basic characters in them need to be surrounded by single quotes.

Also, when putting this logic into a dashboard and allowing user defined variables entered into a dashboard form and available as tokens, you would then do something like

| where match('cve.CVE_data_meta.ID', "$token_name$")

Hope this helps

[joemarty82]

Hey bowesmana,

Thanks for the advise. I verified your idea works for your example. However, I think I did not explain my problem 100% clearly.

I am actually doing 2 separate searches. The first search is where I dynamically obtain my CVE ID. Then with that value I am putting that into my test variable and am performing another search so my current full search is actually more like this:

| inputlookup ********* | eval test2 = cve
| append 
[ search source="main.py"
| eval test=test2    (THIS IS MY PROBLEM I think, I AM NOT ABLE TO SET test2 to test)
| spath "cve.CVE_data_meta.ID" 
| where match('cve.CVE_data_meta.ID', test)
| rename cve.description.description_data{}.value as description]
| table description

Doing some reading online I have a feeling my order of the searches might not work. So with this the search is taking a very long time and I am not getting any results. Apologies on not being clear the first time.

Thanks again for your time,

Marty

[bowesmana]

You are right in that your ordering is wrong. Subsearches are performed before the outser search, so you cannot pass variables into the subsearch.

You need to turn your search around, so that you perform the lookup in the subsearch and use that as a filter to the outer search. The subsearch can be part of the outer query like this example, where as you can see the inner search will set the ID to 123 and return it to the outer search where it becomes part of the outer search command.

| makeresults 
| eval x=split("{\"cve\":{\"CVE_data_meta\":{\"ID\":123,\"description\":\"test123\"}}};{\"cve\":{\"CVE_data_meta\":{\"ID\":234,\"description\":\"test234\"}}}",";")
| mvexpand x
| rename x as _raw
| spath "cve.CVE_data_meta.ID" output=ID
| search [
  | makeresults
  | eval ID=123
]
| spath
| table cve.CVE_data_meta.description

[joemarty82]

Thanks bowesmana,

I got my example to work FINALLY based on your input and query structure. It might be worth mentioning in my case I had to change your format a bit. I did not use (makeresults) as if I did it would not allow my top search to grab the index properly. I also had to use the (return) key word to return my sub search argument.

Other than that I would not have got the job done without your help. Thanks for the guidance as I was starting to become obsessive with this problem as I tried many many variations.

Thanks again,

Marty

[bowesmana]

Well done @joemarty82 

You will often find solutions posted here using the | makeresults method. That is simply a way to create a dummy data set which the author can then use to demonstrate the solution. I should have mentioned that in my post.