Splunk Search

Why is foreach with wildcards not picking up all fields in my search?

_jgpm_
Communicator

I'm not exactly sure why this isn't working. I couldn't find it in the documentation. I'm on 6.4.3.

basic search | table abc*abc def* |

This creates a table populated by fields that fit the wildcards. This would be an example table.

abc1abc|abc2abc|abc3abc|def1|def2|def3
sample1|sample2|sample3|123 |2345|null

Then I use foreach * [eval <<FIELD>>=1 ]

I'm doing this to just test what foreach is being applied to.

The table turns into:

abc1abc|abc2abc|abc3abc|def1|def2|def3
sample1|   1   |   1   |123 |2345|null

I have no idea why the * is only being applied to some subset of the fields. Is this somehow a carryover from the original table definition?

Thanks!

0 Karma
1 Solution

Flynt
Splunk Employee
Splunk Employee

Odd this seems to work fine for me but then I'm just using junk data.

|stats count|eval  abc1abc="sample1",cabc2abc="sample2", abc3abc="sample3", def1=123, def2=123, def3=""|fields - count| foreach * [eval <<FIELD>>=1 ]

On the offchance it's a naming issue, does adding "s work?

|stats count|eval  abc1abc="sample1",cabc2abc="sample2", abc3abc="sample3", def1=123, def2=123, def3=""|fields - count| foreach * [eval "<<FIELD>>"=1 ]

View solution in original post

Flynt
Splunk Employee
Splunk Employee

Odd this seems to work fine for me but then I'm just using junk data.

|stats count|eval  abc1abc="sample1",cabc2abc="sample2", abc3abc="sample3", def1=123, def2=123, def3=""|fields - count| foreach * [eval <<FIELD>>=1 ]

On the offchance it's a naming issue, does adding "s work?

|stats count|eval  abc1abc="sample1",cabc2abc="sample2", abc3abc="sample3", def1=123, def2=123, def3=""|fields - count| foreach * [eval "<<FIELD>>"=1 ]

somesoni2
Revered Legend

+1 on using double quotes around field name.

0 Karma

_jgpm_
Communicator

I'm not sure if I can replicate it here. My base search has 460K+ events. Suffice to say, I have 6 fields: 5 in the format abc*def and 1 in xyz_*.

This is a near cut & paste index=index1 tag=tag1 | table abc*def xyz_* | foreach * [ eval <>_flag=1 ]

the first field abcFOXdef,abcDOGdef have 8 fields that are full, rest are empty. abcCATdef, abcBIRDdef,abcCROWdef all have thousands of entries. xyz_STAR has 12 events.

Only abcCATdef, abcBIRDdef,abcCROWdef had _flag=1 fields created.

0 Karma

somesoni2
Revered Legend

The foreach will be applied to all the events, for each field, so not sure number of events will make a difference here. If you're not getting a <<FIELD>>_flag field created for a field with null values, then try using a fillnull command before foreach. Something like this

basic search | table abc*abc def* | fillnull value="null" abc*abc def* | foreach * [eval "<<FIELD>>_flag"=1 ]

OR

basic search | table abc*abc def* | fillnull value="null" abc*abc def* | foreach abc*abc def* [eval "<<FIELD>>_flag"=1 ]

_jgpm_
Communicator

fillnull definitely forced the FIELD_flag operation to occur for all fields. I used just 'fillnull value=""' and it worked. However, running fillnull on 460K x 6 fields caused my query to go from 38 secs to 56 secs. #firstworldproblems. I'm just using my laptop though. I will have to clean up some logic, but thank you for solving the issue!

0 Karma
Get Updates on the Splunk Community!

Happy CX Day to our Community Superheroes!

Happy 10th Birthday CX Day!What is CX Day? It’s a global celebration recognizing innovation and success in the ...

Check out This Month’s Brand new Splunk Lantern Articles

Splunk Lantern is a customer success center providing advice from Splunk experts on valuable data insights, ...

Routing Data to Different Splunk Indexes in the OpenTelemetry Collector

This blog post is part of an ongoing series on OpenTelemetry. The OpenTelemetry project is the second largest ...