Combine dynamic fields into a multivalued field in a search

[shivanshu1593]

Hi All,

I'm working on a search, where I currently have the following:

..base search..

| table static_name, static_time, static_title, static_owner,  static_id, static_description

Apart from static_title, static_time, static_id, the other fields are dynamic (they change as the search runs with different inputs. static_owner gets replaced by file_name, other fields like fqdn, process, event_hash etc get added to the search)

What I'm trying to achieve is that the static values, which always are present in the search, should be shown as separate fields, while dynamic fields are merged into a new, multivalued field called Combined_Field, irrespective of the new fields showing up. They should be merged into them.

| table static_time, static_owner, static_id, Combined_Field (a multivalued field, comprising the values of all the dynamic fields)

..base search..

| table static_name, static_time, static_title, static_owner,  static_id, static_description

| eval Combined_Field = null()

| foreach * [ eval Combined_Field=if('<<field>>'==static_time OR '<<field>>'==static_owner OR '<<field>>'==static_id, '<<field>>', mvappend('<<field>>', Combined_Field)) ]

| table static_time, static_owner, static_id, Combined_Field

The Combined_Field always remains empty. Could anyone check and let me know as to what am I doing wrong. Or if this can be achieved via a different approach. I've always tried doing the foreach command with case instead of if, no luck.

Thank you in advance,

S

[ITWhisperer]

You need to check is Combined_Field is null before trying to mvappend to it.

| foreach * [ eval Combined_Field=if('<<field>>'==static_time OR '<<field>>'==static_owner OR '<<field>>'==static_id, '<<field>>', if(isnull(Combined_Field),'<<field>>',mvappend('<<field>>', Combined_Field))) ]

[shivanshu1593]

Hello @ITWhisperer ,

Thank you for the suggestion. I tried your suggestion, but the field Combined_Fields is still blank.

I used | eval Combined_Fields = null() before foreach. It didn't help either.

[ITWhisperer]

Try the fields in the other order in the mvappend

| foreach * [ eval Combined_Field=if('<<field>>'==static_time OR '<<field>>'==static_owner OR '<<field>>'==static_id, '<<field>>', if(isnull(Combined_Field),'<<field>>',mvappend(Combined_Field,'<<field>>'))) ]

[shivanshu1593]

It's still blank. Fields didn't get append to the field Combined_Fields. All are showing up as separate fields in the table command and Combined_Fields is empty.

[ITWhisperer]

Did you put the field names in double quotes - also, you should check if <<field>> is Combined_Field so it doesn't get duplicated.

| foreach * [ eval Combined_Field=if('<<field>>'=="static_time" OR '<<field>>'=="static_owner" OR '<<field>>'=="static_id" OR '<<field>>'=="Combined_Field", '<<field>>', if(isnull(Combined_Field),'<<field>>',mvappend(Combined_Field,'<<field>>'))) ]

[shivanshu1593]

I did try running the search with field names under double quotes, it still remained blank. 

Regarding <<field>> not being Combined_Fields, not really sure how to check that. Could you please point me to a direction, using which I can take a look.

[ITWhisperer]

I had included it already - thinking about it, perhaps you need the <<field>> in double quotes too when checking for the names.

| foreach * [ eval Combined_Field=if("<<field>>"=="static_time" OR "<<field>>"=="static_owner" OR "<<field>>"=="static_id" OR "<<field>>"=="Combined_Field", '<<field>>', if(isnull(Combined_Field),'<<field>>',mvappend(Combined_Field,'<<field>>'))) ]

[shivanshu1593]

I tried with "<<field>>"  as well. Still no results. It's a puzzler. Not really sure what's not working here.

[ITWhisperer]

Try with <<FIELD>>

[shivanshu1593]

Tried that too. On passing <<field>> without any quotes around it, the if statement fails.

[ITWhisperer]

I meant use FIELD in caps not lowercase

[shivanshu1593]

My bad. I tried with <<FIELD>> and it has appended all the values, including static_time, static_owner, static_id, which we tried to filter out in the if block. 

[ITWhisperer]

So you need caps and double quotes

| foreach * [ eval Combined_Field=if("<<FIELD>>"=="static_time" OR "<<FIELD>>"=="static_owner" OR "<<FIELD>>"=="static_id" OR "<<FIELD>>"=="Combined_Field", '<<FIELD>>', if(isnull(Combined_Field),'<<FIELD>>',mvappend(Combined_Field,'<<FIELD>>'))) ]

[shivanshu1593]

I've tried to implement that, but its messing up with the fields. There are 17 fields in total in the current iteration, 14 should be in the multivalued field and 3 outside. However, using double quotes and captial fields is reducing the number of fields in the multivalued field down to 8.

[ITWhisperer]

So the process works? Now, it is down to the actual field names. Which field names work, and which ones don't?

[shivanshu1593]

Hello,

None of the fields are working. All of them are present in the multivalued field, Combined_Fields. On covering them and <<FIELD>> in double quotes, they still land up in the multivalued field.

When I put "<<FIELD>>"  == "Combined_Field", then out of 17 fields of this iteration, at least 4 disappear from the multivalued field, leaving only 10 in there instead of 14.