Splunk Search

Create and De-reference New Field Names

tom_porter
Explorer

I have Linux audit records that have a field called type and fields with the naming convention lower(type).field.  I want to be able to combine type, as a prefix, and a set of suffixes to create new field names that exist in the data.  For example, I have a type called FILE_WATCH and fields called file_watch.exe, file_watch.egid, file_watch.comm, etc.

I want to develop a dashboard table by type and suffix in Splunk to show if a particular field exists for a type.  So going back to my example using type=FILE_WATCH, how can I create a new field name along these lines?

base = lower(type)
exe={base}.".exe"  # does not work, but you get the idea.

with exe now equal to the field name, I want to be able to de-reference the new field name to see if it exists.

Labels (2)
0 Karma

richgalloway
SplunkTrust
SplunkTrust

Curly braces on the LHS of an eval says to use the contents of the field as the new field name.

See if this example query helps explain

| makeresults 
| eval base="file_watch"
| eval {base}= base.".exe"
| eval {base}exe=base.".exe"

 

---
If this reply helps you, Karma would be appreciated.

yuanliu
SplunkTrust
SplunkTrust

I stand corrected.  Thanks, @richgalloway !

Now, @tom_porter will need to explain what "does not work" means.  To be clear, this phrase should be forbidden in a forum like this as it conveys very little information in the best scenarios.

  • Illustrate/mock your data (anonymize as needed),
  • show code you tried,
  • show actual results (anonymize as needed),
  • illustrate/mock desired results,
  • explain the logic connecting data and desired results if it is not painfully obvious.

Also explain the difference between actual results and desired results if it is not painfully obvious.

0 Karma

yuanliu
SplunkTrust
SplunkTrust

I'm not sure if I get the idea.  Do you mean to use this?

base = lower(type)
exe=base.".exe" 

What is the intention of the curly brackets? (No, {base} is not a valid expression.)

0 Karma

tom_porter
Explorer

I was trying something along the lines of dynamic field creation.  At issue is that we have multiple dot notation field names with different prefixes, but a common suffix.  (e.g.: file_watch.sgid and execve.sgid).    There are about 40 prefixes and 50 or more suffixes.  Not all prefixes have all suffixes.  What I wanted to do was to create a dashboard that would show the prefixes as rows, and the suffixes as columns, with x marking cells with non-null values for prefix.suffix based on a search over the last 24 hours.

Tags (1)
0 Karma

dtburrows3
Communicator

Not sure if I am interpreting your question correctly but I gave it a shot.

So given that the are many different fieldnames with dot notation.

dtburrows3_0-1702425925462.png


You are trying to get a final table of something like this?

dtburrows3_1-1702425976363.png


I was able to achieve this by utilizing a foreach loop

| makeresults
    | eval
        "tmp.exe"="value1"
    | append
        [
            | makeresults
                | eval
                    "noop.spl"="value2"
            ]
    | append
        [
            | makeresults
                | eval
                    "tmp.spl"="value3"
            ]
    | append
        [
            | makeresults
                | eval
                    "foo.exe"="value4"
            ]
    | append
        [
            | makeresults
                | eval
                    "tmp.tgz"="value5"
            ]
    | append
        [
            | makeresults
                | eval
                    "foo.tgz"="value6",
                    "tmp.exe"="value7"
            ]
    
    ``` Gather unique fieldnames as values of a new field ```
    | foreach *.*
        [
            | eval
                existing_fieldname=if(
                    isnotnull('<<FIELD>>'),
                        mvappend(
                            'existing_fieldname',
                            "<<FIELD>>"
                            ),
                        'existing_fieldname'
                    )
                    
            ]
    ``` Parse out prefix and suffix of the new field ```
    | eval
        prefix=case(
            isnull(existing_fieldname), null(),
            mvcount(existing_fieldname)==1, mvindex(split(existing_fieldname, "."), 0),
            mvcount(existing_fieldname)>1, mvmap(existing_fieldname, mvindex(split(existing_fieldname, "."), 0))
            ),
        suffix=case(
            isnull(existing_fieldname), null(),
            mvcount(existing_fieldname)==1, mvindex(split(existing_fieldname, "."), 1),
            mvcount(existing_fieldname)>1, mvmap(existing_fieldname, mvindex(split(existing_fieldname, "."), 1))
            )
    ``` Use chart function to display unique combos of prefix/suffix from inherited fieldnames ```
    | chart limit=50
        count as count
            over prefix
            by suffix
    ``` Replace numbers in the table with "X" to signify that the prefix/suffix combo was found in the data ```
    | foreach *
        [
            | eval
                <<FIELD>>=if(
                    NOT "<<FIELD>>"=="prefix",
                        if(
                            '<<FIELD>>'>0,
                                "X",
                                null()
                            ),
                        '<<FIELD>>'
                    )
            ]

tscroggins
Motivator

Jumping in on an aging topic, but you may be able to simplify the SPL, albeit with an unknown impact to performance. (Always test!)

| makeresults format=json data="[{\"foo\": {\"field1\": \"value1\", \"field2\": \"value2\"}}, {\"bar\": {\"field1\": \"value3\", \"field2\": \"value4\"}}, {\"baz\": {\"field2\": \"value5\", \"field3\": \"value6\"}}]"
| spath
``` end test data ```
| table *.*
| transpose
| rex field=column "(?<prefix>[^.]+)\\.(?<suffix>.+)"
| foreach row* [ eval value=coalesce('<<FIELD>>', value) ]
| xyseries prefix suffix value

tom_porter
Explorer

Yes!!!  That second table.   Thank you....will try out your solution later this week.  Much appreciation to you all for chiming in on this.

0 Karma
Get Updates on the Splunk Community!

Splunk Observability Cloud | Unified Identity - Now Available for Existing Splunk ...

Raise your hand if you’ve already forgotten your username or password when logging into an account. (We can’t ...

Index This | How many sides does a circle have?

February 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

Registration for Splunk University is Now Open!

Are you ready for an adventure in learning?   Brace yourselves because Splunk University is back, and it's ...