Getting Data In

Automatic lookup on a fieldalias field -- Is it possible?

thisissplunk
Builder

My automatic lookup is not working on fields that were created via FIELDALIAS's.

I have automatic lookups in my "search" app local/props.conf running on things like "src" and "dst" fields. These are global i.e. at the top of props not defined by a sorucetype or anything. Example:

LOOKUP-auto-dst-lookup = subnets Subnet AS dst OUTPUT Description AS dst_description
LOOKUP-auto-dest-lookup = subnets Subnet AS dest OUTPUT Description AS dest_description
LOOKUP-auto-src-lookup = subnets Subnet AS src OUTPUT Description AS src_description

I also want it to work on the "dest" field that you see above, which is the field that most Splunk TAs convert their destination IP field to. Example:

# grep -R "FIELDALIAS-" /opt/splunk/etc/apps
/opt/splunk/etc/apps/Splunk_TA_cool_waf/default/props.conf:FIELDALIAS-alias_for_dst = dst as dest
/opt/splunk/etc/apps/Splunk_TA_cool_av/default/props.conf:FIELDALIAS-alias_for_ComputerName = ComputerName as dest

However, only src and dst are working, not dest. Is there some kind of order of precedence here that I'm missing, or is it impossible for the automatic lookups to work based off of field names created by FIELDALIAS's?

Edit: Seems like there is precedence, and that I should edit System default to achieve what I want. However, system/default/props.conf says to NOT edit that file. So what am I supposed to do then?

1 Solution

thisissplunk
Builder

Local props.conf are loaded after default props.conf. Apps are loaded alphabetically. Splunk's precedence options are slim. The (lame) options are:

  1. Move all of the FIELDALIAS's in the other app's default props.conf to the local props.conf, then move the new LOOKUP in the search app to its default props.conf so that it gets loaded last.
  2. Make a new app that starts with "zzz" or similar and place the LOOKUP in that app's default props.conf. This will ensure that it is loaded last after all of the FIELDALIAS fields are created
  3. There may be another way that I'm not ware of, such as enhancing the data before it even hits Splunk but I cannot do that in my situation.

View solution in original post

thisissplunk
Builder

Local props.conf are loaded after default props.conf. Apps are loaded alphabetically. Splunk's precedence options are slim. The (lame) options are:

  1. Move all of the FIELDALIAS's in the other app's default props.conf to the local props.conf, then move the new LOOKUP in the search app to its default props.conf so that it gets loaded last.
  2. Make a new app that starts with "zzz" or similar and place the LOOKUP in that app's default props.conf. This will ensure that it is loaded last after all of the FIELDALIAS fields are created
  3. There may be another way that I'm not ware of, such as enhancing the data before it even hits Splunk but I cannot do that in my situation.

crsogeti
Engager

Did anyone find a better workaround? I'm having a similar trouble with a calculated field result used in an automatic lookup: it doesn't work.
Since calculated fields are done before lookups this should not be a problem. And yet...
I cannot create apps freely as I am in a very restricted environment. So the solution does not work for me.

0 Karma

thisissplunk
Builder

Yikes, so my only option is to place my lookups in the system default props.conf which explicitly tells me not to because it gets overwritten... Or to create an app that starts with z and add it tonthat new app's default props.conf?

There are too many TA' to modify their settings and place them in local instead.

0 Karma

woodcock
Esteemed Legend

No, this was a suggestion to try, not a definitive statement of how it works. I think that what he is saying is that if App X owns the automatic lookup but App Y owns the field alias, and you are working in App Y, then you need to make sure that you change the permissions of the field alias in App X to Global (AKA All apps) before it will be in scope for the automatic lookup to see it. Do not put anything like this in $SPLUNK_HOME/etc/system/*, not even in local. What you might put in local is a local.meta setting that exports all field aliases to Global:
http://docs.splunk.com/Documentation/Splunk/latest/Admin/Defaultmetaconf

0 Karma

thisissplunk
Builder

They are already global though. The definitions in the TA's default props affect all apps, particularly search, which is where im working and the lookup exists. The dest lookup still doesn't work though.

0 Karma

somesoni2
Revered Legend

For information on the sequence of the search time field operations, you can look at this bookmark-worthy Splunk documentation link:
http://docs.splunk.com/Documentation/Splunk/6.5.0/Knowledge/Searchtimeoperationssequence#Search-time...

Based on above link, the Field aliases are calculated before the automatic lookup field configuration is applied. The issue in my opinion could be due to the field alias being defined in the app which has lower precedence (see this for app directory precedence) because of which automatic lookup of your search app is being applied before your Splunk TA field alias is created. My suggestion would be to try to define the automatic lookup in an app which has lowest precedence (say app name starts with lowercasez) to ensure it's applied last.

thisissplunk
Builder

Yikes, so my only option is to place my lookups in the system default props.conf which explicitly tells me not to because it gets overwritten... Or to create an app that starts with z and add it tonthat new app's default props.conf?

There are too many TA' to modify their settings and place them in local instead.

0 Karma

somesoni2
Revered Legend

Your option (which we're exploring) would only be create an app that starts with z and add it tonthat new app's default props.conf. Updating etc/system/default directory's file is never an option.

0 Karma
Get Updates on the Splunk Community!

New in Observability - Improvements to Custom Metrics SLOs, Log Observer Connect & ...

The latest enhancements to the Splunk observability portfolio deliver improved SLO management accuracy, better ...

Improve Data Pipelines Using Splunk Data Management

  Register Now   This Tech Talk will explore the pipeline management offerings Edge Processor and Ingest ...

3-2-1 Go! How Fast Can You Debug Microservices with Observability Cloud?

Register Join this Tech Talk to learn how unique features like Service Centric Views, Tag Spotlight, and ...