topic Re: Fieldalias Override another Fieldalias in Splunk Search

Fieldalias Override another Fieldalias

shayhibah — Tue, 29 Sep 2020 19:27:04 GMT

Hi,

I have a props file which contains the following:

FIELDALIAS-aob_gen_alias_4 = dst AS dest
FIELDALIAS-aob_gen_alias_5 = destination AS dest

In case I have some logs with both fields (dst, destination), the final result is that the dest field contains the values of destination field ONLY and not dst & destination.

Can someone tell me whats wrong with my conf file?
Thanks

Re: Fieldalias Override another Fieldalias

FrankVl — Wed, 09 May 2018 14:01:42 GMT

What is your desired outcome?

If you are aiming at having dest as a multivalued field in those cases where the event contains dst and destination, you should look at something else than field aliases.

Re: Fieldalias Override another Fieldalias

shayhibah — Wed, 09 May 2018 15:02:04 GMT

I want that dest will contains both dst & destination values

Re: Fieldalias Override another Fieldalias

somesoni2 — Wed, 09 May 2018 16:32:12 GMT

With field aliases, in case of multiple entries with same target/renamed field name, the one that's applied last will take precedence. Since you want the rename to work conditionally (based on availability of field dst and destination, try the calculated field, like this (props.conf on your search head)

EVAL-dest = case(isnotnull(dst) AND isnotnull(destination),dst.",".destination,isnotnull(dst),dst,true(),destination)

Re: Fieldalias Override another Fieldalias

woodcock — Thu, 10 May 2018 05:39:27 GMT

This would be better done as a Calculated field defined like this:

dest = coalesce(destination, dst)

Re: Fieldalias Override another Fieldalias

FrankVl — Thu, 10 May 2018 08:04:30 GMT

Coalesce will only get him the first non-null value. He wants both values in the field in case both exist.

Re: Fieldalias Override another Fieldalias

woodcock — Thu, 10 May 2018 14:36:24 GMT

If you need BOTH, then you need a Calculated field defined like this:

dest = mvappend(destination, dst)

Re: Fieldalias Override another Fieldalias

woodcock — Thu, 10 May 2018 14:36:36 GMT

See new answer.

Re: Fieldalias Override another Fieldalias

lwest_splunk — Thu, 01 Aug 2019 23:04:29 GMT

woodcock is on the right track here. Best method for this from my investigation on several cases as a Splunk TSE yielded the following:

Index time method to accomplish ONE field, renamed as MULTIPLE field names:

EVAL-foo = coalesce (field1,field2,etc...) thus creating indexed data that combines multiple disparate sourcetypes into one common fieldname as indexed.

Further info:

If you have defined a field alias, Splunk will only view the alias name as the data that is being turned into that field name, so for example:

FIELDALIAS- = AS foo

FIELDALIAS- = AS bar

FIELDALIAS- = foo AS

Using this example from above, and with the condition that there is a secondary sourcetype that already ingests a field with the name of 'foo" the following condition will occur -

The data that comes in natively as 'foo' will not return under these conditions, as Auto-KV extracts the native field of foo [no longer indexing the data as 'foo'], and is also parsing the new field alias for foo, however, since != foo this causes a break. The break is that the native 'foo' data will then be overwritten as a null value because it will not include data, thus the field 'foo' will be treated as an empty field with no data. Along with this, multiple iterations of the FIELDALIAS within different app contexts can negatively impact one another and cause "missing data" based on order of operations, and which field ends up overwriting the others, though this behavior may be dependent upon app permissions/sharing.

Prior to 7.2.0, to tie multiple disparate sourcetypes to a single field name, you would have to ingest them all with different names and then perform an EVAL(coalesce) function at search time to make field1,field2, ... all come out as a singular field name similar to the following:

| eval foo=coalesce(field1,field2,field*...)

POST 7.2.0 the ability to inject an EVAL statement at indexing time was introduced, allowing for props.conf entries to create the following:

EVAL-foo = coalesce (field1,field2,...) thus creating indexed data that combines multiple disparate sourcetypes into one common fieldname as indexed.

For expected field alias behavior, the following applies:

FIELDALIAS- = AS foo

FIELDALIAS- = AS bar

The above will work as expected.

For examples of usage that will NOT work, the following applies:

FIELDALIAS- = AS foo

FIELDALIAS- = AS bar

FIELDALIAS- = foo AS

Multiple iterations of

App-Context1 props.conf FIELDALIAS- = AS foo

App-Context2 props.conf FIELDALIAS- = AS foo

App-Context3 props.conf FIELDALIAS- = AS foo

Above examples at index time will fail, as an expected behavior.

Re: Fieldalias Override another Fieldalias

woodcock — Thu, 01 Aug 2019 23:47:06 GMT

I pretty much agree with what you said except that EVAL- is the original search-time field extraction that does not create an indexed field. I think that you meant INGEST_EVAL- for index-time extractions created indexed fields. Also your example seems to imply that transitive field aliases work but they DO NOT. If you do alias foo to bar and you have a foo, you will get a bar but if you further alias bar to bat, you will NOT get a bat.