Archive

Fieldalias Override another Fieldalias

shayhibah
Path Finder

Hi,

I have a props file which contains the following:

FIELDALIAS-aob_gen_alias_4 = dst AS dest
FIELDALIAS-aob_gen_alias_5 = destination AS dest

In case I have some logs with both fields (dst, destination), the final result is that the dest field contains the values of destination field ONLY and not dst & destination.

Can someone tell me whats wrong with my conf file?
Thanks

Tags (1)
0 Karma

lwest_splunk
Splunk Employee
Splunk Employee

woodcock is on the right track here. Best method for this from my investigation on several cases as a Splunk TSE yielded the following:

Index time method to accomplish ONE field, renamed as MULTIPLE field names:

EVAL-foo = coalesce (field1,field2,etc...) thus creating indexed data that combines multiple disparate sourcetypes into one common fieldname as indexed.

Further info:

If you have defined a field alias, Splunk will only view the alias name as the data that is being turned into that field name, so for example:

FIELDALIAS- = AS foo

FIELDALIAS- = AS bar

FIELDALIAS- = foo AS

Using this example from above, and with the condition that there is a secondary sourcetype that already ingests a field with the name of 'foo" the following condition will occur -

The data that comes in natively as 'foo' will not return under these conditions, as Auto-KV extracts the native field of foo [no longer indexing the data as 'foo'], and is also parsing the new field alias for foo, however, since != foo this causes a break. The break is that the native 'foo' data will then be overwritten as a null value because it will not include data, thus the field 'foo' will be treated as an empty field with no data. Along with this, multiple iterations of the FIELDALIAS within different app contexts can negatively impact one another and cause "missing data" based on order of operations, and which field ends up overwriting the others, though this behavior may be dependent upon app permissions/sharing.

Prior to 7.2.0, to tie multiple disparate sourcetypes to a single field name, you would have to ingest them all with different names and then perform an EVAL(coalesce) function at search time to make field1,field2, ... all come out as a singular field name similar to the following:

| eval foo=coalesce(field1,field2,field*...)

POST 7.2.0 the ability to inject an EVAL statement at indexing time was introduced, allowing for props.conf entries to create the following:

EVAL-foo = coalesce (field1,field2,...) thus creating indexed data that combines multiple disparate sourcetypes into one common fieldname as indexed.

For expected field alias behavior, the following applies:


FIELDALIAS- = AS foo

FIELDALIAS- = AS bar

The above will work as expected.

For examples of usage that will NOT work, the following applies:


FIELDALIAS- = AS foo

FIELDALIAS- = AS foo

OR

FIELDALIAS- = AS foo

FIELDALIAS- = AS bar

FIELDALIAS- = foo AS

OR

Multiple iterations of

App-Context1 props.conf FIELDALIAS- = AS foo

App-Context2 props.conf FIELDALIAS- = AS foo

App-Context3 props.conf FIELDALIAS- = AS foo

Above examples at index time will fail, as an expected behavior.

woodcock
Esteemed Legend

I pretty much agree with what you said except that EVAL- is the original search-time field extraction that does not create an indexed field. I think that you meant INGEST_EVAL- for index-time extractions created indexed fields. Also your example seems to imply that transitive field aliases work but they DO NOT. If you do alias foo to bar and you have a foo, you will get a bar but if you further alias bar to bat, you will NOT get a bat.

0 Karma

woodcock
Esteemed Legend

If you need BOTH, then you need a Calculated field defined like this:

dest = mvappend(destination, dst)
0 Karma

woodcock
Esteemed Legend

This would be better done as a Calculated field defined like this:

dest = coalesce(destination, dst)
0 Karma

FrankVl
Ultra Champion

Coalesce will only get him the first non-null value. He wants both values in the field in case both exist.

0 Karma

woodcock
Esteemed Legend

See new answer.

0 Karma

somesoni2
Revered Legend

With field aliases, in case of multiple entries with same target/renamed field name, the one that's applied last will take precedence. Since you want the rename to work conditionally (based on availability of field dst and destination, try the calculated field, like this (props.conf on your search head)

EVAL-dest = case(isnotnull(dst) AND isnotnull(destination),dst.",".destination,isnotnull(dst),dst,true(),destination)
0 Karma

FrankVl
Ultra Champion

What is your desired outcome?

If you are aiming at having dest as a multivalued field in those cases where the event contains dst and destination, you should look at something else than field aliases.

0 Karma

shayhibah
Path Finder

I want that dest will contains both dst & destination values

0 Karma
Take the 2021 Splunk Career Survey

Help us learn about how Splunk has
impacted your career by taking the 2021 Splunk Career Survey.

Earn $50 in Amazon cash!