Hi,
I have a props file which contains the following:
FIELDALIAS-aob_gen_alias_4 = dst AS dest
FIELDALIAS-aob_gen_alias_5 = destination AS dest
In case I have some logs with both fields (dst, destination), the final result is that the dest field contains the values of destination field ONLY and not dst & destination.
Can someone tell me whats wrong with my conf file?
Thanks
woodcock is on the right track here. Best method for this from my investigation on several cases as a Splunk TSE yielded the following:
Index time method to accomplish ONE field, renamed as MULTIPLE field names:
EVAL-foo = coalesce (field1,field2,etc...) thus creating indexed data that combines multiple disparate sourcetypes into one common fieldname as indexed.
Further info:
If you have defined a field alias, Splunk will only view the alias name as the data that is being turned into that field name, so for example:
FIELDALIAS- = AS foo
FIELDALIAS- = AS bar
FIELDALIAS- = foo AS
Using this example from above, and with the condition that there is a secondary sourcetype that already ingests a field with the name of 'foo" the following condition will occur -
The data that comes in natively as 'foo' will not return under these conditions, as Auto-KV extracts the native field of foo [no longer indexing the data as 'foo'], and is also parsing the new field alias for foo, however, since != foo this causes a break. The break is that the native 'foo' data will then be overwritten as a null value because it will not include data, thus the field 'foo' will be treated as an empty field with no data. Along with this, multiple iterations of the FIELDALIAS within different app contexts can negatively impact one another and cause "missing data" based on order of operations, and which field ends up overwriting the others, though this behavior may be dependent upon app permissions/sharing.
Prior to 7.2.0, to tie multiple disparate sourcetypes to a single field name, you would have to ingest them all with different names and then perform an EVAL(coalesce) function at search time to make field1,field2, ... all come out as a singular field name similar to the following:
| eval foo=coalesce(field1,field2,field*...)
POST 7.2.0 the ability to inject an EVAL statement at indexing time was introduced, allowing for props.conf entries to create the following:
EVAL-foo = coalesce (field1,field2,...) thus creating indexed data that combines multiple disparate sourcetypes into one common fieldname as indexed.
For expected field alias behavior, the following applies:
FIELDALIAS- = AS foo
FIELDALIAS- = AS bar
For examples of usage that will NOT work, the following applies:
FIELDALIAS- = AS foo
FIELDALIAS- = AS foo
OR
FIELDALIAS- = AS foo
FIELDALIAS- = AS bar
FIELDALIAS- = foo AS
OR
Multiple iterations of
App-Context1 props.conf FIELDALIAS- = AS foo
App-Context2 props.conf FIELDALIAS- = AS foo
App-Context3 props.conf FIELDALIAS- = AS foo
I pretty much agree with what you said except that EVAL-
is the original search-time
field extraction that does not create an indexed field. I think that you meant INGEST_EVAL-
for index-time
extractions created indexed fields. Also your example seems to imply that transitive field aliases work but they DO NOT. If you do alias foo
to bar
and you have a foo
, you will get a bar
but if you further alias bar
to bat
, you will NOT get a bat
.
If you need BOTH, then you need a Calculated field
defined like this:
dest = mvappend(destination, dst)
This would be better done as a Calculated field
defined like this:
dest = coalesce(destination, dst)
Coalesce will only get him the first non-null value. He wants both values in the field in case both exist.
See new answer.
With field aliases, in case of multiple entries with same target/renamed field name, the one that's applied last will take precedence. Since you want the rename to work conditionally (based on availability of field dst and destination, try the calculated field, like this (props.conf on your search head)
EVAL-dest = case(isnotnull(dst) AND isnotnull(destination),dst.",".destination,isnotnull(dst),dst,true(),destination)
What is your desired outcome?
If you are aiming at having dest as a multivalued field in those cases where the event contains dst and destination, you should look at something else than field aliases.
I want that dest will contains both dst & destination values