Field renaming of some but not all fields

[bowesmana]

While on a mission to eradicate 'join', I was showing someone how to replace a join statement with stats.

However, the use case was that the user wanted to know, in the result output, which fields had come from the joined data set and which had come from the parent, which can be done quite efficiently with

index=a key="X"
| join key
[
  search index=b key=X
  | rename * as joined_*, joined_key as key
]

In principle, it's simple to replace the join with a single stats, but the challenge is how to rename the fields that have come from index=b, so they can be identified by their field names.

I came up with this theoretical example

index=_audit OR index=_internal
| foreach * 
    [ eval audit_<<MATCHSTR>>=if(index="_internal", null(), <<FIELD>>) ] 

 which will create new fields called audit_* for those fields in the _audit index, but it's not a rename. So I added

<<FIELD>>=if(index="_internal", <<FIELD>>, null()) 

to the eval, which didn't work as anticipated, i.e. the original _audit fields did not disappear.

One solution is to rename the fields at ingestion, but not ideal. Aliases doesn't really solve the problem, as the original fields are still present. 

Any other approaches to this?

[tscroggins]

@bowesmana 

You could try something like this:

| foreach * [ | eval {index}_<<FIELD>>=<<FIELD>> | fields - <<FIELD>> ]

Although you'll want to be more selective in the fields you iterate over and delete. Rather than deleting a field, you could just force its value to null in an if or case function.

[bowesmana]

@tscroggins 

Thanks for the suggestion,. I'd tried setting the fields to null in my original experiment, but for some reason the results were unpredictable and it was not delete fields - I didn't get time to investigate fully.