How to use metadata/header for all the following events

[Flobzh]

Dear all,

I'm trying to retrieve some log metadata and associate them to all my events.

Exemple: 

When my application starts, I'll get a few lines with what I'm calling metadata here (version, env, user, ... ) and then, the raw logs start.

2021-05-10T09:53:21.122+02:00|Criticity=INFO|Message=Version:3.4;Env=production

2021-05-10T09:53:46.474+02:00|Criticity=INFO|Message=blabla
2021-05-10T09:53:46.474+02:00|Criticity=DEBUG|Message=blabla2
2021-05-10T09:53:46.478+02:00|Criticity=DEBUG|Message=blabla3

I want this Version and Env to be usable as a field in all my events.

Like if each event looked something like this from a sub-query search standpoint:

2021-05-10T09:53:46.474+02:00|Criticity=INFO|Message=blabla|Version:3.4;Env=production
2021-05-10T09:53:46.474+02:00|Criticity=DEBUG|Message=blabla2|Version:3.4;Env=production
2021-05-10T09:53:46.478+02:00|Criticity=DEBUG|Message=blabla3|Version:3.4;Env=production

What would be the solution to end up with such usage?

Context:

The application I want to monitor is a heavy client, the users can choose the environnement to connect to from their desktop, and I capture the logs via a UniversalForwarder to Splunk Cloud.

I don't have much control on the log format, I've to go with this one.

Thanks in advance for your help

[ITWhisperer · Accepted solution]

| rex "(?<verenv>Version:\d\.\d;Env=\w+)"
| streamstats last(verenv) as lastverenv
| eval lastverenv=if(lastverenv=verenv,null,"|".lastverenv)
| eval _raw=_raw.lastverenv