Review this method for exchange message trace and regex

[jwalzerpitt]

We are ingesting Exchange message trace logs and the username is not being pulled correctly. Reviewing the default/props.conf file I see the following field extraction:

EXTRACT-user = ^(?:[^"\n]*"){7}(?P<user>\w+)

An Exchange message trace example is as follows:

{"EventReceivedTime":"2019-06-27 11:00:44","SourceModuleName":"EXCHGETMESGTRACEPRD","SourceModuleType":"im_file","MessageId":"<ced1764ac97b40e6abb835f03494eaf4@ncmmail.lead.ncmstl.com>","Received":"6/27/2019 10:48:21 AM","SenderAddress":"pthurmond@ncmstl.com","RecipientAddress":"user@blah.blah.edu","Subject":"RE: Information Request","Status":"Delivered","FromIP":"1.1.1.1","Size":"118216"}

I tested a new regex (below) against sample Exchange message trace events and it's pulling the username from the correct location:

^(?:[^"\n]*"){23}(?P<user>\w+)

My plan is to comment out the regex in the default/props.conf file, then create a new local/props.conf file in the local directory for the add-on and then push via my Deployment servers to my deployers servers and from there down to the two SH clusters I have.

Just wanted to throw this up to have someone double check my process to make sure that a) I'm not missing any steps and b) verify that when I search the Exchange message trace events no matter the time range, the user field will be populated with the correct values

Thx

[FrankVl]

See the answer from @woodcock for the actual answer to your question regarding the deployment steps to follow.

Just a question out of curiosity: where did this original regex come from? Is this from a splunkbase add-on, or something custom developed by a colleague of yours or so? Because it seems to be a really weird regex. Using regexes that purely look at counting parts of a message is anyway a really weird way of dealing with what looks like json data. Either write specific regexes like "RecipientAddress":"(?<user>[^"]+) or use automatic json extractions and some field aliasing? That would be much easier to maintain and much more robust against changes in the log format as it wouldn't depend on the number and order of fields.

[jwalzerpitt]

Was custom developed by someone else.

The main issue I'm having is that I'm building a dashboard that I want to be able to enter a username which is them tokenized and runs against three separate searches, each a different sourcetype, but the user from Exchange message trace logs is a) not the real user and b) the value I need to pull for the user I need before the @blah.blah so that the same user token can be used for all three searches in my dashboard.

I modified the regex you provided to "RecipientAddress":"(?<user>[^"]+)@which pulls the user name minus everything after the '@'

Thx again for the reply and info

[FrankVl]

Almost feels like it was developed for a different log format then...

Anyway, glad to hear we've been able to point you in the right direction and bring you closer to a working dashboard 🙂

[woodcock · Accepted solution]

It is not necessary (and indeed is undesirable) to change anything in default. Just put your new stuff in local and it will override the stuff in default. That is the whole point.

[jwalzerpitt]

Thx - totally forgot and glad I asked!