Exchange messagetracking. Again.

[FloydATC]

My universal forwarders (several of them) are now forwarding my Exchange messagetracking logs as sourcetype=messagetracking and they are indexed as such. I'm trying to add "props" and/or "transforms" on that indexer, although these terms confuse me.

I'm further confused by some articles mentioning that this needs to be done at each forwarder (which seems absolutely ludicrous to me) and some even state it needs to be done BOTH at each forwarder as well as the indexer.

As per some suggestions, I have tried editing "./apps/search/local/props.conf" like this:

[messagetracking]
FIELD_DELIMITER=,
FIELDS="exch_date_time","exch_client_ip","exch_client_hostname","exch_server_ip","exch_server_hostname","exch_source_context","exch_connector_id","exch_source","exch_event_id","exch_internal_message_id","exch_message_id","exch_recipient_address","exch_recipient_status","exch_total_bytes","exch_recipient_count","exch_related_recipient_address","exch_reference","exch_message_subject","exch_sender_address","exch_return_path","exch_message_info","exch_directionality","exch_tenant_id","exch_original_client_ip","exch_original_server_ip","exch_custom_data"

Others suggest my "./etc/system/local/props.conf" should contain something like this:

[messagetracking]
SHOULD_LINEMERGE=false
KV_MODE=none
REPORT-messagetracking=messagetracking-transform

...and then my "./etc/system/local/transforms.conf" should contain something like this:

[messagetracking-transform]
DELIMS=","
FIELDS="exch_date_time","exch_client_ip","exch_client_hostname","exch_server_ip","exch_server_hostname","exch_source_context","exch_connector_id","exch_source","exch_event_id","exch_internal_message_id","exch_message_id","exch_recipient_address","exch_recipient_status","exch_total_bytes","exch_recipient_count","exch_related_recipient_address","exch_reference","exch_message_subject","exch_sender_address","exch_return_path","exch_message_info","exch_directionality","exch_tenant_id","exch_original_client_ip","exch_original_server_ip","exch_custom_data"

But that doesn't seem to help either. And maybe that's because "system" is during ingest while "search" makes the conversion from a raw stream into indexable fields occur during search, which may be a good thing for my old data but seems inecfficient for new data.

Anyway, how to test these things...

$ ./splunk test
Command error: Additional arguments are needed for the 'test' command.  Please type "splunk help test" for usage and examples.

Okay?

$ ./splunk help test
     The 'test' and 'train' commands have been deprecated. 
     Type "help [object|topic]" to view help on a specific object or topic.

Well, I guess we won't get much useful information there. What about this suggestion from Google:

$ ./splunk cmd parsetest file /tmp/test.log messagetracking  
Conf is currently being modified by process 24011.
Conf is currently being modified by process 24011.
Conf is currently being modified by process 24011.

Right. I'm completely stuck, transformation doesn't seem to be happening at all and I have no clue why. Anyone care to give me a nudge in the right direction?

[MuS · Accepted solution]

Hi FloydATC,

take a look at this wiki article about where do I configure my Splunk settings, this is a good starting point to learn where you should configure settings.

As you are configuring searching stuff, this will be on the indexer or a search head if you have one.

Also change only one option/stanza at a time, don't forget to | extract reload=t (this reload props and transforms) or restart splunk and check the result and continue after that.

Some changes in transforms.conf like LOOKAHEAD, DEST_KEY, WRITE_META, DEFAULT_VALUE, REPEAT_MATCH will only apply to new data/events not your old/already indexed events.
But since you set DELIMS and FIELDS which are search operators this will be used for all data.

If you feel completely lost, remove everything and start with new empty/fresh props.conf and transforms.conf.

hope this helps to get you going ...

cheers, MuS

[MuS]

You're welcome, please tick the tick to mark this as answered

[FloydATC]

OK, moved the settings from "./etc/system/local/.conf" to "./etc/apps/search/local/.conf". That link was very helpful indeed, thank you so much! Also, the ability to reload changes with "| extract reload=true" should speed up future experimentation since I no longer have to wait for the service to restart all the time.

But most important: Everything now works properly!! 🙂