Where to edit props.config for breaking log into multiple events?

[itdeptPFS]

I am using universal forwarders to move log data from remote servers to a centralized Splunk Light server. Where do I edit the props.config? On the remote server or on the centralized Splunk Light server? When I search for props.config, I am returned server files, which one should I choose? I am a surprised that Splunk doesn't automatically break these into events because my log file has a blank space between each event.

MessageType: INFO | ApplicationName: MicroSaleDataTransporter | ApplicationVersion: 1.3.0.0 | Message: Hourly...

MessageType: INFO | ApplicationName: MicroSaleDataTransporter | ApplicationVersion: 1.3.0.0 | Message: Successfully....

MessageType: INFO | ApplicationName: MicroSaleDataTransporter | ApplicationVersion: 1.3.0.0 | Message: File created...

MessageType: INFO | ApplicationName: MicroSaleDataTransporter | ApplicationVersion: 1.3.0.0 | Message: People Counter...

MessageType: INFO | ApplicationName: MicroSaleDataTransporter | ApplicationVersion: 1.3.0.0 | Message: Successfully...

Thanks,
Chris

[pruthvikrishnap]

Hi itdept,

Description by richgalloway is very details and will work, it should be something like this basing on your data.
[sourcetype]
SHOULD_LINEMERGE=true
NO_BINARY_CHECK=true
BREAK_ONLY_BEFORE=Regex
TIME_FORMAT=%H:%M:%S.%3N
TIME_PREFIX=^
MAX_TIMESTAMP_LOOKAHEAD=12

[richgalloway · Accepted solution]

Edit the props.conf file wherever event parsing is done. This is usually your indexer, but could also be a heavy forwarder.

You probably have several different props.conf files. The one to edit is the one in the app the corresponds to the data you are indexing. Be sure to edit local/props.conf (create it if you need to) rather than default/props.conf.

Splunk expects events to be separated by line-end characters (\n on Linux, \r\n on Windows) and to have a timestamp. It's best, however, not to allow Splunk to make guesses about the format of your events. Instead, use props.conf to describe your data. At the very least, include the TIME_PREFIX, TIME_FORMAT, LINE_BREAKER, SHOULD_LINEMERGE, TRUNCATE, and MAX_TIMESTAMP_LOOKAHEAD attributes.

[itdeptPFS]

Thank you! I created the props.conf file on the indexer and it is working. I am still not exactly sure how this fowarder creates a SourceType name but I am finally getting my logs broken correctly.

[richgalloway]

The sourcetype is specified in the forwarder's inputs.conf file.