topic Re: Where to edit props.config for breaking log into multiple events? in Getting Data In

Where to edit props.config for breaking log into multiple events?

itdeptPFS — Wed, 15 Aug 2018 22:45:52 GMT

I am using universal forwarders to move log data from remote servers to a centralized Splunk Light server. Where do I edit the props.config? On the remote server or on the centralized Splunk Light server? When I search for props.config, I am returned server files, which one should I choose? I am a surprised that Splunk doesn't automatically break these into events because my log file has a blank space between each event.

MessageType: INFO | ApplicationName: MicroSaleDataTransporter | ApplicationVersion: 1.3.0.0 | Message: Hourly...

MessageType: INFO | ApplicationName: MicroSaleDataTransporter | ApplicationVersion: 1.3.0.0 | Message: Successfully....

MessageType: INFO | ApplicationName: MicroSaleDataTransporter | ApplicationVersion: 1.3.0.0 | Message: File created...

MessageType: INFO | ApplicationName: MicroSaleDataTransporter | ApplicationVersion: 1.3.0.0 | Message: People Counter...

MessageType: INFO | ApplicationName: MicroSaleDataTransporter | ApplicationVersion: 1.3.0.0 | Message: Successfully...

Thanks,
Chris

Re: Where to edit props.config for breaking log into multiple events?

richgalloway — Thu, 16 Aug 2018 00:13:19 GMT

Edit the props.conf file wherever event parsing is done. This is usually your indexer, but could also be a heavy forwarder.

You probably have several different props.conf files. The one to edit is the one in the app the corresponds to the data you are indexing. Be sure to edit local/props.conf (create it if you need to) rather than default/props.conf.

Splunk expects events to be separated by line-end characters (\n on Linux, \r\n on Windows) and to have a timestamp. It's best, however, not to allow Splunk to make guesses about the format of your events. Instead, use props.conf to describe your data. At the very least, include the TIME_PREFIX, TIME_FORMAT, LINE_BREAKER, SHOULD_LINEMERGE, TRUNCATE, and MAX_TIMESTAMP_LOOKAHEAD attributes.

Re: Where to edit props.config for breaking log into multiple events?

pruthvikrishnap — Tue, 29 Sep 2020 20:52:43 GMT

Hi itdept,

Description by richgalloway is very details and will work, it should be something like this basing on your data.
[sourcetype]
SHOULD_LINEMERGE=true
NO_BINARY_CHECK=true
BREAK_ONLY_BEFORE=Regex
TIME_FORMAT=%H:%M:%S.%3N
TIME_PREFIX=^
MAX_TIMESTAMP_LOOKAHEAD=12

Re: Where to edit props.config for breaking log into multiple events?

itdeptPFS — Thu, 16 Aug 2018 14:27:54 GMT

Thank you! I created the props.conf file on the indexer and it is working. I am still not exactly sure how this fowarder creates a SourceType name but I am finally getting my logs broken correctly.

Re: Where to edit props.config for breaking log into multiple events?

richgalloway — Fri, 17 Aug 2018 11:02:45 GMT

The sourcetype is specified in the forwarder's inputs.conf file.