Getting Data In

What are the capabilities of the "force_local_processing"

walterk82
Path Finder

Does anyone know the full effects of the new option "force_local_processing "? How does it change the following information: https://wiki.splunk.com/Where_do_I_configure_my_Splunk_settings%3F

What are the aggregator and regex replacement processors?

mglauser_splunk
Splunk Employee
Splunk Employee

Hello,

Splunk does not recommend using the force_local_processing property unless if you’ve been advised to by someone at Splunk. Switching this property on will potentially increase the cpu and memory consumption of the universal forwarder.

The force_local_processing option, when set to true (set to false, by default) forces a universal forwarder to process all data tagged with a sourcetype locally before forwarding it to the indexers. Data with this sourcetype will be processed via the linebreaker, aggregator and the regexreplacement processors in addition to the existing utf8 processor.

Note that force_local_processing is applicable only on a universal forwarder.

freedomson
Explorer

UPDATE:
After further debugging the reason for milliseconds not appear was very simple:

I was using a transforms.conf in the indexer with the format: DEST_KEY = _meta
Changed it to WRITE_META = true and all fine no need to force local processing anymore.

You may still read below for awareness...


Dear mglauser_splunk I recently had an issue parsing milliseconds on sourcetypes that my team had created and were not a default. Milliseconds did not got parsed at all. I tried a lot of setups on indexers and forwarders even changed date_time.xml and tried to train command (deprecated) with no sucess.

Splunk Enterprise
Version: 7.1.0
Build: 2e75b3406c5b
Source file: source::/var/log/springboot/dailyLogFile.log
Mask: [INFO ] 2019-02-01 11:02:13.178 ...

After trying for a couple of hours to get miliseconds parsed correctly the solution I found was to set on the Splunk Universal Forwarders property force_local_processing to true on

/opt/splunkforward/system/local/props.conf

[source::/var/log/springboot/dailyLogFile.log]
# 2019-02-01 11:02:13.178
# TIME_FORMAT=%Y-%m-%d %H:%M:%S.%3N -> It was proven not needed as datetime.xml seems to cover it ???
# TIME_PREFIX=^\[\w*\s*\]\s -> It was proven not needed as datetime.xml seems to cover it ???
force_local_processing=true

You need to restart Splunk Universal Forwarder to changes to take place:
/opt/splunkforwarder/bin/splunk restart

As I mentioned I tried to configure the setting at the indexers level using multiple configuration without sucess.
Ultimatly I found this property, force_local_processing at in https://docs.splunk.com/Documentation/Splunk/7.2.3/Admin/Propsconf

This was the only way I got milliseconds to be ingested.

Any ideia why ?

0 Karma
*NEW* Splunk Love Promo!
Snag a $25 Visa Gift Card for Giving Your Review!

It's another Splunk Love Special! For a limited time, you can review one of our select Splunk products through Gartner Peer Insights and receive a $25 Visa gift card!

Review:





Or Learn More in Our Blog >>