Getting Data In

How come the index set on the forwarder is not respected on the indexer?

brianpreston
Path Finder

Hello!

I have a number of transforms setting indexes on my forwarder in transforms.conf, like:

[syslog_change_innobackup_log_to_index_bar]
REGEX = ^<15\d>.*\sinnobackupex\s.*
FORMAT = bar
DEST_KEY = _MetaData:Index
WRITE_META = true

When I view these on a test system, which is both running as an indexer and has these transforms, the indexes are set correctly. Yay!

BUT!

In production, when I forward these logs with the new index, to the production indexer, the indexes are no longer set! They are all the original index as set in inputs.conf.

Am I missing something on the forwarder? Maybe having to do with outputs.conf?

0 Karma

MuS
Legend

Hi brainpreston,

look at this wiki page (Sorry docs team, I only have this handy 😉 ) http://wiki.splunk.com/Where_do_I_configure_my_Splunk_settings%3F and you see that your settings must be done where the parsing of the events takes place. So either place your props.conf and transforms.conf on a heavy weight forwarder or the indexer.

Remember to restart Splunk to apply the configuration and it will only apply to new events.

Hope this helps ...

cheers, MuS

brianpreston
Path Finder

MuS, a few more architecture questions:

  • A heavyweight forwarder would be something that installs the package "splunk" rather than "splunkforwarder", is that correct?
  • Let's say I have these machines be Splunk Enterprise installations. When they receive the syslogs, that counts against the license, is that correct?
  • Can these logs be then forwarded to the index (and stored) without the logs being "double counted" in the license? Is that inherent in "heavyweight forwarder" ?
0 Karma

MuS
Legend

to answer the additional questions:

  1. Install the splunk package
  2. The received syslog will be count against the license on the indexer
  3. yes they can be forwarder and no they will not be double count, because only the indexer will do this part not a heavyweight forwarder

Hope this helps ...

0 Karma

brianpreston
Path Finder

yes thanks!

0 Karma

brianpreston
Path Finder
0 Karma
Get Updates on the Splunk Community!

Introducing the Splunk Community Dashboard Challenge!

Welcome to Splunk Community Dashboard Challenge! This is your chance to showcase your skills in creating ...

Get the T-shirt to Prove You Survived Splunk University Bootcamp

As if Splunk University, in Las Vegas, in-person, with three days of bootcamps and labs weren’t enough, now ...

Wondering How to Build Resiliency in the Cloud?

IT leaders are choosing Splunk Cloud as an ideal cloud transformation platform to drive business resilience,  ...