Getting Data In

Universal Forwarder install/config questions/regex

rhansen
Explorer

I have read through the many documentation articles but they are all so broken apart that it is hard to piece together all the needed info.

Under - Manager» Data inputs» Add data» Files & directories» Data preview

It allows to me edit using a regex to define my fields. All my record events start with "At 16:38:54 -0500 - " so a regex of [0-9][0-9]:[0-9][0-9]:[0-9][0-9] or something similar will catch the lines.

This works, but only for the local Splunk server. If I try to grab the data off of a remote system, then I need to use the universal forwarder. This is where my problem starts.

First, I configured the forwarder via the GUI and it appears to work, kind of. It never presented me with any options on how to define the data that it found in the log files.

Then, when I thought it wasn't working, I am unable to figure out how to modify the forwarder to change its configuration. It just runs as a service and I'm unable to modify any options.

So, how do I modify the config of the forwarder? and is it possible to define the data via regex?

This is running 5.0.1 running on Windows for both server and clients.

0 Karma

Ayn
Legend

Fields are generally extracted at search-time, so any configuration related to that will be done on the instance you use for searching (typically the indexer in a simple deployment). So, you shouldn't need to worry about forwarder configurations.

Once you have data coming into the indexer from your forwarders, you can use the interactive field extractor to define your fields. More information on field extraction is available here: http://docs.splunk.com/Documentation/Splunk/5.0/knowledge/Addfieldsatsearchtime

yannK
Splunk Employee
Splunk Employee

the format is defined by the sourcetype.
It can be event linebreaking, timestamp detection, field extraction

  • on the forwarders, you specify the sourcetype in the inputs.conf
  • on the indexer (and any heavy forwarders if any ), you define the sourcetype in props.conf to do the indextime parsing (linebreaking and timestamp)
  • any search time field extractions will need to exist on the search-head (if any) in props.conf and transforms.conf
0 Karma

Ayn
Legend

I'm not sure I understand your first question - what do you mean by "data types" and where would the pre-definition of this be done (and, to what end)?

Splunk configuration, particularly on forwarders where the web interface is generally disabled, is most often done using its configuration files. These reside in various directories under etc in the directory where Splunk is installed.

0 Karma

rhansen
Explorer

I will read that article, but let me expand my question so that we are all on the same page.

Is there no way to pre-define what data types will be coming from a specific host via the Universal Forwarder? I did read in the documentation that the forwarder is just a dumb process, it does not parse, just sends the data blindly, so I understand that it will not do the parsing.

Secondly, how do I reconfigure the forwarder that is installed on my client? Once it was setup and running, I found no way to modify it's configuration.

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...