Getting Data In

Filter the data of a logfile at Universal Forwarder?

sarvesh_11
Communicator

Hi Splunker,

I am using splunkforwarder 6.5 on windows 2k8 servers. I am monitoring a log file, from splunk.
I have modified inputs.conf at Universal Forwarder.
The size of the file that i am monitoring is 130mb, out of which my useful data is of somewhere around 20Mb.
Can i restrict the unwanted data?
I have list of keywords, for which log event is required and to be indexed.

Is it possible to do the same at Universal Fowarder level?

TIA

0 Karma
1 Solution

gcusello
SplunkTrust
SplunkTrust

Hi òsarvesh_11,
you can filter your logs on Indexers or (if present) on Heavy Forwarders.
The instructions to filter events are at https://docs.splunk.com/Documentation/Splunk/8.0.3/Forwarding/Routeandfilterdatad#Filter_event_data_...

As you can read, you can:

  • discard all events and take some of them,
  • take all events and discard some of them.

The job to do is to find one or more regex to filter your data.

Only one hint: put attention to your architecture, because the props.conf and transforms.conf files to filter events must be located on:

  • Indexers (if you haven't Heavy Forwarders),
  • on Heavy Forwarders (if present and oll the traffic pass through them),
  • both on Indexers and Heavy Forwarders, if you have a mixed situation (some Universal Forwarders directly send data to Indexers and sone others through HFs).

Ciao.
Giuseppe

View solution in original post

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi òsarvesh_11,
you can filter your logs on Indexers or (if present) on Heavy Forwarders.
The instructions to filter events are at https://docs.splunk.com/Documentation/Splunk/8.0.3/Forwarding/Routeandfilterdatad#Filter_event_data_...

As you can read, you can:

  • discard all events and take some of them,
  • take all events and discard some of them.

The job to do is to find one or more regex to filter your data.

Only one hint: put attention to your architecture, because the props.conf and transforms.conf files to filter events must be located on:

  • Indexers (if you haven't Heavy Forwarders),
  • on Heavy Forwarders (if present and oll the traffic pass through them),
  • both on Indexers and Heavy Forwarders, if you have a mixed situation (some Universal Forwarders directly send data to Indexers and sone others through HFs).

Ciao.
Giuseppe

0 Karma

sarvesh_11
Communicator

Hi @gcusello ,
Yeah i am clear about indexer and/or Heavy forwarder, i was checking if we can place props.conf and transforms.conf at Universal Forwarder.
To filter out the logs at forwarder level only.

Because i ready mixed views for this.

Thanks,

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @sarvesh_11,
at Universal Forwarders level, you can filter events only from Windows eventlogs, as you can see at https://docs.splunk.com/Documentation/Splunk/latest/Admin/Inputsconf#WINDOWS_INPUTS:

Ciao.
Giuseppe

0 Karma
Get Updates on the Splunk Community!

Updated Team Landing Page in Splunk Observability

We’re making some changes to the team landing page in Splunk Observability, based on your feedback. The ...

New! Splunk Observability Search Enhancements for Splunk APM Services/Traces and ...

Regardless of where you are in Splunk Observability, you can search for relevant APM targets including service ...

Webinar Recap | Revolutionizing IT Operations: The Transformative Power of AI and ML ...

The Transformative Power of AI and ML in Enhancing Observability   In the realm of IT operations, the ...