topic Where do I exclude data from input? in Getting Data In

Where do I exclude data from input?

sov_gwright — Tue, 29 Sep 2020 22:30:01 GMT

Hi,

I'm sorry in advance for the really basic question but Splunk is all new to me and I couldn't find exactly what I want in the documentation.

I have a server class (_server_app_PIA_App_Servers) that has an input to read logs from a specific application log directory (Peopletools Application Servers in this case). The logs have a lot of unimportant and repeating data and I don't want to index, mainly just things such as "checking for processes" that repeats every 5 seconds. I want to exclude this data from the index so it's not taking up unnecessary space and I'm pretty sure I need to add a props.conf and transforms.conf to do this (sending those lines to null with a transform) however I don't know WHERE to do it - do I put these in the Universal Forwarder /etc/apps/ directory for these specific servers, or do I put them on the Indexer in the DeployedApps directory and redeploy the app?

Or somewhere else??

Thanks!

Grahame

Re: Where do I exclude data from input?

gcusello — Tue, 18 Dec 2018 12:50:19 GMT

Hi sov_gwright,
parsing phase is on Indexers or on Heavy Forwarders.
Follow documentation at https://docs.splunk.com/Documentation/Splunk/7.2.1/Forwarding/Routeandfilterdatad to filer your events.

Bye.
Giuseppe

Re: Where do I exclude data from input?

dkeck — Tue, 18 Dec 2018 12:51:57 GMT

HI,

this is set on the indexer. Depending if you have a cluster set it in /master-apps and apply a new bundle, or if standalone set it in an app/local or in system/local / or deploy it with deployment server in /deployment-apps

Like this:

props.conf

  [sourcetype]

     TRANSFORMS-<name>=<name_in_transforms>





transforms.conf

     [<name_in_transforms>]

    REGEX=""


     DEST_KEY=queue

     FORMAT=nullQueue

Re: Where do I exclude data from input?

sov_gwright — Tue, 18 Dec 2018 12:57:54 GMT

Thanks, I had looked at that but the section on filtering data to null is not explicit on where to put the props and transforms files.

Re: Where do I exclude data from input?

sov_gwright — Tue, 18 Dec 2018 12:59:13 GMT

Perfect, thanks! It's not a cluster, but since this app is not installed on the indexer (it's not in etc/apps/, only in etc/deployed-apps) I'll throw the files in system/local.

Re: Where do I exclude data from input?

dkeck — Tue, 18 Dec 2018 13:02:09 GMT

There is no directory called etc/deployed-apps, only deployment-apps. Deployed apps from a deployment server are deployed to /etc/apps on the client 🙂 have this in mind please 🙂

you can also just create an app in /etc/apps/ on the indexer to contain the props and transforms in

> /etc/apps/<app_name>/local