Splunk Dev

Keep specific events and discard the rest -Heavy Forwarder Setup

kevinbullock
New Member

I am setting up a heavy forwarder to keep specific events and discard the rest. My heavy forwarder is forwarding all of the events and not discarding anything. I am guessing I am either editing the incorrect files or my modifications are incorrect. The only events I would like to keep are Fatal and Warning events

All of the documentation I have read says to update the transforms.conf and props.conf in /etc/system/local. I am on a windows machine so that directory structure does not exist. There are few different props.conf and transforms.conf. I am editing the ones in c:\Program Files\Splunk\etc\apps\search\default. Are these the correct ones? If so, then I must have a problem somewhere else.

I have updated the transforms.conf and props.conf in c:\Program Files\Splunk\etc\apps\search\default as follows:

props.conf:
[source::C:\ProgramData\Folder1\Folder2\*.sts]
TRANSFORMS-set= setnull,setparsing

transforms.conf:
[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[setparsing]
REGEX = \[(Fatal|Warning)\]
DEST_KEY = queue
FORMAT = indexQueue

My Sample Data looks like this:
2019/01/14 14:29:36.356 - (47) [Informational] : TEST File: SCHD_Scd.Cpp Line: 963
SCHD-S-Information (1860) Releasing Locking logs...
2019/01/14 14:29:36.231 - (47) [Informational] : TEST File: SCHD_Scd.Cpp Line: 963
SCHD-S-Information (1860) Cleaning up Locked logs...
2019/01/14 14:29:36.106 - (47) [Informational] : TEST File: SCHD_Scd.Cpp Line: 963
SCHD-S-Information (1860) 479 Loaded 225 Scheduled
2019/01/14 14:29:35.950 - (47) [Informational] : TEST File: SCHD_Scd.Cpp Line: 963
SCHD-S-Information (1860) Releasing Locking logs...
2019/01/14 14:29:35.263 - (126) [Fatal] : TEST: c:\test\1\4\s\r\h\DB_Tran.Inl Line: 83
DB-F-RoutineFail (1272) Failure occurred in Routine: [CCDbTransaction::Commit].
2019/01/14 14:29:35.263 - (126) [Fatal] : TEST: DB_Con.Cpp Line: 601
DB-F-RoutineFail (1272) Failure occurred in Routine: [CCDbConnect::Execute].
2019/01/14 14:29:35.263 - (126) [Fatal] : TEST: DB_Con.Cpp Line: 598
DB-F-GeneralFailure (1272) A General Failure Occurred In Routine [CCDbConnect::Execute]. COMMIT TRANSACTION;.

The Source files that are being monitored by the universal forwarder and sent to the heavy forwarder are like this
C:\ProgramData\Folder1\Folder2\Test1.sts
C:\ProgramData\Folder1\Folder2\Test2.sts

The universal forwarder inputs.conf has the following:
[monitor://C:\ProgramData\Folder1\Folder2\*.sts]
current_only = 1
disabled = 0
start_from = oldest
sourcetype = stslog
index = sts

Any help would be appreciated! Thank you

Tags (1)
0 Karma

kevinbullock
New Member

Björn,
Thank you for the reply. This was all very useful information.
I tried populating C:\Program Files\Splunk\etc\apps\search\local with my settings for props.conf and transforms.conf, but that didn't work.
I ended up populating the settings in C:\Program Files\Splunk\etc\system\local for props.conf and transforms.conf. However, at first, this still didn't work.

The real problem was in Splunks documentation that I was using found here: Forwarding Data

Under the section, Keep specific events and discard the rest, I copied the this specific line for the profs.conf configuration:
TRANSFORMS-set= setnull,setparsing

The problem was that there needs to be a space inbetween the comma and "setparsing". This line should read like:
TRANSFORMS-set = setnull, setparsing

After adding the space, everything is working correctly now. I can't tell you how many hours I have spent trying to figure this out the last two days.

Again, thank you for your help! It is greatly appreciated.

0 Karma

bjoernjensen
Contributor

Hi,
c:\Program Files\Splunk\etc\apps\search\default
you should "never" edit c:\Program Files\Splunk\etc\apps\search\default ... that is product release defaults, since it is a default app.

In your case you should create and edit files in c:\Program Files\Splunk\etc\apps\search\local. Splunk will "merge" the configs at runtime.
https://docs.splunk.com/Documentation/Splunk/7.2.1/Admin/Configurationfiledirectories

In order to debug your current runtime configuration it is very handy to use the btool:
https://docs.splunk.com/Documentation/Splunk/7.2.3/Troubleshooting/Usebtooltotroubleshootconfigurati...

Note: If you want to grep in Windows, use the PowerShell in the following way (example):
C:\Program Files\Splunk\bin> .\splunk.exe cmd btool outputs list --debug | Select-String -Pattern "<REGEX_PATTERN>

Configuration should be described as here:
https://docs.splunk.com/Documentation/SplunkCloud/7.1.3/Forwarding/Routeandfilterdatad#Discard_speci...

Hope that guides you a little.

Cheerz - Björn

0 Karma
Get Updates on the Splunk Community!

Detecting Remote Code Executions With the Splunk Threat Research Team

WATCH NOWRemote code execution (RCE) vulnerabilities pose a significant risk to organizations. If exploited, ...

Enter the Splunk Community Dashboard Challenge for Your Chance to Win!

The Splunk Community Dashboard Challenge is underway! This is your chance to showcase your skills in creating ...

.conf24 | Session Scheduler is Live!!

.conf24 is happening June 11 - 14 in Las Vegas, and we are thrilled to announce that the conference catalog ...