Splunk Search

inputs.conf - blacklist regex performance assistance

antb
Path Finder

Thank you in advance. Looking for some assistance with inputs.conf on Windows Systems. First, we modified inputs.conf located:
/opt/apps/splunk/etc/deployment-apps/Splunk_TA_windows/local/inputs.conf

1) Do we need to do anything for our splunk clients to pick up changes?
2) In terms of performance, and syntax does anyone have any concerns or recommendations to improve the performance on the 6 blacklists below? We believe they work but are unsure on performance.

[WinEventLog://Security]
disabled = 0
start_from = oldest
current_only = 0
evt_resolve_ad_obj = 1
checkpointInterval = 5
renderXml = false
index = windowsevents

blacklist1 = EventCode="4662" Message="(?i)Object Type:(?!\s*groupPolicyContainer)"
blacklist2 = EventCode="566" Message="(?i)Object Type:(?!\s*groupPolicyContainer)"
blacklist3 = 5154,4663,4689,5152,4627
blacklist4 = EventCode="4688" Message="(?im:New Process Name:).*(?i:SplunkUniversalForwarder\\bin\\)(?i:splunk\.exe|btool\.exe)"
blacklist5 = EventCode="4688" ComputerName="verybadscripts\.myco\.com" Message="(?im:New Process Name:.*(\\grep\.exe|\\awk\.exe))"
Blacklist6 = EventCode="(4624|4634|4672)" ComputerName="(?i:(.+noisycomp|.+loudercomp).+\.myco\.com)" Message="(?im:.*Account Name:\s+.*(noisycomp|loudercomp).*\$)"

As an example computers have names like abloudercomp01 and bcdloudercompx02 and so the account names would be abloudercomp01$ and bcdloudercompx02$ respectfully... I don't want to presume two chars followed by two numbers but can presume <=5 chars on either side of the loudercomp string(s) if it would make a large difference in performance.

For some reason, we aren't currently seeing any 4688 events at all (from any comp) after the BL 4 and 5 were added early today but it could just be that there isn't a lot of volume on the weekend.

0 Karma
1 Solution

moliminous
Path Finder

1) Assuming your Deployment Server (DS) is configured and the appropriate clients belong to Server Classes with the App, you just need to reload the DS by running the following command on the CLI of the DS:

/opt/splunk/bin/splunk reload deploy-server

For more information:
https://docs.splunk.com/Documentation/Splunk/7.3.2/Updating/Updateconfigurations#Redeploy_an_app_aft...

2) The first few regex statements should be the simpler ones that filter out the most events quickly, that way there are less events that the more complex regex has to process.
Specifically, I would move blacklist3 up to be blacklist1, and shift the rest down. Also, I would put blacklist5 above blacklist4 since it is more specific by having ComputerName without wildcards.

Without seeing the data it would be difficult to make other specific suggestions, but try these generic suggestions:

  • Limit the wildcards as much as possible
  • Put the raw logs in regex101.com (or similar site) and test your regex to ensure it matches all the desired events while being as specific as possible

Hope this helps!

View solution in original post

woodcock
Esteemed Legend

I usually just restart splunk on the DS (does not require auth) to make all changes active.

0 Karma

moliminous
Path Finder

1) Assuming your Deployment Server (DS) is configured and the appropriate clients belong to Server Classes with the App, you just need to reload the DS by running the following command on the CLI of the DS:

/opt/splunk/bin/splunk reload deploy-server

For more information:
https://docs.splunk.com/Documentation/Splunk/7.3.2/Updating/Updateconfigurations#Redeploy_an_app_aft...

2) The first few regex statements should be the simpler ones that filter out the most events quickly, that way there are less events that the more complex regex has to process.
Specifically, I would move blacklist3 up to be blacklist1, and shift the rest down. Also, I would put blacklist5 above blacklist4 since it is more specific by having ComputerName without wildcards.

Without seeing the data it would be difficult to make other specific suggestions, but try these generic suggestions:

  • Limit the wildcards as much as possible
  • Put the raw logs in regex101.com (or similar site) and test your regex to ensure it matches all the desired events while being as specific as possible

Hope this helps!

Get Updates on the Splunk Community!

Webinar Recap | Revolutionizing IT Operations: The Transformative Power of AI and ML ...

The Transformative Power of AI and ML in Enhancing Observability   In the realm of IT operations, the ...

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...