Splunk Enterprise Security

Why do discarded events on the forwarder keep failing?

MJA411
Explorer

Hello!

I am trying to exclude a specific computer_name from showing up in our carbonblack index in Splunk using a Heavy Forwarder where the data is monitored on. 

Below is an example of my props.conf & transform.conf

props.conf

[source::/var/data/events.json]
TRANSFORMS-null = nullFilter

transform.conf

[nullFilter]
REGEX = (?ms)(.*"computer_name":\s*"test-machine".*)
DEST_KEY = queue
FORMAT = nullQueue

 

Raw data in Splunk:

 

 

 

{"path":"/usr/sbin/abrt-server","md5":"9F469AA349AA64009C3DB7BE","sha256":"","command_line":"abrt-server -s","parent_path":"/usr/sbin/abrtd","parent_pid":546,"parent_guid":-390649270232,"filtering_known_dlls":false,"parent_md5":"97E3CDA03CB1A8CDF9","expect_followon_w_md5":false,"link_parent":"https://server-name:443/#analyze/00000000-0000-0000-0000-74e9a5a/1","username":"root","parent_create_time":1682147484,"pid":27474,"process_guid":"00000000-0000-0000-0000-0000000e","link_process":"https://server-name:443/#analyze/00000000-0000-0000-0000-99132070551e/0","link_sensor":"https://server-name:443/#/host/518","process_path":"/usr/sbin/abrt-server","cb_server":"server-name","type":"ingress.event.procstart","sensor_id":123,"computer_name":"test-machine","event_type":"proc","timestamp":1686123541}

 

 

 

 

List format data in Splunk (there are two ways the data is displayed in Splunk but the json file produced the above raw data)

 

 

 

{ [-]
   cb_server: server-name
   command_line: abrt-server -s
   computer_name: test-machine
   event_type: proc
   expect_followon_w_md5: false
   filtering_known_dlls: false
   link_parent: https://server-name:443/#analyze/00000000-0000-0000-0000-74e9a5a/1
   link_process: https://server-name:443/#analyze/00000000-0000-0000-0000-99132070551e/0
   link_sensor: https://sever-name:443/#/host/123
   md5: 9F469AA349AA64009C3DB7BE
   parent_create_time: 1682147484
   parent_guid: -390649270232
   parent_md5: 97E3CDA03CB1A8CDF9
   parent_path: /usr/sbin/abrtd
   parent_pid: 546
   path: /usr/sbin/abrt-server
   pid: 27474
   process_guid: 00000000-0000-0000-0000-99132070551e
   process_path: /usr/sbin/abrt-server
   sensor_id: 123
   sha256:
   timestamp: 1686123541
   type: ingress.event.procstart
   username: root
}

 

 

 

 

I have tried a few different regex entries but they keep failing. I was using a UF initially then read the Splunk docs and upgraded to a Heavy Forwarder but still keep getting the same problem. Can you please provide any assistance, would be very much appreciated.

My initial aim is to get this working for a single machine then hopefully look to exclude multiple machines sharing a similar naming convention for example "PC123..." grateful if you can provide best way to tackle both scenarios. 

Thanks!

Labels (3)
0 Karma
1 Solution

isoutamo
SplunkTrust
SplunkTrust

Last options. What happened when you switch [source::/var/data/events.json] to [bit9:carbonblack:json] on your props.conf?

If it didn't help then I suppose that you still have some precedence issue on your conf files, but it's hard to say without looking those trough with splunk btool <file> list <param> --debug.

View solution in original post

0 Karma

isoutamo
SplunkTrust
SplunkTrust

Hi

have you try it like

REGEX = (?ms).*"computer_name":\s*"test-machine".*

or even

REGEX = (?ms)"computer_name":\s*"test-machine"

 and if those are not working you could try to escape " as \"

r. Ismo

0 Karma

MJA411
Explorer

I tried both regex as entries but didn't change anything with the data coming into Splunk. I am doing this on the Heavy Forwarder where the data is monitored. Do I need to run this anywhere else? 

I thought it should work on the HF without any further changes elsewhere. 

0 Karma

isoutamo
SplunkTrust
SplunkTrust

You should run it on HF and after changes you must restart splunkd on HF. No need to change it on IDX as HF is the 1st full splunk instance and it modifying it only there.

Your example is _raw on splunk, but do you have raw on that /var/data/events.json file?

0 Karma

MJA411
Explorer

the /var/data/events.json file produces an identifcal events to the raw data. Splunk then show the data in either "raw" or "list" format as provided above. 

I have restarted splunk on the HF after making the changes by running

./splunk stop

./splunk start

under /opt/splunk/bin

0 Karma

isoutamo
SplunkTrust
SplunkTrust

Time by time those seems to contains same data, but when you are looking source file there could be some hidden data (like ctrl characters etc.). Those could leads the situation where your regex didn't work. For that reason I always try to use real source file instead of _raw from splunk when creating props + transforms.

0 Karma

MJA411
Explorer

Here is an output from the events.json as per the source in props.conf

 

{"Protocol":17,"domain":"","direction":"inbound","local_ip":"1.2.3.4","local_port":12345,"remote_ip":"1.2.3.4","remote_port":00,"proxy":false,"proxy_ip":"","proxy_domain":"","md5":"A652C59871009183","pid":10,"process_guid":"0000000e-0000-0000-0000-8e1da","link_process":"https://server-name:443/#analyze/0000003e-0000-0000-0000-8e1da/0","link_sensor":"https://server-name:443/#/host/00","process_path":"/opt/splunk/bin/splunkd","cb_server":"server-name","type":"ingress.event.netconn","sensor_id":00,"computer_name":"test-machine","event_type":"netconn","timestamp":000000000}

 

The regex works perfectly fine on Regex101.com and match all the events where the computer_name = test-machine. Not sure why the data is still indexed in Splunk. I tried to do a different field for example: event_type="proc" to be excluded but it didn't work either. So there has to be something stopping this from working but I can't seem to get my head around it. I had a few second opinions and they think what I did looks good and it should work. 

0 Karma

isoutamo
SplunkTrust
SplunkTrust

You are using normal monitor input for it?
Can you share that? 
Also you are sure that this props.conf are in use? You have checked it with splunk btool?

0 Karma

MJA411
Explorer

Here's the inputs.conf monitor part:

 

[monitor:///var/data/events.json]
sourcetype = bit9:carbonblack:json
index = carbonblack
initCrcLength = 2000

I have confirmed I can see the props line by running the splunk btool so it's visible. 

0 Karma

isoutamo
SplunkTrust
SplunkTrust

Last options. What happened when you switch [source::/var/data/events.json] to [bit9:carbonblack:json] on your props.conf?

If it didn't help then I suppose that you still have some precedence issue on your conf files, but it's hard to say without looking those trough with splunk btool <file> list <param> --debug.

0 Karma

MJA411
Explorer

Thanks! 

After running Splunk Btool transforms I noticed the entries are not there. Turned out the transforms.conf was incorrectly named to transform.conf (missing an ‘s’)….

quite a silly mistake but all working as expected now so thanks for your help!

Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In December, the Splunk Threat Research Team had 1 release of new security content via the Enterprise Security ...

Why am I not seeing the finding in Splunk Enterprise Security Analyst Queue?

(This is the first of a series of 2 blogs). Splunk Enterprise Security is a fantastic tool that offers robust ...

Index This | What are the 12 Days of Splunk-mas?

December 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...