About kmuellercm

kmuellercm · ‎02-04-2020

Oh I answered my own question.... I was going about it incorrectly, I needed to use SEDCMD rather than a transform. SEDCMD is also way easier and more straightforward but took a bit to get the syntax correct. The pipe's in the input were throwing me off, needed to escape them with a backslash replace top|otherClass| with just top| in all locations in the event (g) s/top\|otherClass\|/top\|/g Be sure you understand that this applies to _raw so make sure your match is specific and only ever matches that string. this is why i made sure to include the top| parameter. hopefully these don't move around arbitrarily 🙂 props.conf (on the indexer and search head--honestly not sure which worked) [ActiveDirectory] SEDCMD-fixObjectClass = s/top\|otherClass\|/top\|/g

kmuellercm · ‎02-03-2020

I have a unique situation where some of my users have a slightly different objectClass than usual and I'm trying to find a way to mask that so the default searches in the MS AD Objects app work properly Basically the users are being parsed as objectClass="top|otherClass|person|organizationalPerson|user" I want to selectively remote otherClass using a transform or props stanza but i'm unable to do so. I've tried the following on the indexer in the windows TA application: transforms.conf: [msad_fix_objectClass] SOURCE_KEY = _raw REGEX = (?ms).objectClass=(top|)(?:otherClass|)(person|organizationalPerson|user). FORMAT = objectClass::"$1$2" props.conf [ActiveDirectory] TRANSFORMS-objectClass = msad_fix_objectClass But it's not working properly. Anyone have ideas?

kmuellercm · ‎09-13-2019

This changes the sourcetype to 'stash' and is not configurable, just in case anyone finds this answer and thinks it's a work-around 🙂

kmuellercm · ‎09-06-2016

Yeah I see the same registry errors but not the CPDIR, but that could just be because you're running the lea_loggrabber manually. I'm definitely not seeing the invalid response error during the CRL download though, I think you should use openssl to verify it

kmuellercm · ‎09-06-2016

Seems to be choking on the CRL, is the URL http://cpserver.example.com:18264/ICA_CRL0.crl accessible? make sure you can pull it from the server. From the log server itself try the following command: curl http://cpserver.example.com:18264/ICA_CRL0.crl | openssl crl -inform der -text Here are the logs from the same point in my LEA connection: 2016-08-24 14:14:56,405 +0000 log_level=INFO, pid=5273, tid=Thread-9, file=ta_opseclea_data_collector.py, func_name=get_logs, code_line_no=62 | [input_name="Checkpoint NonAudit Events" connection="SplunkLEA-dedicated" data="non_audit"] CRL distribution Points: 2016-08-24 14:14:56,405 +0000 log_level=INFO, pid=5273, tid=Thread-9, file=ta_opseclea_data_collector.py, func_name=get_logs, code_line_no=62 | [input_name="Checkpoint NonAudit Events" connection="SplunkLEA-dedicated" data="non_audit"] URI: http://cpserver01.example.com:18264/ICA_CRL0.crl 2016-08-24 14:14:56,405 +0000 log_level=INFO, pid=5273, tid=Thread-9, file=ta_opseclea_data_collector.py, func_name=get_logs, code_line_no=62 | [input_name="Checkpoint NonAudit Events" connection="SplunkLEA-dedicated" data="non_audit"] DN: CN=ICA_CRL0,O=cpserver01.example.com.9my9sy 2016-08-24 14:14:56,406 +0000 log_level=INFO, pid=5273, tid=Thread-9, file=ta_opseclea_data_collector.py, func_name=get_logs, code_line_no=62 | [input_name="Checkpoint NonAudit Events" connection="SplunkLEA-dedicated" data="non_audit"] 2016-08-24 14:14:56,406 +0000 log_level=INFO, pid=5273, tid=Thread-9, file=ta_opseclea_data_collector.py, func_name=get_logs, code_line_no=62 | [input_name="Checkpoint NonAudit Events" connection="SplunkLEA-dedicated" data="non_audit"][ 5335 4152019776]@logserver01.example.com[24 Aug 10:14:56] fwCRL_CRLisValid: 2016-08-24 14:14:56,406 +0000 log_level=INFO, pid=5273, tid=Thread-9, file=ta_opseclea_data_collector.py, func_name=get_logs, code_line_no=62 | [input_name="Checkpoint NonAudit Events" connection="SplunkLEA-dedicated" data="non_audit"][ 5335 4152019776]@logserver01.example.com[24 Aug 10:14:56] thisUpdate: Tue Aug 23 05:00:02 2016 Local Time 2016-08-24 14:14:56,406 +0000 log_level=INFO, pid=5273, tid=Thread-9, file=ta_opseclea_data_collector.py, func_name=get_logs, code_line_no=62 | [input_name="Checkpoint NonAudit Events" connection="SplunkLEA-dedicated" data="non_audit"][ 5335 4152019776]@logserver01.example.com[24 Aug 10:14:56] nextUpdate: Tue Aug 30 05:00:02 2016 Local Time 2016-08-24 14:14:56,406 +0000 log_level=INFO, pid=5273, tid=Thread-9, file=ta_opseclea_data_collector.py, func_name=get_logs, code_line_no=62 | [input_name="Checkpoint NonAudit Events" connection="SplunkLEA-dedicated" data="non_audit"][ 5335 4152019776]@logserver01.example.com[24 Aug 10:14:56] now: Wed Aug 24 10:14:56 2016 Local Time 2016-08-24 14:14:56,406 +0000 log_level=INFO, pid=5273, tid=Thread-9, file=ta_opseclea_data_collector.py, func_name=get_logs, code_line_no=62 | [input_name="Checkpoint NonAudit Events" connection="SplunkLEA-dedicated" data="non_audit"][ 5335 4152019776]@logserver01.example.com[24 Aug 10:14:56] crl start grace period=600 crl end grace period=600 crl size val=0 2016-08-24 14:14:56,406 +0000 log_level=INFO, pid=5273, tid=Thread-9, file=ta_opseclea_data_collector.py, func_name=get_logs, code_line_no=62 | [input_name="Checkpoint NonAudit Events" connection="SplunkLEA-dedicated" data="non_audit"][ 5335 4152019776]@logserver01.example.com[24 Aug 10:14:56] fwCRL_CRLisValid: NOT checking nextUpdate 2016-08-24 14:14:56,406 +0000 log_level=INFO, pid=5273, tid=Thread-9, file=ta_opseclea_data_collector.py, func_name=get_logs, code_line_no=62 | [input_name="Checkpoint NonAudit Events" connection="SplunkLEA-dedicated" data="non_audit"][ 5335 4152019776]@logserver01.example.com[24 Aug 10:14:56] fwCRL_CRLisValid: NOT checking thisUpdate 2016-08-24 14:14:56,406 +0000 log_level=INFO, pid=5273, tid=Thread-9, file=ta_opseclea_data_collector.py, func_name=get_logs, code_line_no=62 | [input_name="Checkpoint NonAudit Events" connection="SplunkLEA-dedicated" data="non_audit"][ 5335 4152019776]@logserver01.example.com[24 Aug 10:14:56] fwCRL_CRLisValid: NOT checking CRL size 2016-08-24 14:14:56,406 +0000 log_level=INFO, pid=5273, tid=Thread-9, file=ta_opseclea_data_collector.py, func_name=get_logs, code_line_no=62 | [input_name="Checkpoint NonAudit Events" connection="SplunkLEA-dedicated" data="non_audit"][ 5335 4152019776]@logserver01.example.com[24 Aug 10:14:56] fwCRL_validateRevocation: validate cert: 2016-08-24 14:14:56,407 +0000 log_level=INFO, pid=5273, tid=Thread-9, file=ta_opseclea_data_collector.py, func_name=get_logs, code_line_no=62 | [input_name="Checkpoint NonAudit Events" connection="SplunkLEA-dedicated" data="non_audit"][ 5335 4152019776]@logserver01.example.com[24 Aug 10:14:56] Subject: CN=cpserver02,O=cpserver01.example.com.9my9sy

kmuellercm · ‎08-23-2016

Thank you! For some reason i didn't come across your answer in my searches.

kmuellercm · ‎08-23-2016

R77 with dedicated logging server Enabled a LEA connection and I get just a few logs, then the process bombs out. Logs from var/log/splunk/splunk_ta_checkpoint-opseclea_modinput.log indicate EventWriter encountered an exception, then stops processing inputs. I get about 150k log lines parsed properly. No obvious errors in splunkd.log and the loggrabber process remains running: /opt/splunk/etc/apps/Splunk_TA_checkpoint-opseclea/bin/../bin/lea_loggrabber --data non_audit --debug_level 2 --appname Splunk_TA_checkpoint-opseclea --lea_server_ip 1.2.3.4 --lea_server_auth_port 18184 --lea_server_auth_type sslca --opsec_sslca_file /opt/splunk/etc/apps/Splunk_TA_checkpoint-opseclea/certs/SplunkLEA.p12 --opsec_sic_name CN=SplunkLEA--opsec_entity_sic_name CN=MGMTSVR --last_record_location 1471966084 115426 --online --no_resolve var/log/splunk/splunk_ta_checkpoint-opseclea_modinput.log: 2016-08-23 16:13:16,654 +0000 log_level=INFO, pid=17396, tid=Thread-9, file=ta_opseclea_data_collector.py, func_name=get_logs, code_line_no=62 | [input_name="Checkpoint NonAudit Events" connection="SplunkLEA-dedicated" data="non_audit"]log_level=2 file:lea_loggrabber.cpp func_name:read_fw1_logfile_collogs code_line_no:2052 :LEA collected logfile handler was invoked 2016-08-23 16:13:27,679 +0000 log_level=ERROR, pid=17396, tid=Thread-1, file=event_writer.py, func_name=_do_write_events, code_line_no=79 | EventWriter encounter exception which maycause data loss, queue leftsize=3 Traceback (most recent call last): File "/opt/splunk/etc/apps/Splunk_TA_checkpoint-opseclea/bin/splunk_ta_checkpoint_opseclea/splunktalib/event_writer.py", line 62, in _do_write_events for evt in event: File "/opt/splunk/etc/apps/Splunk_TA_checkpoint-opseclea/bin/splunk_ta_checkpoint_opseclea/splunktaucclib/data_collection/ta_data_collector.py", line 59, in <genexpr> index, scu.escape_cdata(event.event)) for event File "/opt/splunk/etc/apps/Splunk_TA_checkpoint-opseclea/bin/splunk_ta_checkpoint_opseclea/splunktalib/common/util.py", line 71, in escape_cdata data = data.encode("utf-8", errors="xmlcharrefreplace") UnicodeDecodeError: 'utf8' codec can't decode byte 0xe9 in position 330: invalid continuation byte 2016-08-23 16:13:27,679 +0000 log_level=INFO, pid=17396, tid=Thread-1, file=event_writer.py, func_name=_do_write_events, code_line_no=84 | Event writer stopped, queue leftsize=4 2016-08-23 16:13:27,680 +0000 log_level=INFO, pid=17396, tid=Thread-4, file=ta_data_collector.py, func_name=_write_events, code_line_no=122 | [input_name="Checkpoint NonAudit Events" data="non_audit"] the event queue is closed and the received data will be discarded 2016-08-23 16:13:27,681 +0000 log_level=INFO, pid=17396, tid=Thread-4, file=ta_data_collector.py, func_name=index_data, code_line_no=114 | [input_name="Checkpoint NonAudit Events" data="non_audit"] End of indexing data for Checkpoint NonAudit Events_non_audit 2016-08-23 16:13:27,681 +0000 log_level=INFO, pid=17396, tid=Thread-4, file=thread_pool.py, func_name=_run, code_line_no=261 | Thread work_queue_size=0

kmuellercm · ‎08-23-2016

Might need a bit more info to troubleshoot, it seems the SIC connection isn't created What does your $splunkhome/etc/apps/Splunk_TA_checkpoint-opseclea/local/opseclea_connection.conf look like? What version is your management server? R77? What's the SIC status in the management server in Checkpoint? Does it have Trust Established? Do you have a separate logging server?

kmuellercm · ‎05-26-2016

Oh yeah I fixed this! Thanks for reminding me 🙂 I had 2 issues. First I'm using syslog-ng to recieve all my syslog before sending it to splunk. Syslog-ng adds a timestamp and the IP to each log line, secondly v11 has an extra field for protocol or log type (I'm not sure which) in the middle of the log, so the extract breaks. This will correctly parse the syslog header: \w{3}\s+\d{2}\s+\d{2}:\d{2}:\d{1,3}\s+\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\s+ This will correctly parse the v11 log line: \d{2}\/\d{2}\/\d{4}(:\d{2}){3}\s+\w{3}\s+(?P<ns_name>\S+)\s+(?P<packet_engine_name>\S+)\s+:\s+(?P<event_source>\w+)\s+(?P<proto>\w+)\s+(?P<event_name>\w+)\s+(?P<event_id>\d+)\s+0\s+:\s+(?P<syslog_message>.+) So the combined extract in my props.conf on the indexer in the splunkhome\etc\apps\Splunk_TA_netscaler\local\props.conf is: [citrix:netscaler:syslog] EXTRACT-1-syslog = \w{3}\s+\d{2}\s+\d{2}:\d{2}:\d{1,3}\s+\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\s+\d{2}\/\d{2}\/\d{4}(:\d{2}){3}\s+\w{3}\s+(?P<ns_name>\S+)\s+(?P<packet_engine_name>\S+)\s+:\s+(?P<event_source>\w+)\s+(?P<proto>\w+)\s+(?P<event_name>\w+)\s+(?P<event_id>\d+)\s+0\s+:\s+(?P<syslog_message>.+)

kmuellercm · ‎05-11-2016

Hello I'm using Splunk 6.3 and the Splunk Add-on for Citrix NetScaler 6.1.0. I am collecting syslog at a heavy forwarder (using syslog-ng) and forwarding via a file-monitor. I'm statically assigning the sourcetype (citrix:netscaler:syslog) and it shows up when I search for events with that sourcetype, however, none of the field extractions are populating except 'vendor' "Citrix Systems". I checked with btool and I can see my props on both the indexer and the heavy forwarder seem to be correct. Any ideas? Here is a sample raw log entry from syslog: May 11 08:41:52 10.250.255.187 05/11/2016:12:54:07 GMT ns01 0-PPE-0 : default SSLVPN LOGIN 7292348 0 : Context username@10.x.x.176 - SessionId: 22366- User username- Client_ip 10.x.x.176 - Nat_ip "Mapped Ip" - Vserver 10.x.x.51:443 - Browser_type "Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; Trident/6.0)" - SSLVPN_client_type ICA - Group(s) "VDIUsersGroup" Here is my inputs.conf on the heavy forwarder: [monitor:///opt/splunk/syslog/citrix/.../host.log] sourcetype = citrix:netscaler:syslog disabled = 0 host_segment = 5

Posts	10
Solutions	1
Karma Given	1
Karma Received	3
Member Since	‎05-11-2016

Online Status	Offline
Date Last Visited	‎06-05-2020 02:04 AM

MS Windows AD Objects change objectClass

Splunk Add-on for Check Point OPSEC LEA 4.0.0: "Un...

Splunk Add-on for Citrix NetScaler 6.1.0: Why are ...

Re: MS Windows AD Objects change objectClass

MS Windows AD Objects change objectClass

Re: How do I change the sourcetype after the data ...

Re: Splunk Add-on for Check Point OPSEC LEA (4.0):...

Re: Splunk Add-on for Check Point OPSEC LEA (4.0):...

Re: Splunk Add-on for Check Point OPSEC LEA 4.0.0:...

Splunk Add-on for Check Point OPSEC LEA 4.0.0: "Un...

Re: Splunk Add-on for Check Point OPSEC LEA (4.0):...

Re: Splunk Add-on for Citrix NetScaler 6.1.0: Why ...

Splunk Add-on for Citrix NetScaler 6.1.0: Why are ...