topic Re: How we can parse these Event ? in Splunk Search

How we can parse these Event ?

AL3Z — Tue, 02 Jan 2024 15:46:22 GMT

Hi friends,

Could anyone pls help me in parsing these event and use case( when ever we launch rdp/proxy from secret server we are seeing some drop in the connection like one for the client and another for the user) out of this event. how we can we extract the relevant fields to make a search out of these events ?

1. Sample event :

2024-01-02 10:04:01,420 [CID:] [C:] [TID:151] ERROR Thycotic.RDPProxy.CLI.Session.ProxyConnection - Error encountered in RDP handshake for client xx.xx.xx.xx:53475 - (null)
System.Exception: Assertion violated: stream.ReadByteInto(bufferStream) == 0x03
at Thycotic.RDPProxy.ContractSlim.Assert(Boolean condition, String conditionStr, String actualStr)
at Thycotic.RDPProxy.Readers.ConnectionRequestProvider.ReadConnectionRequest(Stream stream, AuthenticationState clientState)
at Thycotic.RDPProxy.CLI.Session.ProxyConnection.<DoHandshakeAndForward>d__20.MoveNext()

2. Sample event :

2024-01-02 09:27:42,911 [CID:] [C:] [TID:137] ERROR Thycotic.DE.Feature.SS.RdpProxy.EngineRdpProxySessionService - An error was encountered while attempt to fetch proxy credentials for user 'Jhoncena' - (null)

Thanks

Re: How we can parse these Event ?

dtburrows3 — Tue, 02 Jan 2024 16:10:59 GMT

Something like this maybe?

<base_search> | rex field=_raw "\]\s+(?<log_level>[A-Z]+)\s+(?<log_type>[^\s]+)\s+\-\s+(?<message>.*?)\s+\-" | rex field=_raw "\-\s+\([^\)]+\)\s+(?<message_detail>(?:.*(?:[\r\n]+)*)*)" | rex field=_raw "user\s+\'(?<user>[^\']+)\'" | rex field=_raw "client\s+(?<client>\d{1,3}(?:\.\d{1,3}){3})\:(?<port>\d+)" | rex field=_raw "\[TID\:(?<tid>[^\]]+)\]"

Not familiar with the data but based on the 2 sample logs you provided was able to identify a few fields that I think would be useful. You can see from the screenshot below that these fields appear to be successfully extracted.

And to standardize the messages (at least for these examples) you can add an eval like this after the field extractions.

| eval standardized_message=replace(replace(message, "client\s+(\d{1,3}(?:\.\d{1,3}){3})\:(\d+)", "client"), "user\s+\'([^\']+)\'", "user")

You can see in the screenshot that the field 'standardized_message' removes any variable data so you can do some sort of aggregate analysis against the standardized message itself if needed.

Re: How we can parse these Event ?

AL3Z — Tue, 02 Jan 2024 16:29:04 GMT

Thanks @dtburrows3 ,

I have another use case is discovery process from AD to Secret server like scan AD and finds the local id creates the id/password into secret.

1. Sample event:

2024-01-02 11:18:28,597 [CID:] [C:] [TID:131] ERROR Thycotic.Discovery.Sources.Scanners.Dependency.ApplicationPoolScanner - WMI (IIS) Unable to connect to WINDOWS1.mf.win.cisco.com with Exception System.Management.ManagementException: Invalid namespace
at System.Management.ThreadDispatch.Start()
at System.Management.ManagementScope.Initialize()
at Thycotic.Discovery.Sources.Scanners.Dependency.ApplicationPoolScanner.<>c__DisplayClass10_0.<IsIisRunningWmi>b__0(Object x) - (null)

2. Sample event:

2024-01-02 11:18:14,330 [CID:] [C:] [TID:PriorityScheduler Elastic Thread @ Normal] ERROR Thycotic.Discovery.Sources.Scanners.Dependency.ApplicationPoolScanner - Service Controller (IIS) Unable to connect to LINUX.mf.win.cisco.com with Exception System.InvalidOperationException: Cannot open W3SVC service on computer 'LINUX.mf.win.cisco.com'. ---> System.ComponentModel.Win32Exception: Access is denied
--- End of inner exception stack trace ---
at System.ServiceProcess.ServiceController.GetServiceHandle(Int32 desiredAccess)
at System.ServiceProcess.ServiceController.GenerateStatus()
at System.ServiceProcess.ServiceController.get_Status()
at Thycotic.Discovery.Sources.Scanners.Dependency.ApplicationPoolScanner.IsIisRunningServiceController() - (null)

3. Sample Event :

2024-01-02 11:18:06,544 [CID:] [C:] [TID:PriorityScheduler Elastic Thread @ Normal] ERROR Thycotic.Discovery.Sources.Scanners.Dependency.ApplicationPoolScanner - Retrieving the COM class factory for remote component with CLSID {45RFGHDH-FFSG-6474-HDHDH-1454TGGD} from machine UBUNTU.mf.win.cisco.com failed due to the following error: 80070005 UBUNTU.mf.win.cisco.com. - (null)
System.UnauthorizedAccessException: Retrieving the COM class factory for remote component with CLSID {2YHFDO-3443-4FGDH-5353-RFRRE6} from machine UBUNTU.mf.win.cisco.com failed due to the following error: 80070005 UBUNTU.mf.win.cisco.com.
at Thycotic.Discovery.Sources.Scanners.Dependency.ApplicationPoolScanner.ScanIIS7AndAbove()

4. Sample event :

2024-01-02 11:18:03,664 [CID:] [C:] [TID:PriorityScheduler Elastic Thread @ Normal] ERROR Thycotic.Discovery.Sources.Scanners.Dependency.ApplicationPoolScanner - Service Controller (IIS) Unable to connect to IHUIHD13.mf.win.cisco.com with Exception System.InvalidOperationException: Cannot open W3SVC service on computer 'YHGVDJ.mf.win.cisco.com'. ---> System.ComponentModel.Win32Exception: Access is denied
--- End of inner exception stack trace ---
at System.ServiceProcess.ServiceController.GetServiceHandle(Int32 desiredAccess)
at System.ServiceProcess.ServiceController.GenerateStatus()
at System.ServiceProcess.ServiceController.get_Status()
at Thycotic.Discovery.Sources.Scanners.Dependency.ApplicationPoolScanner.IsIisRunningServiceController() - (null)

Thanks.

Re: How we can parse these Event ?

dtburrows3 — Tue, 02 Jan 2024 17:13:16 GMT

Getting a bit more complex but I think something like this will get you started.

| rex field=_raw "\]\s+(?<log_level>[A-Z]+)\s+(?<log_type>[^\s]+)\s+\-" | rex field=_raw "\-\s+\([^\)]+\)\s+(?<message_detail>(?:.*(?:[\r\n]+)*)*)" | rex field=_raw "user\s+\'(?<user>[^\']+)\'" | rex field=_raw "client\s+(?<client>\d{1,3}(?:\.\d{1,3}){3})\:(?<port>\d+)" | rex field=_raw "\[TID\:(?<tid>[^\]]+)\]" | rex field=_raw "\]\s+[A-Z]+\s+[a-zA-Z0-9]+(?:\.[a-zA-Z0-9]+)*\s+\-\s+(?<message>.*)" | rex field=_raw "(?i)cannot\s+open\s+(?<service>[^\s]+)\s+service\s+on\s+computer\s+\'(?<computer>[^\']+)\'" ``` log_type="Thycotic.Discovery.Sources.Scanners.Dependency.ApplicationPoolScanner" ``` | rex field=_raw "(?i)\s+\-\s+(?<app>.*?)\s+unable\s+to\s+connect\s+to\s+(?<hostname>[^\s]+)\s+with\s+exception\s+(?<exception_type>[^\:]+)\:\s+(?<exception_message>.*)" ``` log_type="Thycotic.Discovery.Sources.Scanners.Dependency.ApplicationPoolScanner" error_message ``` | rex field=_raw "(?i)\s+\-\s+retrieving\s+the\s+(?<class>[^\s]+)\s+class\s+factory\s+for\s+remote\s+component\s+with\s+clsid\s+\{(?<clsid>[^\}]+)\}\s+from\s+machine\s+(?<hostname>[^\s]+)\s+failed\s+due\s+to\s+the\s+following\s+error\:\s+(?<error_code>[^\s]+)" | rex field=_raw "(?i)(?<exception_type>\w+(\.\w+)*exception)\:\s+(?<exception_message>.*)"

Since the evals to standardize messages probably isn't all that scalable with these logs, it may make sense to set up eventtyping around them to assist with any sort of analysis. Once you have eventtypes set up that can assist with targeting specific type of events and finding patterns that you can develop regex for and pull out all the necessary fields.

screenshot below for reference of what the above regex should do.

Re: How we can parse these Event ?

AL3Z — Tue, 09 Jan 2024 10:27:18 GMT

I'm trying to build a custom app to parse the all these events and above field extraction into props and tranforms can you help me in adding all these extractions in to props.conf ?
Sample Events:
2024-01-03 05:06:09,590 [TID:450] ERROR Thycotic.DistributedEngineRunner - Error getting Capabilities from Distributed Engine server - (null)
System.NullReferenceException: Object reference not set to an instance of an object.

TQ

Re: How we can parse these Event ?

dtburrows3 — Wed, 03 Jan 2024 19:15:28 GMT

A props.conf for these extractions would look like this.

[<sourcetype_name>] EXTRACT-log_level_and_type = \[TID\:(?<tid>[^\]]+)\]\s+(?<log_level>[A-Z]+)\s+(?<log_type>[^\s]+) EXTRACT-cid = \[CID\:(?<cid>[^\]]+)\] EXTRACT-message = [A-Z]+\s+\w+(?:\.\w+)*\s+\-\s+(?<message>.*)\s+\-\s+\( EXTRACT-user = user\s+\'(?<user>[^\']+)\' EXTRACT-client_ip = client\s+(?<client>\d{1,3}(?:\.\d{1,3}){3})\:(?<port>\d+) EXTRACT-cannot_open_service_error = (?i)cannot\s+open\s+(?<service>[^\s]+)\s+service\s+on\s+computer\s+\'(?<computer>[^\']+)\' EXTRACT-unable_to_connect_to_host_exception = (?i)\s+\-\s+(?<app>.*?)\s+unable\s+to\s+connect\s+to\s+(?<hostname>[^\s]+)\s+with\s+exception\s+(?<exception_type>[^\:]+)\:\s+(?<exception_message>.*) EXTRACT-retrieving_class_failed_due_to_error = (?i)\s+\-\s+retrieving\s+the\s+(?<class>[^\s]+)\s+class\s+factory\s+for\s+remote\s+component\s+with\s+clsid\s+\{(?<clsid>[^\}]+)\}\s+from\s+machine\s+(?<hostname>[^\s]+)\s+failed\s+due\s+to\s+the\s+following\s+error\:\s+(?<error_code>[^\s]+) EXTRACT-exception_messages = (?i)(?<exception_type>\w+(\.\w+)*exception)\:\s+(?<exception_message>.*) EXTRACT-error_codes = (?i)due\s+to\s+error\s+(?<error_code>[^\s]+)

And the accompanying default.meta something like this (depending on your desired permissions)

[props] access = read : [ * ], write : [ admin, power ] export = system