How do we Parse logs of Secret Server in Splunk Cloud

[AL3Z]

Hi,

Are there any available applications to address the issue of incorrect parsing of secret server logs in Splunk cloud?

Thnks

[AL3Z]

Hi @richgalloway ,
my final tranforms.conf and props.conf looks like this correct if format is not valid

[key_value_pair]
REGEX = (\w+)=(.*?)(?=\s\w+=|$)
FORMAT = $1::$2

props.conf

[mysourcetype]
category = Custom
description = Secret Server 
EXTRACT-event_id_name = \|Secret Server\|[^\|]*\|(?<event_id>[^\|]*)\|(?<event_name>[^\|]*)\|
#REPORT-key_value_pair = key_value_pair
KV_MODE = auto
SEDCMD-rm_cs2 = s/(cs2=.*?(cs|\s*$))/\2/
SEDCMD-rm_cs2Label = s/(cs2Label=.*?(cs|\s*$))/\2/
EVAL-group = if(match(cs2, "^Secret Server"), cs2, null())
EVAL-user = if(match(cs2, "^Secret Server"), null(), cs2)

Thanks

Thanks

[AL3Z]

@richgalloway Hi,

Instead of using SEDCMD-rm can we use like this ?

REGEX = cs4=(\w+) cs4Label=([\s]+)
FORMAT = $2::$1

eg: cs4= will smith
cs4Label=suser display name

Thanks..

[richgalloway]

That's an intriguing idea.  I think it would work except for the matter of the cs4Label value contains embedded spaces.  Field names with spaces can be a problem in Splunk.

[AL3Z]

Hi @richgalloway ,

Could you pls write the sed command to remove the space between the field names.

Thanks..

[richgalloway]

I don't have a SEDCMD that would do that well.  SEDCMD applies to _raw rather than individual fields so a regex would have to be very specific to avoid removing too many spaces or otherwise damaging other fields.

[AL3Z]

Hi@richgalloway ,
I've heard that the field name removal isn't feasible while we're receiving logs from the syslog server. Is that correct to your knowledge?
Thanks

[richgalloway]

I think that would depend on how the syslog data is received, but I believe it's still possible.

[AL3Z]

...

[richgalloway]

It should be done with two SEDCMD statements since the fields may not be next to each other.

SEDCMD-no_cs4 = s/cs4=.*?(\w+=)|$/\1/
SEDCMD-no_cs4Label = s/cs4Label=.*?(\w+=)|$/\1/
SEDCMD-no_cs3 = s/cs3=.*//
SEDCMD-no_cs3Label = s/cs3Label=.*?(\w+=)|$/\1/

I've also assumed the cs3 field is always last, but that may be erroneous..

[AL3Z]

Hi @richgalloway ,

Pls help me in extracting  the fields from the details value i.e approved=xyz, from=11/17/2023 06:22 AM , until =11/18/2023 12:00 AM, it should not be the event specific !

Details: Approved xyz from 11/17/2023 06:22 AM until 11/18/2023 12:00 AM.  

Thanks

[richgalloway]

In case the field may be in a different order, use multiple rex commands to extract them.

| rex "approved=(?<approved>[^,]+)"
| rex "from=(?<from>[^,]+)"
| rex "until =(?<until>[^,]+)"

I hope you see the pattern.

[AL3Z]

@richgalloway ,

How we can correlate data across different languages or datasets.

For the specific case of merging "Approuvé" (French) and "Approved" (English) fields..

 english :    Approved Sachin tendulakr from 11/25/2023 07:03 AM until 11/25/2023 03:03 PM. 

 french : Approuvé - Approuvé Salmon du 11/23/2023 02:10 PM au 12/23/2023 02:10 PM . 

English           French

Approved     Approuvé

from                du

until                au

Thanks

[richgalloway]

Any time multiple words are used for the same meaning, whether in different languages or the same language, they should be normalized before use.   I like to use the case function for that.

| eval status=case(status="Approved" OR status="Approuvé", "Approved",
                   1==1, "Denied")

 As for separator words in different languages, just incorporate them into your regex

| rex "(from|du) (?<from_time>.+?) (until|au) (?<until_time>.+"

[AL3Z]

@richgalloway @inventsekar ,

Could you pls help me in extracting the fields from below events .

2023-12-05 07:57:02,995 [CID:] [C:] [TID:PriorityScheduler Elastic Thread @ Normal] VERBOSE Thycotic.Discovery.Sources.Scanners.PowershellDiscoveryScanner - Value: xxx.com - (null)
2023-12-05 07:57:02,991 [CID:] [C:] [TID:PriorityScheduler Elastic Thread @ Normal] VERBOSE Thycotic.Discovery.Sources.Scanners.PowershellDiscoveryScanner - Name: xxx - (null)
2023-12-05 07:57:02,986 [CID:] [C:] [TID:PriorityScheduler Elastic Thread @ Normal] VERBOSE 

[richgalloway]

We've been through this exercise several times now.  Show us what you've learned.  What have you tried as a means of extracting fields from those events?  What were the results?  Have you at least identified what fields must be extracted?

[richgalloway]

Looks good to me.  Of course, change "mysourcetype" to the actual sourcetype name.

[AL3Z]

Hello @richgalloway,

• What is the difference between KV_Mode=auto and KV_Mode=none.
• Add-on builder is not supported in splunk cloud,how we can build the app in windows?

Thanks..

[richgalloway]

1. KV_MODE=auto means Splunk will automatically extract fields when it finds data in key=value format.  KV_MODE=none means Splunk disables search-time extraction of the host, source, and sourcetype fields.  This can be useful if you extract these fields yourself.
2. The add-on builder must be used in a local, non-clustered instance.  It should work on Windows, but I've not done so myself.  Apps built on a Windows platform will not pass Splunk Cloud app vetting because Windows does not set the file permissions correctly.

[AL3Z]

...

[richgalloway]

Please identify the user and domain fields in each event and I'll try to help you extract them.

[AL3Z]

@richgalloway ,

How we can exclude these cs2,Cs2label fields using the sed command in props.
Thanks.