topic Field Extraction Using Tranforms Configration in Security

Field Extraction Using Tranforms Configration

SplunkDash — Sun, 19 Nov 2023 03:11:06 GMT

Hello,

I have some issues to perform field extractions using transform configuration. It's not giving field value pairs as expected. Sample events and configuration files are given below. Some non-uniformities within the events are also marked in Bold. Any recommendations will be highly appreciated. Thank you so much.

My Configuration Files

[mypropfConf]

REPORT-mytranforms=myTransConf

[myTransConf]
REGEX = ([^"]+?):'([^"]+?)'
FORMAT = $1::$2

Sample Events

2023-11-15T18:56:29.098Z OTESTN097MA4515620 TESTuser20248: UserID: '90A', UserType: 'TempEMP', System: 'TEST', UAT: 'UTA-True', EventType: 'TEST', EventID: 'Lookup', Subject: 'A5367817222', Scode: '' EventStatus: 0, TimeStamp: '2023-11-03T15:56:29.099Z', Device: 'OTESTN097MA4513020', Msg: 'lookup ok', var: 'Sec'

2023-11-15T18:56:29.021Z OTESTN097MB7513020 TESTuser20249: UserID: '95B', UserType: 'TempEMP', System: 'TEST', UAT: 'UTA-True', EventType: 'TEST', EventID: 'Lookup', Subject: 'A516670222', Scode: '' EventStatus: 0, TimeStamp: '2023-11-03T15:56:29.099Z', Device: 'OTESTN097MA4513020', Msg: 'lookup ok', var: 'tec'

2023-11-15T18:56:29.009Z OTESTN097MB9513020 TESTuser20248: UserID: '95A', UserType: 'TempEMP', System: 'TEST', UAT: 'UTA-True', EventType: 'TEST', EventID: 'Lookup', Subject: 'A546610222', Scode: '' EventStatus: 0, TimeStamp: '2023-11-03T15:56:29.099Z', Device: 'OTESTN097MA4513020', Msg: 'lookup ok', var: 'test'

Re: Field Extraction Using Tranforms Configration

gcusello — Sun, 19 Nov 2023 06:59:01 GMT

Hi @SplunkDash,

you can use REPORT i you have a list of fields separated by comma or another char.

In your case I'd use a regex in props.conf like the following

EXTRACT-your_sourcetype = ^(?<timestamp>\d+-\d+-\d+T\d+:\d+:\d+\.\d+\w)\s+(?<host>[^ ]+)\s+(?<user>[^:]+):\s+UserID:\s+\'(?<UserID>[^\']+)\',\s+UserType:\s+\'(?<UserType>[^\']+)\',\s+System:\s+\'(?<System>[^\']+)\',\s+UAT:\s+\'(?<UAT>[^\']+)\',\s+EventType:\s+\'(?<EventType>[^\']+)\',\s+EventID:\s+\'(?<EventID>[^\']+)\',\s+Subject:\s+\'(?<Subject>[^\']+)\',\s+Scode:\s+\'(?<Scode>[^\']*)\'\s+EventStatus:\s+(?<EventStatus>\d*),\s+TimeStamp:\s*\'(?<TimeStamp>[^\']*)\',\s+Device:\s*\'(?<Device>[^\']*)\',\s+Msg:\s*\'(?<Msg>[^\']*)\',\s+var:\s*\'(?<var>[^\']*)\'

You can test the regex at https://regex101.com/r/iQZi9K/1

Ciao.

Guseppe

Re: Field Extraction Using Tranforms Configration

PickleRick — Sun, 19 Nov 2023 07:56:44 GMT

I'm not sure if there was any modification to the copy-pasted config and/or events but your regex doesn't allow for spaces between the semicolon after the key name and the value.