topic Re: Using regex in field extraction in Splunk Search

Using regex in field extraction

TheBravoSierra — Wed, 09 Jun 2021 14:58:58 GMT

Hi,

I'm trying to create a field extraction(extension) that goes off an existing field(TargetFilename) but it isn't working. Any ideas? Tried with and without quotes.

For example:

xmlwineventlog : EXTRACT-extension

Inline

TargetFilename .*\.(?<extension>[\S\s+]*)

Works here in search:

| rex field=TargetFilename ".*\.(?<extension>[\S\s+]*)"

Re: Using regex in field extraction

alemarzu — Wed, 09 Jun 2021 15:41:17 GMT

@TheBravoSierra hello there.

I believe you are having some issues with your regex. Without knowing your events I could suggest something like this.

| rex field=TargetFilename "\.(?<file_extension>[^\.]+$)"

Re: Using regex in field extraction

TheBravoSierra — Wed, 09 Jun 2021 15:44:29 GMT

I apologize, the issue is not with the rex command but with adding it to Settings >> Fields >> Field Extractions.

I want it to be automatically extracted so this rex command is not needed in the search bar to populate the field.

Does that help?

Re: Using regex in field extraction

kamlesh_vaghela — Wed, 09 Jun 2021 17:32:14 GMT

@TheBravoSierra

Can you please try this?

props.conf

[My_source_type] REPORT-extension = extract_extension . . .

transforms.conf

[extract_extension] REGEX =<Data Name='TargetFilename'>([^.]+)(?<extension>.\w+)<\/Data> FORMAT = extension:$3

Note: I have tried with below sample events.

TargetFilename=a.txt TargetFilename=b.txt TargetFilename="c.txt"

Please share sample events in case this is not working .

Thanks
KV
▄︻̷̿┻̿═━一

If any of my reply helps you to solve the problem Or gain knowledge, an upvote would be appreciated.

Re: Using regex in field extraction

TheBravoSierra — Wed, 09 Jun 2021 16:34:19 GMT

<Event xmlns='omitted><System><Provider Name='Microsoft-Windows-Sysmon' Guid='{omitted}'/><EventID>2</EventID><Version>4</Version><Level>4</Level><Task>2</Task><Opcode>0</Opcode><Keywords>omitted</Keywords><TimeCreated SystemTime='2021-06-09T16:31:46.813927400Z'/><EventRecordID>947063</EventRecordID><Correlation/><Execution ProcessID='4824' ThreadID='6932'/><Channel>Microsoft-Windows-Sysmon/Operational</Channel><Computer>omitted</Computer><Security UserID='S-1-5-18'/></System><EventData><Data Name='RuleName'></Data><Data Name='UtcTime'>2021-06-09 16:31:46.813</Data><Data Name='ProcessGuid'>{omitted}</Data><Data Name='ProcessId'>11932</Data><Data Name='Image'>C:\Users\omitted\AppData\Local\Microsoft\Teams\current\Teams.exe</Data><Data Name='TargetFilename'>C:\Users\omitted\AppData\Roaming\Microsoft\Windows\Recent\CustomDestinations\EJ5T0WEDS801S4OF2UEY.temp</Data><Data Name='CreationUtcTime'>2020-04-21 21:00:25.187</Data><Data Name='PreviousCreationUtcTime'>2021-06-09 16:31:46.802</Data></EventData></Event>

So this one for example would extract .temp from the filename EJ5T0WEDS801S4OF2UEY.temp
Thanks a bunch for the help.

Re: Using regex in field extraction

richgalloway — Wed, 09 Jun 2021 17:14:50 GMT

Are you sure TargetFilename is an extracted field? Do you get results from this query?

index=foo | table TargetFilename

If not then the field is not extracted and you should try a different regex.

Share a sample event and we may be able to help more.

Re: Using regex in field extraction

kamlesh_vaghela — Wed, 09 Jun 2021 17:31:52 GMT

@TheBravoSierra

Can you please check my updated answer?

Re: Using regex in field extraction

TheBravoSierra — Wed, 09 Jun 2021 18:02:29 GMT

Yes that worked, thank you!

Could you also help me figure out why these won't work?

[extract_file]
REGEX =<Data Name='TargetFilename'>.*\\\\(?<file>[\S\s+]*)<\/Data>
FORMAT = file:$3

[extract_md5]
REGEX =<Data Name='Hashes'>MD5=(?<MD5>[a-fA-F0-9]{1,32}),SHA256=(?<SHA256>[a-zA-Z0-9]{33,64})<\/Data>
FORMAT = md5:$3

[extract_sha256]
REGEX =<Data Name='Hashes'>SHA256=(?<SHA256>[a-zA-Z0-9]{33,64})<\/Data>
FORMAT = sha256:$3

TargetFilename data can use event previously provided.
Example data with Hashes:

<Event xmlns='http://schemas.microsoft.com/win/2004/08/events/event'><System><Provider Name='Microsoft-Windows-Sysmon' Guid='{}'/><EventID>1</EventID><Version>5</Version><Level>4</Level><Task>1</Task><Opcode>0</Opcode><Keywords></Keywords><TimeCreated SystemTime='2021-06-09T17:56:51.779403700Z'/><EventRecordID></EventRecordID><Correlation/></Data><Data Name='TerminalSessionId'>0</Data><Data Name='IntegrityLevel'>System</Data><Data Name='Hashes'>MD5=E17F9289D944626831D63E4394C195B3,SHA256=5F403F6BD0F55162B8E98786D1FB3475C0D414AB18128C270119E18AA53E4CDE</Data><Data Name='ParentProcessGuid'>{}</Data><Data Name='ParentProcessId'></Data><Data Name='ParentImage'>C:\Program Files\Confer\RepMgr.exe</Data><Data Name='ParentCommandLine'>"C:\Program Files\Confer\RepMgr.exe"</Data></EventData></Event>

Re: Using regex in field extraction

kamlesh_vaghela — Thu, 10 Jun 2021 06:43:13 GMT

@TheBravoSierra

Can you please try this?

[extract_file] REGEX =<Data Name='TargetFilename'>([^<]*) FORMAT = file::$1 [extract_md5] REGEX =<Data Name='Hashes'>MD5=([^,]+) FORMAT = md5::$1 [extract_sha256] REGEX =SHA256=([^<]+) FORMAT = sha256::$1

Thanks
KV
▄︻̷̿┻̿═━一

If any of my reply helps you to solve the problem Or gain knowledge, an upvote would be appreciated.