Using regex in field extraction

[TheBravoSierra]

Hi,

I'm trying to create a field extraction(extension) that goes off an existing field(TargetFilename) but it isn't working. Any ideas? Tried with and without quotes. 

For example: 

xmlwineventlog : EXTRACT-extension

Inline

TargetFilename .*\.(?<extension>[\S\s+]*)

Works here in search: 

| rex field=TargetFilename ".*\.(?<extension>[\S\s+]*)"

[richgalloway]

Are you sure TargetFilename is an extracted field?  Do you get results from this query?

index=foo | table TargetFilename

If not then the field is not extracted and you should try a different regex.

Share a sample event and we may be able to help more.

[kamlesh_vaghela · Accepted solution]

@TheBravoSierra 

Can you please try this?

props.conf

[My_source_type]
REPORT-extension = extract_extension
.
.
.

transforms.conf

[extract_extension]
REGEX =<Data Name='TargetFilename'>([^.]+)(?<extension>.\w+)<\/Data>
FORMAT = extension:$3

Note: I have tried with below sample events.

TargetFilename=a.txt
TargetFilename=b.txt
TargetFilename="c.txt"

Please share sample events in case this is  not working .

Thanks
KV
▄︻̷̿┻̿═━一

If any of my reply helps you to solve the problem Or gain knowledge, an upvote would be appreciated.

[TheBravoSierra]

<Event xmlns='omitted><System><Provider Name='Microsoft-Windows-Sysmon' Guid='{omitted}'/><EventID>2</EventID><Version>4</Version><Level>4</Level><Task>2</Task><Opcode>0</Opcode><Keywords>omitted</Keywords><TimeCreated SystemTime='2021-06-09T16:31:46.813927400Z'/><EventRecordID>947063</EventRecordID><Correlation/><Execution ProcessID='4824' ThreadID='6932'/><Channel>Microsoft-Windows-Sysmon/Operational</Channel><Computer>omitted</Computer><Security UserID='S-1-5-18'/></System><EventData><Data Name='RuleName'></Data><Data Name='UtcTime'>2021-06-09 16:31:46.813</Data><Data Name='ProcessGuid'>{omitted}</Data><Data Name='ProcessId'>11932</Data><Data Name='Image'>C:\Users\omitted\AppData\Local\Microsoft\Teams\current\Teams.exe</Data><Data Name='TargetFilename'>C:\Users\omitted\AppData\Roaming\Microsoft\Windows\Recent\CustomDestinations\EJ5T0WEDS801S4OF2UEY.temp</Data><Data Name='CreationUtcTime'>2020-04-21 21:00:25.187</Data><Data Name='PreviousCreationUtcTime'>2021-06-09 16:31:46.802</Data></EventData></Event>

So this one for example would extract .temp from the filename EJ5T0WEDS801S4OF2UEY.temp
Thanks a bunch for the help.

[kamlesh_vaghela]

@TheBravoSierra 

Can you please check my updated answer?

KV

[TheBravoSierra]

Yes that worked, thank you!

Could you also help me figure out why these won't work?

[extract_file]
REGEX =<Data Name='TargetFilename'>.*\\\\(?<file>[\S\s+]*)<\/Data>
FORMAT = file:$3

[extract_md5]
REGEX =<Data Name='Hashes'>MD5=(?<MD5>[a-fA-F0-9]{1,32}),SHA256=(?<SHA256>[a-zA-Z0-9]{33,64})<\/Data>
FORMAT = md5:$3

[extract_sha256]
REGEX =<Data Name='Hashes'>SHA256=(?<SHA256>[a-zA-Z0-9]{33,64})<\/Data>
FORMAT = sha256:$3

TargetFilename data can use event previously provided. 
Example data with Hashes:

<Event xmlns='http://schemas.microsoft.com/win/2004/08/events/event'><System><Provider Name='Microsoft-Windows-Sysmon' Guid='{}'/><EventID>1</EventID><Version>5</Version><Level>4</Level><Task>1</Task><Opcode>0</Opcode><Keywords></Keywords><TimeCreated SystemTime='2021-06-09T17:56:51.779403700Z'/><EventRecordID></EventRecordID><Correlation/></Data><Data Name='TerminalSessionId'>0</Data><Data Name='IntegrityLevel'>System</Data><Data Name='Hashes'>MD5=E17F9289D944626831D63E4394C195B3,SHA256=5F403F6BD0F55162B8E98786D1FB3475C0D414AB18128C270119E18AA53E4CDE</Data><Data Name='ParentProcessGuid'>{}</Data><Data Name='ParentProcessId'></Data><Data Name='ParentImage'>C:\Program Files\Confer\RepMgr.exe</Data><Data Name='ParentCommandLine'>"C:\Program Files\Confer\RepMgr.exe"</Data></EventData></Event>

[kamlesh_vaghela]

@TheBravoSierra 

Can you please try this?

[extract_file]
REGEX =<Data Name='TargetFilename'>([^<]*)
FORMAT = file::$1

[extract_md5]
REGEX =<Data Name='Hashes'>MD5=([^,]+)
FORMAT = md5::$1

[extract_sha256]
REGEX =SHA256=([^<]+)
FORMAT = sha256::$1

Thanks
KV
▄︻̷̿┻̿═━一

If any of my reply helps you to solve the problem Or gain knowledge, an upvote would be appreciated.

[alemarzu]

@TheBravoSierra hello there.

I believe you are having some issues with your regex. Without knowing your events I could suggest something like this. 

| rex field=TargetFilename "\.(?<file_extension>[^\.]+$)"

[TheBravoSierra]

I apologize, the issue is not with the rex command but with adding it to Settings >> Fields >> Field Extractions. 

I want it to be automatically extracted so this rex command is not needed in the search bar to populate the field.

Does that help?