How to extract fields in the data model?

[Daksesh]

Hi, 

I'm new to the regex, can someone please help me in regex to extract file name and file path separately in the data model. 

Field value is variable in the fields file name and file path. Thank you.

Below is the sample data.

"evidence": [{"entityType": "File", "evidenceCreation Time": "2022-12-19T10:43:56.51Z", "sha1": "336466254f9fe9b5a09f27848317525481dd5dd6", "sha256": "59de220b8d7961086e8d2d1fde61b71a810a32f78a9175f1f87ecacd692b85c9", "fileName": "Nero-8.1.1.0b_fra_trial.exe", "filePath": "F:\\Desktop new backup\\Musique \\Nero 8", "processId": null, "process CommandLine": null, "processCreation Time": null, "parentProcessId":

[richgalloway]

What do you mean by "in the data model"?  Which data model?  

It's a simple matter to use rex to extract the file name and path fields from the raw event since each has a distinct eye-catcher.  All you need is a regex that looks for '"fileName": "' or '"filePath": "' and takes everything until the next quotation mark as the field value.  Those regex strings would be

"fileName": "(?<fileName>[^"]+)
"filePath": "(?<filePath>[^"]+)

Use them in a search by escaping the quotation marks (yes 3 backslashes are needed).

| rex "\\\"fileName\\\": \\\"(?<fileName>[^\\\"]+)"
| rex "\\\"filePath\\\": \\\"(?<filePath>[^\\\"]+)"

[Daksesh]

Hi Rich,

Thank you for answer.

When I run the regex for both file name and file path in the search extraction is happening. But extraction is not happening in the Endpoint data model. 

Can you please help me if any changes are required in the regex for Endpoint datamodel.

[PickleRick]

The | rex command obviously works only in the time of your specific search containing that command. So the fields will not be extracted otherwise.

In this case however it seems that your event is a json structure so maybe it would be better to use automatic json KV extraction. Now the only part would be to get the data to conform to the CIM-defined schema by defining proper aliases and/or calculated fields for your particular sourcetype.

In other words - you don't modify the CIM-supplied datamodels because they are the standard that other apps follow but you apply modifications on your own sourcetype to make it compliant with CIM-datamodel

[Daksesh]

Hi, 

Thank you so much for your clear explanation and support.

[richgalloway]

The trick to getting fields extracted by a data model is to use the CIM name for the fields, in this case file_name and file_path.  Also, the fields must be extracted automatically rather than in a search.  Add EXTRACT or FIELDALIAS settings to the appropriate props.conf file.

[mysourcetype]
EXTRACT-file_name = "fileName": "(?<file_name>[^"]+)
EXTRACT-file_path = "filePath": "(?<file_path>[^"]+)

[Daksesh]

Hi,

Thank you for your reply.

I would like to have double check that, below regex to be included in the add on(Splunk_TA_MS_Security) or in the CIM addon.

The logs are being onboarding by using the add on Splunk_TA_MS_Security, so I bit confused where exactly regex requires either in Splunk_TA_MS_Security or Splunk Common Information Model addon.

Could you please support for the above query.

Thank you.

[richgalloway · Accepted solution]

The regexes can go in any app because Splunk will combine the settings for a given sourcetype from all apps, subject to sharing level.  It may make the most sense to put them in a custom app or in Splunk_TA_MS_Security, but not in the CIM add-on.

[Daksesh]

Hi, 

Thank you so much for your clear explanation and support.

[richgalloway]

If your problem is resolved, then please click the "Accept as Solution" button to help future readers.