topic Re: To extract fields in the data model in Splunk Enterprise

How to extract fields in the data model?

Daksesh — Wed, 21 Dec 2022 15:03:02 GMT

Hi,

I'm new to the regex, can someone please help me in regex to extract file name and file path separately in the data model.

Field value is variable in the fields file name and file path. Thank you.

Below is the sample data.

"evidence": [{"entityType": "File", "evidenceCreation Time": "2022-12-19T10:43:56.51Z", "sha1": "336466254f9fe9b5a09f27848317525481dd5dd6", "sha256": "59de220b8d7961086e8d2d1fde61b71a810a32f78a9175f1f87ecacd692b85c9", "fileName": "Nero-8.1.1.0b_fra_trial.exe", "filePath": "F:\\Desktop new backup\\Musique \\Nero 8", "processId": null, "process CommandLine": null, "processCreation Time": null, "parentProcessId":

Re: To extract fields in the data model

richgalloway — Wed, 21 Dec 2022 15:02:45 GMT

What do you mean by "in the data model"? Which data model?

It's a simple matter to use rex to extract the file name and path fields from the raw event since each has a distinct eye-catcher. All you need is a regex that looks for '"fileName": "' or '"filePath": "' and takes everything until the next quotation mark as the field value. Those regex strings would be

"fileName": "(?<fileName>[^"]+) "filePath": "(?<filePath>[^"]+)

Use them in a search by escaping the quotation marks (yes 3 backslashes are needed).

| rex "\\\"fileName\\\": \\\"(?<fileName>[^\\\"]+)" | rex "\\\"filePath\\\": \\\"(?<filePath>[^\\\"]+)"

Re: To extract fields in the data model

Daksesh — Thu, 22 Dec 2022 15:13:56 GMT

Hi Rich,

Thank you for answer.

When I run the regex for both file name and file path in the search extraction is happening. But extraction is not happening in the Endpoint data model.

Can you please help me if any changes are required in the regex for Endpoint datamodel.

Re: To extract fields in the data model

richgalloway — Thu, 22 Dec 2022 15:42:39 GMT

The trick to getting fields extracted by a data model is to use the CIM name for the fields, in this case file_name and file_path. Also, the fields must be extracted automatically rather than in a search. Add EXTRACT or FIELDALIAS settings to the appropriate props.conf file.

[mysourcetype] EXTRACT-file_name = "fileName": "(?<file_name>[^"]+) EXTRACT-file_path = "filePath": "(?<file_path>[^"]+)

Re: To extract fields in the data model

Daksesh — Mon, 26 Dec 2022 08:12:20 GMT

Hi,

Thank you for your reply.

I would like to have double check that, below regex to be included in the add on(Splunk_TA_MS_Security) or in the CIM addon.

The logs are being onboarding by using the add on Splunk_TA_MS_Security, so I bit confused where exactly regex requires either in Splunk_TA_MS_Security or Splunk Common Information Model addon.

Could you please support for the above query.

Thank you.

Re: To extract fields in the data model

richgalloway — Mon, 26 Dec 2022 17:01:28 GMT

The regexes can go in any app because Splunk will combine the settings for a given sourcetype from all apps, subject to sharing level. It may make the most sense to put them in a custom app or in Splunk_TA_MS_Security, but not in the CIM add-on.

Re: To extract fields in the data model

PickleRick — Tue, 27 Dec 2022 10:34:55 GMT

The | rex command obviously works only in the time of your specific search containing that command. So the fields will not be extracted otherwise.

In this case however it seems that your event is a json structure so maybe it would be better to use automatic json KV extraction. Now the only part would be to get the data to conform to the CIM-defined schema by defining proper aliases and/or calculated fields for your particular sourcetype.

In other words - you don't modify the CIM-supplied datamodels because they are the standard that other apps follow but you apply modifications on your own sourcetype to make it compliant with CIM-datamodel

Re: To extract fields in the data model

Daksesh — Wed, 28 Dec 2022 11:25:18 GMT

Hi,

Thank you so much for your clear explanation and support.

Re: To extract fields in the data model

Daksesh — Wed, 28 Dec 2022 11:26:58 GMT

Hi,

Thank you so much for your clear explanation and support.

Re: To extract fields in the data model

richgalloway — Wed, 28 Dec 2022 14:38:08 GMT

If your problem is resolved, then please click the "Accept as Solution" button to help future readers.