Splunk Enterprise Security

JSON to CIM mapping

loginsoftresear
Explorer

Hi All,

We have a scripted input, which indexes JSON data into Splunk and using SPATH we have writing our correlation rules. Now that we have Splunk ES, we would like to map JSON data to CIM in Splunk. Can anyone please guide us to understand how and is it possible to map JSON to CIM in Splunk which can be used in Splunk ES.

Thanks.

1 Solution

to4kawa
Ultra Champion
|makeresults
| eval _raw="{\"findings_summary\": [
      {
        \"end_date\": \"1020-01-26\",
        \"start_date\": \"1019-11-27\",
        \"stats\": [
          {
            \"confidence\": \"LOW\",
            \"event_count\": \"1\",
            \"first_seen\": \"1019-11-06\",
            \"host_count\": \"1\",
            \"id\": \"CVE-2011-0117\",
            \"name\": \"CVE-2011-0117\",
            \"severity\": \"7.30\",
            \"severity_category\": \"null\"
          },
          {
            \"confidence\": \"HIGH\",
            \"event_count\": \"3\",
            \"first_seen\": \"10019-08-05\",
            \"host_count\": \"3\",
            \"id\": \"CVE-2011-11581\",
            \"name\": \"CVE-2011-11581\",
            \"severity\": \"9.80\",
            \"severity_category\": \"null\"
          },
          { 
            \"confidence\": \"LOW\",
            \"event_count\": \"1\",
            \"first_seen\": \"2011-11-06\",
            \"host_count\": \"1\",
            \"id\": \"CVE-2014-0118\",
            \"name\": \"CVE-2014-0118\",
            \"severity\": \"5.30\",
            \"severity_category\": \"null\"
          }
 ] }"
 | spath path=findings_summary{}.end_date output=end_date
 | spath path=findings_summary{}.start_date output=start_date
 | spath path="findings_summary{}.stats{}" output=json
 | stats values(*_date) as *_date by json
 | spath input=json
 | fields - json

I haven't try following conf:

props.conf

KV_MODE = json
FIELDALIAS-cim = findings_summary{}.stats{}.id AS cve

reference: https://docs.splunk.com/Documentation/CIM/latest/User/Vulnerabilities

View solution in original post

gurulee
Explorer

Has anyone been able to map Wazuh JSON alerts that are forwarded in JSON format to raw/XML for Spunk ES Windows event stanza and Windows TA app?

0 Karma

to4kawa
Ultra Champion
|makeresults
| eval _raw="{\"findings_summary\": [
      {
        \"end_date\": \"1020-01-26\",
        \"start_date\": \"1019-11-27\",
        \"stats\": [
          {
            \"confidence\": \"LOW\",
            \"event_count\": \"1\",
            \"first_seen\": \"1019-11-06\",
            \"host_count\": \"1\",
            \"id\": \"CVE-2011-0117\",
            \"name\": \"CVE-2011-0117\",
            \"severity\": \"7.30\",
            \"severity_category\": \"null\"
          },
          {
            \"confidence\": \"HIGH\",
            \"event_count\": \"3\",
            \"first_seen\": \"10019-08-05\",
            \"host_count\": \"3\",
            \"id\": \"CVE-2011-11581\",
            \"name\": \"CVE-2011-11581\",
            \"severity\": \"9.80\",
            \"severity_category\": \"null\"
          },
          { 
            \"confidence\": \"LOW\",
            \"event_count\": \"1\",
            \"first_seen\": \"2011-11-06\",
            \"host_count\": \"1\",
            \"id\": \"CVE-2014-0118\",
            \"name\": \"CVE-2014-0118\",
            \"severity\": \"5.30\",
            \"severity_category\": \"null\"
          }
 ] }"
 | spath path=findings_summary{}.end_date output=end_date
 | spath path=findings_summary{}.start_date output=start_date
 | spath path="findings_summary{}.stats{}" output=json
 | stats values(*_date) as *_date by json
 | spath input=json
 | fields - json

I haven't try following conf:

props.conf

KV_MODE = json
FIELDALIAS-cim = findings_summary{}.stats{}.id AS cve

reference: https://docs.splunk.com/Documentation/CIM/latest/User/Vulnerabilities

loginsoftresear
Explorer

Thank you, I could able to extract values of id when FIELDALIAS-cim = findings_summary{}.stats{}.id AS cve to props.conf, but when clicked on individual results, I was not able to see the indexed log or is there anyway I can show results as above SPL query.

0 Karma

to4kawa
Ultra Champion

cim field:ok
findings_summary{}.stats{}.id field:ok
findings_summary{}.stats{}.severity field:ok
right?

0 Karma

anmolpatel
Builder

@loginsoftresearch
to begin with, you would want to understand what DataModel the JSON is most relevant too, so look into this guide for that.
https://docs.splunk.com/Documentation/CIM/4.15.0/User/Howtousethesereferencetables

Download and Install the Splunk CIM app on the SH or SHC:
https://splunkbase.splunk.com/app/1621/

Lastly, you want to go through this and follow the details which apply:
https://docs.splunk.com/Documentation/CIM/4.15.0/User/HowtouseCIM

This is something I follow:
Run the index=IndexName sourcetype=SourcetType search for the last 24 hours. I then write the evals / field alias in spl and check that the mapping is correct and returns all the fields. EG:
eval action = if(isnull(action_success),if(isnotnull(action_failure),"failure",null()),"success")
rex field=_raw Message=(?[^.]*)

which would be as such in local/props.conf
[SourceType]
EVAL-action = if(isnull(action_success),if(isnotnull(action_failure),"failure",null()),"success")
EXTRACT-signature = Message=(?[^.]*)

local/eventtypes.conf
[EventType]
search=index=IndexName sourcetype=SourcetType

tag = authentication ### note: i add the # tag reference here so its easier later on when there are many mappings

local/tags.conf
[eventtype=EventType]
authentication = enabled

add any other tags that apply

Once this is complete, you would want to restart the SH(C) and run the index=IndexName sourcetype=SourcetType search to see that the fields extracted are displayed in the panel.

Also make yourself familiar with tstats and how to search through the datamodels:
https://docs.splunk.com/Documentation/Splunk/8.0.2/SearchReference/Tstats

Note: Before jumping into data model acceleration read this doc.
https://docs.splunk.com/Documentation/Splunk/8.0.2/Knowledge/Acceleratedatamodels
Some of the things to note are: the root object mapping, storage impact, acceleration and impact on searches etc.

0 Karma

loginsoftresear
Explorer

We are indexing JSON data as raw data into Splunk, can you please guide us.

0 Karma

anmolpatel
Builder

@loginsoftresearch can you post a sample data set and what model it needs to be mapped to ?
Please eliminate all sensitive info. Also the method of ingestion.

0 Karma

loginsoftresear
Explorer

We tried to map CVE field in the Vulnerabilities data model. We have created related tags but data doesn't show in the Vulnerabilities data model when we called by tag in the pivot.

Another error we get when we tried to extract "id/name" field from above JSON using Regexis as follows:
"The extraction failed. If you are extracting multiple fields, try removing one or more fields. Start with extractions that are embedded within longer text strings."

We want to convert the above JSON to the CIM model which has to be mapped to the Vulnerabilities data model.

findings_summary: [ [-]
     { [-]
       end_date: 1020-01-26
       start_date: 1019-11-27
       stats: [ [-]
         { [-]
           confidence: LOW
           event_count: 1
           first_seen: 1019-11-06
           host_count: 1
           id: CVE-2011-0117
           name: CVE-2011-0117
           severity: 7.30
           severity_category: null
         }
         { [-]
           confidence: HIGH
           event_count: 3
           first_seen: 10019-08-05
           host_count: 3
           id: CVE-2011-11581
           name: CVE-2011-11581
           severity: 9.80
           severity_category: null
         }
         { [-]
           confidence: LOW
           event_count: 1
           first_seen: 2011-11-06
           host_count: 1
           id: CVE-2014-0118
           name: CVE-2014-0118
           severity: 5.30
           severity_category: null
         }
]
0 Karma
Get Updates on the Splunk Community!

New in Observability - Improvements to Custom Metrics SLOs, Log Observer Connect & ...

The latest enhancements to the Splunk observability portfolio deliver improved SLO management accuracy, better ...

Improve Data Pipelines Using Splunk Data Management

  Register Now   This Tech Talk will explore the pipeline management offerings Edge Processor and Ingest ...

3-2-1 Go! How Fast Can You Debug Microservices with Observability Cloud?

Register Join this Tech Talk to learn how unique features like Service Centric Views, Tag Spotlight, and ...