JSON to CIM mapping

[loginsoftresear]

Hi All,

We have a scripted input, which indexes JSON data into Splunk and using SPATH we have writing our correlation rules. Now that we have Splunk ES, we would like to map JSON data to CIM in Splunk. Can anyone please guide us to understand how and is it possible to map JSON to CIM in Splunk which can be used in Splunk ES.

Thanks.

[gurulee]

Has anyone been able to map Wazuh JSON alerts that are forwarded in JSON format to raw/XML for Spunk ES Windows event stanza and Windows TA app?

[to4kawa · Accepted solution]

|makeresults
| eval _raw="{\"findings_summary\": [
      {
        \"end_date\": \"1020-01-26\",
        \"start_date\": \"1019-11-27\",
        \"stats\": [
          {
            \"confidence\": \"LOW\",
            \"event_count\": \"1\",
            \"first_seen\": \"1019-11-06\",
            \"host_count\": \"1\",
            \"id\": \"CVE-2011-0117\",
            \"name\": \"CVE-2011-0117\",
            \"severity\": \"7.30\",
            \"severity_category\": \"null\"
          },
          {
            \"confidence\": \"HIGH\",
            \"event_count\": \"3\",
            \"first_seen\": \"10019-08-05\",
            \"host_count\": \"3\",
            \"id\": \"CVE-2011-11581\",
            \"name\": \"CVE-2011-11581\",
            \"severity\": \"9.80\",
            \"severity_category\": \"null\"
          },
          { 
            \"confidence\": \"LOW\",
            \"event_count\": \"1\",
            \"first_seen\": \"2011-11-06\",
            \"host_count\": \"1\",
            \"id\": \"CVE-2014-0118\",
            \"name\": \"CVE-2014-0118\",
            \"severity\": \"5.30\",
            \"severity_category\": \"null\"
          }
 ] }"
 | spath path=findings_summary{}.end_date output=end_date
 | spath path=findings_summary{}.start_date output=start_date
 | spath path="findings_summary{}.stats{}" output=json
 | stats values(*_date) as *_date by json
 | spath input=json
 | fields - json

I haven't try following conf:

props.conf

KV_MODE = json
FIELDALIAS-cim = findings_summary{}.stats{}.id AS cve

reference: https://docs.splunk.com/Documentation/CIM/latest/User/Vulnerabilities

[loginsoftresear]

Thank you, I could able to extract values of id when FIELDALIAS-cim = findings_summary{}.stats{}.id AS cve to props.conf, but when clicked on individual results, I was not able to see the indexed log or is there anyway I can show results as above SPL query.

[to4kawa]

cim field:ok
findings_summary{}.stats{}.id field:ok
findings_summary{}.stats{}.severity field:ok
right?

[anmolpatel]

@loginsoftresearch
to begin with, you would want to understand what DataModel the JSON is most relevant too, so look into this guide for that.
https://docs.splunk.com/Documentation/CIM/4.15.0/User/Howtousethesereferencetables

Download and Install the Splunk CIM app on the SH or SHC:
https://splunkbase.splunk.com/app/1621/

Lastly, you want to go through this and follow the details which apply:
https://docs.splunk.com/Documentation/CIM/4.15.0/User/HowtouseCIM

This is something I follow:
Run the index=IndexName sourcetype=SourcetType search for the last 24 hours. I then write the evals / field alias in spl and check that the mapping is correct and returns all the fields. EG:
eval action = if(isnull(action_success),if(isnotnull(action_failure),"failure",null()),"success")
rex field=_raw Message=(?[^.]*)

which would be as such in local/props.conf
[SourceType]
EVAL-action = if(isnull(action_success),if(isnotnull(action_failure),"failure",null()),"success")
EXTRACT-signature = Message=(?[^.]*)

local/eventtypes.conf
[EventType]
search=index=IndexName sourcetype=SourcetType

tag = authentication ### note: i add the # tag reference here so its easier later on when there are many mappings

local/tags.conf
[eventtype=EventType]
authentication = enabled

add any other tags that apply

Once this is complete, you would want to restart the SH(C) and run the index=IndexName sourcetype=SourcetType search to see that the fields extracted are displayed in the panel.

Also make yourself familiar with tstats and how to search through the datamodels:
https://docs.splunk.com/Documentation/Splunk/8.0.2/SearchReference/Tstats

Note: Before jumping into data model acceleration read this doc.
https://docs.splunk.com/Documentation/Splunk/8.0.2/Knowledge/Acceleratedatamodels
Some of the things to note are: the root object mapping, storage impact, acceleration and impact on searches etc.

[loginsoftresear]

We are indexing JSON data as raw data into Splunk, can you please guide us.

[anmolpatel]

@loginsoftresearch can you post a sample data set and what model it needs to be mapped to ?
Please eliminate all sensitive info. Also the method of ingestion.

[loginsoftresear]

We tried to map CVE field in the Vulnerabilities data model. We have created related tags but data doesn't show in the Vulnerabilities data model when we called by tag in the pivot.

Another error we get when we tried to extract "id/name" field from above JSON using Regexis as follows:
"The extraction failed. If you are extracting multiple fields, try removing one or more fields. Start with extractions that are embedded within longer text strings."

We want to convert the above JSON to the CIM model which has to be mapped to the Vulnerabilities data model.

findings_summary: [ [-]
     { [-]
       end_date: 1020-01-26
       start_date: 1019-11-27
       stats: [ [-]
         { [-]
           confidence: LOW
           event_count: 1
           first_seen: 1019-11-06
           host_count: 1
           id: CVE-2011-0117
           name: CVE-2011-0117
           severity: 7.30
           severity_category: null
         }
         { [-]
           confidence: HIGH
           event_count: 3
           first_seen: 10019-08-05
           host_count: 3
           id: CVE-2011-11581
           name: CVE-2011-11581
           severity: 9.80
           severity_category: null
         }
         { [-]
           confidence: LOW
           event_count: 1
           first_seen: 2011-11-06
           host_count: 1
           id: CVE-2014-0118
           name: CVE-2014-0118
           severity: 5.30
           severity_category: null
         }
]