Splunk Enterprise Security
Highlighted

JSON to CIM mapping

Hi All,

We have a scripted input, which indexes JSON data into Splunk and using SPATH we have writing our correlation rules. Now that we have Splunk ES, we would like to map JSON data to CIM in Splunk. Can anyone please guide us to understand how and is it possible to map JSON to CIM in Splunk which can be used in Splunk ES.

Thanks.

0 Karma

Re: JSON to CIM mapping

Builder

@loginsoftresearch
to begin with, you would want to understand what DataModel the JSON is most relevant too, so look into this guide for that.
https://docs.splunk.com/Documentation/CIM/4.15.0/User/Howtousethesereferencetables

Download and Install the Splunk CIM app on the SH or SHC:
https://splunkbase.splunk.com/app/1621/

Lastly, you want to go through this and follow the details which apply:
https://docs.splunk.com/Documentation/CIM/4.15.0/User/HowtouseCIM

This is something I follow:
Run the index=IndexName sourcetype=SourcetType search for the last 24 hours. I then write the evals / field alias in spl and check that the mapping is correct and returns all the fields. EG:
eval action = if(isnull(actionsuccess),if(isnotnull(actionfailure),"failure",null()),"success")
rex field=_raw Message=(?[^.]*)

which would be as such in local/props.conf
[SourceType]
EVAL-action = if(isnull(actionsuccess),if(isnotnull(actionfailure),"failure",null()),"success")
EXTRACT-signature = Message=(?[^.]*)

local/eventtypes.conf
[EventType]
search=index=IndexName sourcetype=SourcetType

tag = authentication ### note: i add the # tag reference here so its easier later on when there are many mappings

local/tags.conf
[eventtype=EventType]
authentication = enabled

add any other tags that apply

Once this is complete, you would want to restart the SH(C) and run the index=IndexName sourcetype=SourcetType search to see that the fields extracted are displayed in the panel.

Also make yourself familiar with tstats and how to search through the datamodels:
https://docs.splunk.com/Documentation/Splunk/8.0.2/SearchReference/Tstats

Note: Before jumping into data model acceleration read this doc.
https://docs.splunk.com/Documentation/Splunk/8.0.2/Knowledge/Acceleratedatamodels
Some of the things to note are: the root object mapping, storage impact, acceleration and impact on searches etc.

0 Karma
Highlighted

Re: JSON to CIM mapping

We are indexing JSON data as raw data into Splunk, can you please guide us.

0 Karma
Highlighted

Re: JSON to CIM mapping

Builder

@loginsoftresearch can you post a sample data set and what model it needs to be mapped to ?
Please eliminate all sensitive info. Also the method of ingestion.

0 Karma
Highlighted

Re: JSON to CIM mapping

We tried to map CVE field in the Vulnerabilities data model. We have created related tags but data doesn't show in the Vulnerabilities data model when we called by tag in the pivot.

Another error we get when we tried to extract "id/name" field from above JSON using Regexis as follows:
"The extraction failed. If you are extracting multiple fields, try removing one or more fields. Start with extractions that are embedded within longer text strings."

We want to convert the above JSON to the CIM model which has to be mapped to the Vulnerabilities data model.

findings_summary: [ [-]
     { [-]
       end_date: 1020-01-26
       start_date: 1019-11-27
       stats: [ [-]
         { [-]
           confidence: LOW
           event_count: 1
           first_seen: 1019-11-06
           host_count: 1
           id: CVE-2011-0117
           name: CVE-2011-0117
           severity: 7.30
           severity_category: null
         }
         { [-]
           confidence: HIGH
           event_count: 3
           first_seen: 10019-08-05
           host_count: 3
           id: CVE-2011-11581
           name: CVE-2011-11581
           severity: 9.80
           severity_category: null
         }
         { [-]
           confidence: LOW
           event_count: 1
           first_seen: 2011-11-06
           host_count: 1
           id: CVE-2014-0118
           name: CVE-2014-0118
           severity: 5.30
           severity_category: null
         }
]
0 Karma
Highlighted

Re: JSON to CIM mapping

Ultra Champion
|makeresults
| eval _raw="{\"findings_summary\": [
      {
        \"end_date\": \"1020-01-26\",
        \"start_date\": \"1019-11-27\",
        \"stats\": [
          {
            \"confidence\": \"LOW\",
            \"event_count\": \"1\",
            \"first_seen\": \"1019-11-06\",
            \"host_count\": \"1\",
            \"id\": \"CVE-2011-0117\",
            \"name\": \"CVE-2011-0117\",
            \"severity\": \"7.30\",
            \"severity_category\": \"null\"
          },
          {
            \"confidence\": \"HIGH\",
            \"event_count\": \"3\",
            \"first_seen\": \"10019-08-05\",
            \"host_count\": \"3\",
            \"id\": \"CVE-2011-11581\",
            \"name\": \"CVE-2011-11581\",
            \"severity\": \"9.80\",
            \"severity_category\": \"null\"
          },
          { 
            \"confidence\": \"LOW\",
            \"event_count\": \"1\",
            \"first_seen\": \"2011-11-06\",
            \"host_count\": \"1\",
            \"id\": \"CVE-2014-0118\",
            \"name\": \"CVE-2014-0118\",
            \"severity\": \"5.30\",
            \"severity_category\": \"null\"
          }
 ] }"
 | spath path=findings_summary{}.end_date output=end_date
 | spath path=findings_summary{}.start_date output=start_date
 | spath path="findings_summary{}.stats{}" output=json
 | stats values(*_date) as *_date by json
 | spath input=json
 | fields - json

I haven't try following conf:

props.conf

KV_MODE = json
FIELDALIAS-cim = findings_summary{}.stats{}.id AS cve

reference: https://docs.splunk.com/Documentation/CIM/latest/User/Vulnerabilities

View solution in original post

0 Karma
Highlighted

Re: JSON to CIM mapping

Thank you, I could able to extract values of id when FIELDALIAS-cim = findings_summary{}.stats{}.id AS cve to props.conf, but when clicked on individual results, I was not able to see the indexed log or is there anyway I can show results as above SPL query.

0 Karma
Highlighted

Re: JSON to CIM mapping

Ultra Champion

cim field:ok
findings_summary{}.stats{}.id field:ok
findings_summary{}.stats{}.severity field:ok
right?

0 Karma