Reporting

Is there a quick way to list all fields in a data model within Splunk?

Runals
Motivator

I've read about the pivot and datamodel commands. What I'm trying to do is run some sort of search in Splunk (rest perhaps) to pull out the fields defined in any loaded datamodel. I'm not trying to run a search against my data as seen through the eyes of any particular datamodel. In other words I'd like an output of something like

DataModel  Object    Fields
Web        Web       action, app, bytes, bytes_in, ...

I'm not as concerned about the exact formatting as much as the list of fields. You can run something like this but the description field is a bear to go through

| rest /servicesNS/-/-/datamodel/model | dedup title | table title description
Labels (1)

malvidin
Communicator

Removing the last comment of the following search will create a lookup table of all of the values. 

 

| datamodel 
| spath output=modelName modelName 
| search modelName!=Splunk_CIM_Validation `comment("mvexpand on the fields value for this model fails with default settings for limits.conf/[mvexpand]/ max_mem_usage_mb")`
| spath output=fields objects{}.fields{} 
| mvexpand fields 
| eval _time = now(), _raw = fields `comment("fill the values required for display in the Events viewer")`
| spath
| rename comment.* AS comment_*
| rename *{}* AS **
`comment("| fields - _time, _raw, fields, comment | outputlookup datamodel_info.csv")`

 

This then permits simpler lookups against each data model's field names, including the description, type, etc. 

 

That doesn't directly answer the initial question. That can be directly extracted with spath as commented previously.

 

| datamodel 
| spath output=modelName modelName 
| spath output=field_names objects{}.fields{}.displayName
| stats list(field_names) as field_names by modelName

 

 The field displayName includes the object names, but for CIM datamodels they can be removed relatively easily because they are the only values that include uppercase characters. 

 

If you prefer using the rest command, the following have the same output

 

| rest /servicesNS/-/-/datamodel/model 
| rename description AS _raw 
| fields _raw
| spath

| datamodel 
| spath

 

 

The datamodelsimple command is an easy way to get basic information from a datamodel, like the field name and lineage. 

| datamodelsimple datamodel="Network_Resolution" object=DNS type=attributes

 For that example,  it returns 

lineageattribute
DNS.dest_portdest_port
hosthost
DNS.answer_countanswer_count
_raw_raw
0 Karma

rrlong89
Loves-to-Learn

I know this is an old thread but I came up with this. Its shows the datamodel name, the index and sourcetype that feeds that datamodel, and what fields are in that datamodel.

| datamodel 
| rex field=_raw "\"modelName\"\s*\:\s*\"(?<modelName>[^\"]+)\"" 
| spath output=fieldList objects{}.fields{}.displayName
| table modelName fieldList
| where modelName!="Splunk_CIM_Validation"
| table modelName fieldList
| map maxsearches=40 search="tstats `summariesonly` count from datamodel=$modelName$ by sourcetype,index | eval modelName=\"$modelName$\" | eval fieldList=\"$fieldList$\""
| stats values(fieldList) as fieldList values(index) as index, values(sourcetype) as sourcetype by modelName
0 Karma

mipeters_splunk
Splunk Employee
Splunk Employee

You could try something like this:

| rest /servicesNS/-/-/datamodel/model 
| fields displayName, tags_whitelist, description 
| mvexpand description 
| eval desription=replace(description,"'","\"") 
| rex field=description mode=sed "s/'/\"/g" 
| spath input=description 
| fields displayName, objects{}.constraints{}.search, objects{}.fields{}.displayName
0 Karma
.conf21 CFS Extended through 5/20!

Don't miss your chance
to share your Splunk
wisdom in-person or
virtually at .conf21!

Call for Speakers has
been extended through
Thursday, 5/20!