Other Usage

Is there a quick way to list all fields in a data model within Splunk?


I've read about the pivot and datamodel commands. What I'm trying to do is run some sort of search in Splunk (rest perhaps) to pull out the fields defined in any loaded datamodel. I'm not trying to run a search against my data as seen through the eyes of any particular datamodel. In other words I'd like an output of something like

DataModel  Object    Fields
Web        Web       action, app, bytes, bytes_in, ...

I'm not as concerned about the exact formatting as much as the list of fields. You can run something like this but the description field is a bear to go through

| rest /servicesNS/-/-/datamodel/model | dedup title | table title description
Labels (1)


Removing the last comment of the following search will create a lookup table of all of the values. 


| datamodel 
| spath output=modelName modelName 
| search modelName!=Splunk_CIM_Validation `comment("mvexpand on the fields value for this model fails with default settings for limits.conf/[mvexpand]/ max_mem_usage_mb")`
| spath output=fields objects{}.fields{} 
| mvexpand fields 
| eval _time = now(), _raw = fields `comment("fill the values required for display in the Events viewer")`
| spath
| rename comment.* AS comment_*
| rename *{}* AS **
`comment("| fields - _time, _raw, fields, comment | outputlookup datamodel_info.csv")`


This then permits simpler lookups against each data model's field names, including the description, type, etc. 


That doesn't directly answer the initial question. That can be directly extracted with spath as commented previously.


| datamodel 
| spath output=modelName modelName 
| spath output=field_names objects{}.fields{}.displayName
| stats list(field_names) as field_names by modelName


 The field displayName includes the object names, but for CIM datamodels they can be removed relatively easily because they are the only values that include uppercase characters. 


If you prefer using the rest command, the following have the same output


| rest /servicesNS/-/-/datamodel/model 
| rename description AS _raw 
| fields _raw
| spath

| datamodel 
| spath



The datamodelsimple command is an easy way to get basic information from a datamodel, like the field name and lineage. 

| datamodelsimple datamodel="Network_Resolution" object=DNS type=attributes

 For that example,  it returns 



I know this is an old thread but I came up with this. Its shows the datamodel name, the index and sourcetype that feeds that datamodel, and what fields are in that datamodel.

| datamodel 
| rex field=_raw "\"modelName\"\s*\:\s*\"(?<modelName>[^\"]+)\"" 
| spath output=fieldList objects{}.fields{}.displayName
| table modelName fieldList
| where modelName!="Splunk_CIM_Validation"
| table modelName fieldList
| map maxsearches=40 search="tstats `summariesonly` count from datamodel=$modelName$ by sourcetype,index | eval modelName=\"$modelName$\" | eval fieldList=\"$fieldList$\""
| stats values(fieldList) as fieldList values(index) as index, values(sourcetype) as sourcetype by modelName

Splunk Employee
Splunk Employee

Finding this much later and love it. I made some slight edits to include calculated fields (the mvfilter NOT match is for the sub-model names that start with capital letters and the is_/is_not_ stuff for each sub-model):

| datamodel 
| rex field=_raw "\"modelName\"\s*\:\s*\"(?<modelName>[^\"]+)\"" 
| spath output=fieldList objects{}.calculations{}.outputFields{}.displayName
| spath output=fieldList2 objects{}.fields{}.displayName
| eval fieldList = mvappend(fieldList,fieldList2)
| where modelName!="Splunk_CIM_Validation"
| table modelName fieldList
| eval fieldList = mvdedup(mvfilter(NOT match(fieldList,"is_.*|^[A-Z]")))

 The check index/sourcetype is a handy addition. I also highly recommend Outpost's Data Model Mechanic for troubleshooting DMs.

your friendly neighborhood cyberwitch
0 Karma

Splunk Employee
Splunk Employee

You could try something like this:

| rest /servicesNS/-/-/datamodel/model 
| fields displayName, tags_whitelist, description 
| mvexpand description 
| eval desription=replace(description,"'","\"") 
| rex field=description mode=sed "s/'/\"/g" 
| spath input=description 
| fields displayName, objects{}.constraints{}.search, objects{}.fields{}.displayName
0 Karma
Get Updates on the Splunk Community!

Observability | How to Think About Instrumentation Overhead (White Paper)

Novice observability practitioners are often overly obsessed with performance. They might approach ...

Cloud Platform | Get Resiliency in the Cloud Event (Register Now!)

IDC Report: Enterprises Gain Higher Efficiency and Resiliency With Migration to Cloud  Today many enterprises ...

The Great Resilience Quest: 10th Leaderboard Update

The tenth leaderboard update (11.23-12.05) for The Great Resilience Quest is out &gt;&gt; As our brave ...