Other Usage

Is there a quick way to list all fields in a data model within Splunk?

Runals
Motivator

I've read about the pivot and datamodel commands. What I'm trying to do is run some sort of search in Splunk (rest perhaps) to pull out the fields defined in any loaded datamodel. I'm not trying to run a search against my data as seen through the eyes of any particular datamodel. In other words I'd like an output of something like

DataModel  Object    Fields
Web        Web       action, app, bytes, bytes_in, ...

I'm not as concerned about the exact formatting as much as the list of fields. You can run something like this but the description field is a bear to go through

| rest /servicesNS/-/-/datamodel/model | dedup title | table title description
Labels (1)

malvidin
Communicator

Removing the last comment of the following search will create a lookup table of all of the values. 

 

| datamodel 
| spath output=modelName modelName 
| search modelName!=Splunk_CIM_Validation `comment("mvexpand on the fields value for this model fails with default settings for limits.conf/[mvexpand]/ max_mem_usage_mb")`
| spath output=fields objects{}.fields{} 
| mvexpand fields 
| eval _time = now(), _raw = fields `comment("fill the values required for display in the Events viewer")`
| spath
| rename comment.* AS comment_*
| rename *{}* AS **
`comment("| fields - _time, _raw, fields, comment | outputlookup datamodel_info.csv")`

 

This then permits simpler lookups against each data model's field names, including the description, type, etc. 

 

That doesn't directly answer the initial question. That can be directly extracted with spath as commented previously.

 

| datamodel 
| spath output=modelName modelName 
| spath output=field_names objects{}.fields{}.displayName
| stats list(field_names) as field_names by modelName

 

 The field displayName includes the object names, but for CIM datamodels they can be removed relatively easily because they are the only values that include uppercase characters. 

 

If you prefer using the rest command, the following have the same output

 

| rest /servicesNS/-/-/datamodel/model 
| rename description AS _raw 
| fields _raw
| spath

| datamodel 
| spath

 

 

The datamodelsimple command is an easy way to get basic information from a datamodel, like the field name and lineage. 

| datamodelsimple datamodel="Network_Resolution" object=DNS type=attributes

 For that example,  it returns 

lineageattribute
DNS.dest_portdest_port
hosthost
DNS.answer_countanswer_count
_raw_raw

rrlong89
Engager

I know this is an old thread but I came up with this. Its shows the datamodel name, the index and sourcetype that feeds that datamodel, and what fields are in that datamodel.

| datamodel 
| rex field=_raw "\"modelName\"\s*\:\s*\"(?<modelName>[^\"]+)\"" 
| spath output=fieldList objects{}.fields{}.displayName
| table modelName fieldList
| where modelName!="Splunk_CIM_Validation"
| table modelName fieldList
| map maxsearches=40 search="tstats `summariesonly` count from datamodel=$modelName$ by sourcetype,index | eval modelName=\"$modelName$\" | eval fieldList=\"$fieldList$\""
| stats values(fieldList) as fieldList values(index) as index, values(sourcetype) as sourcetype by modelName

7thdrxn
Splunk Employee
Splunk Employee

Finding this much later and love it. I made some slight edits to include calculated fields (the mvfilter NOT match is for the sub-model names that start with capital letters and the is_/is_not_ stuff for each sub-model):

| datamodel 
| rex field=_raw "\"modelName\"\s*\:\s*\"(?<modelName>[^\"]+)\"" 
| spath output=fieldList objects{}.calculations{}.outputFields{}.displayName
| spath output=fieldList2 objects{}.fields{}.displayName
| eval fieldList = mvappend(fieldList,fieldList2)
| where modelName!="Splunk_CIM_Validation"
| table modelName fieldList
| eval fieldList = mvdedup(mvfilter(NOT match(fieldList,"is_.*|^[A-Z]")))

 The check index/sourcetype is a handy addition. I also highly recommend Outpost's Data Model Mechanic for troubleshooting DMs.

☆.。.:・°☆.。.:・°☆.。.:・°☆.。.:・°☆
your friendly neighborhood cyberwitch
0 Karma

mipeters_splunk
Splunk Employee
Splunk Employee

You could try something like this:

| rest /servicesNS/-/-/datamodel/model 
| fields displayName, tags_whitelist, description 
| mvexpand description 
| eval desription=replace(description,"'","\"") 
| rex field=description mode=sed "s/'/\"/g" 
| spath input=description 
| fields displayName, objects{}.constraints{}.search, objects{}.fields{}.displayName
0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...