Splunk Search

Data models: how to use a wildcard for field names?

helge
Builder

Some of my events have an unknown number of field names with a common naming scheme. In my searches I use a wildcard to include them all, e.g.

stats first(GroupPolicyCSE*) by host

How can I transfer this to data model attributes?

1 Solution

aneels_splunk
Splunk Employee
Splunk Employee

Data models cannot be dynamically added to data models in Splunk 6.0. The data model feature assumes that you know the structure of your data when building the model.

If you have a list of all the fields that can possibly occur, you could create a data model with all of them, and simply allow there to be zero matching fields for those attributes on some of your installs.

You could also look into building a search-based object that creates fields with consistent names, and then building a data model on top of that. Search-based objects can incorporate arbitrary splunk searches, which gives you a lot of power. The downside is they can't be accelerated.

View solution in original post

aneels_splunk
Splunk Employee
Splunk Employee

Data models cannot be dynamically added to data models in Splunk 6.0. The data model feature assumes that you know the structure of your data when building the model.

If you have a list of all the fields that can possibly occur, you could create a data model with all of them, and simply allow there to be zero matching fields for those attributes on some of your installs.

You could also look into building a search-based object that creates fields with consistent names, and then building a data model on top of that. Search-based objects can incorporate arbitrary splunk searches, which gives you a lot of power. The downside is they can't be accelerated.

jtrucks
Splunk Employee
Splunk Employee

If these fields are all defined in your Splunk instance where the data model is being created, it should be able to auto-discover what those are based on your field definitions.

To do this, when editing your Object in this Data Model, go to "Add Attribute" -> "Auto-Extracted" and then save.

--
Jesse Trucks
Minister of Magic

helge
Builder

I understand what you are saying, but that does not help. The field names depend on what is installed in the customer's network and there is no finite list of field names that can occur.
I guess it boils down to the fact that data models just do not work with wildcarded field names (yet). In other words, data models do not expose the full functionality of Splunk.
Thanks very much for your time!

0 Karma

jtrucks
Splunk Employee
Splunk Employee

Part of the problem in your situation is that data models don't appear to be easily built in a dynamically updated way. http://docs.splunk.com/Documentation/Splunk/6.0/Knowledge/Designdatamodelobjects#Add_an_auto-extract... indicates it will present new fields it discovers anytime you run the Auto-Extract function. Do this via API on a scheduled basis, and you can keep your data model updated regularly. See http://docs.splunk.com/Documentation/Splunk/6.0/RESTAPI/RESTknowledge#datamodel.2Fmodel.2F.7Bname.7D for docs.

--
Jesse Trucks
Minister of Magic

jtrucks
Splunk Employee
Splunk Employee

I suggest you put your search extract config files into deployment server to uniformly pull the same fields.

If you are not accelerating this data model, you could build auto-extract and update the models with scripting (REST or API are likely approaches).

OR go to each Splunk instance to gather up the various extracts for the relevant source(s) and sourcetype(s) and merge them all, then generate your data models.

The last approach would be to manually set up each one based on the collectively known extracts using the Eval or Regex options to Add Attribute.

--
Jesse Trucks
Minister of Magic
0 Karma

helge
Builder

As I wrote, the number of fields is unknown. My Splunk instance my have 5, while another instance may have 10 different fields matching the wildcard.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...