Splunk Search

Is there a way to use a wildcard in field names when running a search? (ex: field*_name=X for field1_name, field2_name, etc)

splunknewbie05
Explorer

I have csv data indexed in Splunk. The fields are unique, but have some patterns:

As an example, the following first row are fields and values in the following rows. I'm making up the content here for illustration purposes.

field1_name   field2_name   field3_name   field4_name ............fieldn_name                   
book          pen           paper         pencil
laptop        gold          pen           pencil
pen           paper         pencil        fan          

Is there a way to add a wildcard into the fields' names during a search as you notice there is a pattern in the field names.

I want to search for "Give me all the events containing "pen" of all the fields"

The search would be some thing like this:

sourcetype=csv   index=myindex | search  field*_name="pen"

But it appears splunk only allows to have wildcard for the field values and not the field names? Is there a way to add a wildcard into the field names??

I don't want to use "_raw" event searching.

Tags (4)

thisissplunk
Builder

I've seen at least two other questions for this when one portion of the field name will be unknown, but I've never found an answer. You haven't found one by chance have you?

My current question is at: https://answers.splunk.com/answers/340010/how-to-search-over-a-field-when-its-json-and-has-m-1.html?...

0 Karma

btt
Path Finder

hi,
For your question "Give me all the events containing "pen" of all the fields"
try this

 .....|foreach field* [eval newfield=<<FIELD>>]| where like(newfield, "pen%")
0 Karma

stephanefotso
Motivator

Here is my new solution also with regular expressions. Since you are using a csv file with headers, here is what you can do with i think.

sourcetype=csv index=myindex |rex "(?<fields_Name>pen)"|stats count by fields_Name

Now let suppose that you have a csv like this

state percentage
A1        10
B1        20
A3        10
A2        4
C1        1
B2        12
C2        15

Here is the search to write

    source="wildcard.csv"|rex "(?<fields_Name>[^\d])"|stats sum(percentage) by fields_Name

Wich is giving the table bellow:

       count
A        24
B        32
C        15

Thanks!

SGF
0 Karma

fdi01
Motivator

first call all your fields by same field name to do this use rename command.
after you can filter .

try like this:

sourcetype=csv index=myindex|rename field*_name as pen|where  pen="pen"
0 Karma

jeffland
SplunkTrust
SplunkTrust

If a regex is not a solution for you, then I dont know if what you're looking for exists.

I'm not sure this is exactly the answer either, but you may want to have a look at the possibility to transform your data into key-value pairs. If idieally your lines after the first line contain a timestamp, you can let splunk read the data as

(Event1)
[timestamp] field1_name = book field2_name = pen ...
(Event2)
[timestamp] field1_name = laptop field2_name = gold ...

With this data, you could easily identify all your field names (and work with them, as I don't think just identifying them is what you are looking to do in the end).
If this looks helpful, have a look at what REPORT in props.conf (and the associated part in transforms.conf) does. Right now I can't pin down where in the docs you find that unfortunately, but if what I described above is moving in the right direction for you, I'll look it up further.
Update: here are the docs http://docs.splunk.com/Documentation/Splunk/6.2.2/Knowledge/Createandmaintainsearch-timefieldextract...

0 Karma

splunknewbie05
Explorer

I didn't get the answer I'm looking for. The above answers still has wild card on the content. I'm looking for wild card on field names

You can organize data in couple of following ways.

Lets say I have State and percentage fields

Table1:

State percentage
A1 10
B1 20
A3 10
A4 5
C1 6

Lets say I want to compute total of all A states percentage.
You can do a wild card on the content some thing like "A*" and extract the percentages and compute total

Lets organize the same table in other way
State_A1_Percentage State_B1_Percentage State_ A3_Percentage State_A4_Percentage State_C1_Percentage
10 20 10 5 6

Now if I want to compute all A states percentage, I can extract State_A1_Percentage, State_ A3_Percentage, State_A4_Percentage and compute total.

What I'm asking is if there is a way to extract values using wild card on field "State_A*_Percentage"

0 Karma

ramdaspr
Contributor

Try foreach if you are looking at fields (columns)

0 Karma

stephanefotso
Motivator

Hello!
Since you need events , not a statistic table, i 'm going to propose a regular expression instead of the fieldsummary command wich is very interesting..
I don't know how are your events, but i think this will help :

sourcetype=csv index=myindex |rex "\s(?P<fields_Name>[^=]+)=pen"

How does it work?
The rex command will match all pen values of field1_name field2_name field3_name field4_name .... ..... .........................fieldn_name and will create a new field called fields_Name which values are only fields having pen as a value.
So you will have only events with pen as the value of , field1_name field2_name field3_name field4_name .... ..... .........................fieldn_name

If you have indexed the tutorialdata.zip,Take this as a template and see how it works:

source="tutorialdata.zip:*"|rex "\s(?P<field_Name>[^=]+)=7026"|stats count by field_Name
SGF
0 Karma

chimell
Motivator

Hi splunknewbie05
Use this search code

 index="myindex"    sourcetype="csv"   "pen" | table *
0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...