topic Is there a way to use a wildcard in field names when running a search? (ex: field*_name=X for field1_name, field2_name, etc) in Splunk Search

Is there a way to use a wildcard in field names when running a search? (ex: field*_name=X for field1_name, field2_name, etc)

splunknewbie05 — Sat, 11 Apr 2015 20:24:07 GMT

I have csv data indexed in Splunk. The fields are unique, but have some patterns:

As an example, the following first row are fields and values in the following rows. I'm making up the content here for illustration purposes.

field1_name   field2_name   field3_name   field4_name ............fieldn_name                   
book          pen           paper         pencil
laptop        gold          pen           pencil
pen           paper         pencil        fan

Is there a way to add a wildcard into the fields' names during a search as you notice there is a pattern in the field names.

I want to search for "Give me all the events containing "pen" of all the fields"

The search would be some thing like this:

sourcetype=csv   index=myindex | search  field*_name="pen"

But it appears splunk only allows to have wildcard for the field values and not the field names? Is there a way to add a wildcard into the field names??

I don't want to use "_raw" event searching.

Re: Is there a way to use a wildcard in field names when running a search? (ex: field*_name=X for field1_name, field2_name, etc)

chimell — Sat, 11 Apr 2015 21:06:38 GMT

Hi splunknewbie05
Use this search code

 index="myindex"    sourcetype="csv"   "pen" | table *

Re: Is there a way to use a wildcard in field names when running a search? (ex: field*_name=X for field1_name, field2_name, etc)

stephanefotso — Mon, 28 Sep 2020 19:30:39 GMT

Hello!
Since you need events , not a statistic table, i 'm going to propose a regular expression instead of the fieldsummary command wich is very interesting..
I don't know how are your events, but i think this will help :

sourcetype=csv index=myindex |rex "\s(?P<fields_Name>[^=]+)=pen"

How does it work?
The rex command will match all pen values of field1_name field2_name field3_name field4_name .... ..... .........................fieldn_name and will create a new field called fields_Name which values are only fields having pen as a value.
So you will have only events with pen as the value of , field1_name field2_name field3_name field4_name .... ..... .........................fieldn_name

If you have indexed the tutorialdata.zip,Take this as a template and see how it works:

source="tutorialdata.zip:*"|rex "\s(?P<field_Name>[^=]+)=7026"|stats count by field_Name

Re: Is there a way to use a wildcard in field names when running a search? (ex: field*_name=X for field1_name, field2_name, etc)

splunknewbie05 — Mon, 28 Sep 2020 19:35:18 GMT

I didn't get the answer I'm looking for. The above answers still has wild card on the content. I'm looking for wild card on field names

You can organize data in couple of following ways.

Lets say I have State and percentage fields

Table1:

State percentage
A1 10
B1 20
A3 10
A4 5
C1 6

Lets say I want to compute total of all A states percentage.
You can do a wild card on the content some thing like "A*" and extract the percentages and compute total

Lets organize the same table in other way
State_A1_Percentage State_B1_Percentage State_ A3_Percentage State_A4_Percentage State_C1_Percentage
10 20 10 5 6

Now if I want to compute all A states percentage, I can extract State_A1_Percentage, State_ A3_Percentage, State_A4_Percentage and compute total.

What I'm asking is if there is a way to extract values using wild card on field "State_A*_Percentage"

Re: Is there a way to use a wildcard in field names when running a search? (ex: field*_name=X for field1_name, field2_name, etc)

jeffland — Wed, 15 Apr 2015 05:42:36 GMT

If a regex is not a solution for you, then I dont know if what you're looking for exists.

I'm not sure this is exactly the answer either, but you may want to have a look at the possibility to transform your data into key-value pairs. If idieally your lines after the first line contain a timestamp, you can let splunk read the data as

(Event1)
[timestamp] field1_name = book field2_name = pen ...
(Event2)
[timestamp] field1_name = laptop field2_name = gold ...

With this data, you could easily identify all your field names (and work with them, as I don't think just identifying them is what you are looking to do in the end).
If this looks helpful, have a look at what REPORT in props.conf (and the associated part in transforms.conf) does. Right now I can't pin down where in the docs you find that unfortunately, but if what I described above is moving in the right direction for you, I'll look it up further.
Update: here are the docs http://docs.splunk.com/Documentation/Splunk/6.2.2/Knowledge/Createandmaintainsearch-timefieldextractionsthroughconfigurationfiles

Re: Is there a way to use a wildcard in field names when running a search? (ex: field*_name=X for field1_name, field2_name, etc)

ramdaspr — Wed, 15 Apr 2015 06:23:33 GMT

Try foreach if you are looking at fields (columns)

Re: Is there a way to use a wildcard in field names when running a search? (ex: field*_name=X for field1_name, field2_name, etc)

fdi01 — Wed, 15 Apr 2015 07:41:34 GMT

first call all your fields by same field name to do this use rename command.
after you can filter .

try like this:

sourcetype=csv index=myindex|rename field*_name as pen|where  pen="pen"

Re: Is there a way to use a wildcard in field names when running a search? (ex: field*_name=X for field1_name, field2_name, etc)

stephanefotso — Wed, 15 Apr 2015 09:13:05 GMT

Here is my new solution also with regular expressions. Since you are using a csv file with headers, here is what you can do with i think.

sourcetype=csv index=myindex |rex "(?<fields_Name>pen)"|stats count by fields_Name

Now let suppose that you have a csv like this

state percentage
A1        10
B1        20
A3        10
A2        4
C1        1
B2        12
C2        15

Here is the search to write

    source="wildcard.csv"|rex "(?<fields_Name>[^\d])"|stats sum(percentage) by fields_Name

Wich is giving the table bellow:

       count
A        24
B        32
C        15

Thanks!

Re: Is there a way to use a wildcard in field names when running a search? (ex: field*_name=X for field1_name, field2_name, etc)

btt — Wed, 15 Apr 2015 12:04:47 GMT

hi,
For your question "Give me all the events containing "pen" of all the fields"
try this

 .....|foreach field* [eval newfield=<<FIELD>>]| where like(newfield, "pen%")

Re: Is there a way to use a wildcard in field names when running a search? (ex: field*_name=X for field1_name, field2_name, etc)

thisissplunk — Thu, 07 Jan 2016 22:51:16 GMT

I've seen at least two other questions for this when one portion of the field name will be unknown, but I've never found an answer. You haven't found one by chance have you?

My current question is at: https://answers.splunk.com/answers/340010/how-to-search-over-a-field-when-its-json-and-has-m-1.html?minQuestionBodyLength=80