I have csv data indexed in Splunk. The fields are unique, but have some patterns:
As an example, the following first row are fields and values in the following rows. I'm making up the content here for illustration purposes.
field1_name field2_name field3_name field4_name ............fieldn_name book pen paper pencil laptop gold pen pencil pen paper pencil fan
Is there a way to add a wildcard into the fields' names during a search as you notice there is a pattern in the field names.
I want to search for "Give me all the events containing "pen" of all the fields"
The search would be some thing like this:
sourcetype=csv index=myindex | search field*_name="pen"
But it appears splunk only allows to have wildcard for the field values and not the field names? Is there a way to add a wildcard into the field names??
I don't want to use "_raw" event searching.
I've seen at least two other questions for this when one portion of the field name will be unknown, but I've never found an answer. You haven't found one by chance have you?
My current question is at: https://answers.splunk.com/answers/340010/how-to-search-over-a-field-when-its-json-and-has-m-1.html?...
Here is my new solution also with regular expressions. Since you are using a csv file with headers, here is what you can do with i think.
sourcetype=csv index=myindex |rex "(?<fields_Name>pen)"|stats count by fields_Name
Now let suppose that you have a csv like this
state percentage A1 10 B1 20 A3 10 A2 4 C1 1 B2 12 C2 15
Here is the search to write
source="wildcard.csv"|rex "(?<fields_Name>[^\d])"|stats sum(percentage) by fields_Name
Wich is giving the table bellow:
count A 24 B 32 C 15
If a regex is not a solution for you, then I dont know if what you're looking for exists.
I'm not sure this is exactly the answer either, but you may want to have a look at the possibility to transform your data into key-value pairs. If idieally your lines after the first line contain a timestamp, you can let splunk read the data as
(Event1) [timestamp] field1_name = book field2_name = pen ... (Event2) [timestamp] field1_name = laptop field2_name = gold ...
With this data, you could easily identify all your field names (and work with them, as I don't think just identifying them is what you are looking to do in the end).
If this looks helpful, have a look at what
REPORT in props.conf (and the associated part in transforms.conf) does. Right now I can't pin down where in the docs you find that unfortunately, but if what I described above is moving in the right direction for you, I'll look it up further.
Update: here are the docs http://docs.splunk.com/Documentation/Splunk/6.2.2/Knowledge/Createandmaintainsearch-timefieldextract...
I didn't get the answer I'm looking for. The above answers still has wild card on the content. I'm looking for wild card on field names
You can organize data in couple of following ways.
Lets say I have State and percentage fields
Lets say I want to compute total of all A states percentage.
You can do a wild card on the content some thing like "A*" and extract the percentages and compute total
Lets organize the same table in other way
State_A1_Percentage State_B1_Percentage State_ A3_Percentage State_A4_Percentage State_C1_Percentage
10 20 10 5 6
Now if I want to compute all A states percentage, I can extract State_A1_Percentage, State_ A3_Percentage, State_A4_Percentage and compute total.
What I'm asking is if there is a way to extract values using wild card on field "State_A*_Percentage"
Since you need events , not a statistic table, i 'm going to propose a regular expression instead of the fieldsummary command wich is very interesting..
I don't know how are your events, but i think this will help :
sourcetype=csv index=myindex |rex "\s(?P<fields_Name>[^=]+)=pen"
How does it work?
The rex command will match all pen values of field1_name field2_name field3_name field4_name .... ..... .........................fieldn_name and will create a new field called fields_Name which values are only fields having pen as a value.
So you will have only events with pen as the value of , field1_name field2_name field3_name field4_name .... ..... .........................fieldn_name
If you have indexed the tutorialdata.zip,Take this as a template and see how it works:
source="tutorialdata.zip:*"|rex "\s(?P<field_Name>[^=]+)=7026"|stats count by field_Name