Getting Data In

How to add my csv file with headers

akarivaratharaj
Communicator

I have to add an input file to Splunk which is in csv format.
Example:
Server,OS,Month,Total_size,avg_size,max_size
prod_host,Linux,January,682.59,309.99,362.87
prod_host,Linux,January,682.59,309.99,362.87

I am trying to add the file through Add Data -> Upload. After selecting my input file, in the “ set Source Type” page, I am selecting source type as Structured -> csv. In the right hand side of the page, it show the headers as field names and the corresponding values under each field names. But after I finish with all the steps and start searching with the respective source and source type, my events contain only the values with comma separation.
prod_host,Linux,January,682.59,309.99,362.87,316.96

But how I need is
Server=prod_host,OS=Linux,Month=January,Total_size=682.59,avg_size=309.99,max_size=362.87

Could anyone please help me on this.

Tags (2)
0 Karma

ashajambagi
Communicator

@akarivaratharaj

Why do you want the events in that way? As all the fields are already being extracted using header.

As per my knowledge,events cant be modified.You can use rex in sed mode,but that will only replace the string values of the events but not make it to key value pair which can be searched.

A   Time
1    20th September 2018 1:32:55 PM 

this is my csv file,with header.The fields will be extracted with respect to header, which will be searchable.
Now,if i use "rex mode=sed" to get events in following format:

a=9,time=20th September 2018 1:58:00 PM

I wont be able to search values of first field using "a".

If only representation is needed,using rex can do the thing for you!

0 Karma

vishaltaneja070
Motivator

Hello @akarivaratharaj

The event will show like because the actual event is like this. But the fields are extracted properly. Just try to run the search with Smart or Verbose mode, you will be able to see fields like month, OS, Server.

So you can run search like index= OS=Linux

0 Karma

akarivaratharaj
Communicator

Hi @chrisyoungerjds & @vishaltaneja07011993 ,

I understood the way which you guys are speaking about. I have already tried in the same way. I can see my field and their values, only if I use the "|table" command.

But we would like to see the same in the events too, as like "key-value" pair.

Any idea or way to achieve the same?

0 Karma

vishaltaneja070
Motivator

Try this:
| rex mode=sed "s/(\w+),(\w+),(\w+),(\w+.\w+),(\w+.\w+),(\w+.\w+)/Server=\1, OS=\2, Month=\3, Total_size=\4, avg_size=\5,max_size=\6/g"

You can add this search at index time or even at search time extraction in props.conf

chrisyounger
SplunkTrust
SplunkTrust

Hi @akarivaratharaj

The best way to add a lookup file is Settings >Lookups >Lookup table files >Add new

Edit: Whoops sorry I misunderstood the question. If you do this search does it give you the answer you want: <search> | table *

Good luck!

0 Karma

chrisyounger
SplunkTrust
SplunkTrust

Hi,

you can do this:

<your search> | format "" "" "" "" "" ""

and it will produce what you want.

0 Karma

akarivaratharaj
Communicator

Hi,

I guess both of your answers are similar. Instead of table command you are telling format command.
Actually I am aware that if I use those command, I can get the list of my values with their corresponding fields.
So the format you are suggesting is <basic search>|table <all fields> otherwise <basic search> | format "" "" ""
I am speaking about the indexed events, why it is not showing the field-value combination in each of the indexed event.
So if I run only my basic search like index=x souretype=y source=z, this will show the respective events.
So my events should look like
1. Server=prod_host_1,OS=Linux,Month=January,Total_size=650,avg_size=309.99,max_size=362.87
2. Server=prod_host_2,OS=Linux,Month=January,Total_size=700,avg_size=300.5,max_size=450

Why the events are not showing with this kind of field-value pairs by default.

0 Karma

chrisyounger
SplunkTrust
SplunkTrust

HI.

Run your search, the do this:

  1. Click "All FIelds" in the left pane.
  2. Click "Select all within filter"
  3. Make sure that you are in "Verbose" search mode and have selected "List" view type.
  4. You should then see that you have the field pairs you want.

Good luck 🙂

0 Karma
Get Updates on the Splunk Community!

Splunk APM & RUM | Upcoming Planned Maintenance

There will be planned maintenance of the streaming infrastructure for Splunk APM and Splunk RUM in the coming ...

Part 2: Diving Deeper With AIOps

Getting the Most Out of Event Correlation and Alert Storm Detection in Splunk IT Service Intelligence   Watch ...

User Groups | Upcoming Events!

If by chance you weren't already aware, the Splunk Community is host to numerous User Groups, organized ...