Getting Data In

How to use sourcetype with inputcsv?

ketaka
Explorer

I want to apply "sourcetype" when reading csv file by "inputcsv" command.

Is this possible by setting "props.conf"?

I want to set the header arbitrarily as below and before.

Before

AAA,AAA,AAA
System1,User1,NW1
System2,User2,NW2
System3,User3,NW3

After

Sys,Usr,NW
System1,User1,NW1
System2,User2,NW2
System3,User3,NW3

This is because duplicate field names are fixed in a system that outputs CSV.

0 Karma

DavidHourani
Super Champion

Hi @ketaka,

If you wish to apply a sourcetype to a CSV file you need to index it. Using inputcsvonly reads the csv file just as an inputlookup would. If you don't want to index your data then you can rename the fields using the rename command.

If your only solution is to apply a sourcetype you will have to index your csv file. Below some docs with options and params for indexing csv and other structured files.

This is a great read for indexing csv files of all kinds :
https://docs.splunk.com/Documentation/Splunk/7.3.0/Data/Extractfieldsfromfileswithstructureddata
All available configurations for structured data in props can be found here :
https://docs.splunk.com/Documentation/Splunk/latest/Admin/Propsconf#Structured_Data_Header_Extractio...

Let me know if that helps.

Cheers,
David

0 Karma

ketaka
Explorer

Hi David,

Thank you for your comment.

Unfortunately I don't want to make an index.
I want to load a CSV file each time and perform new tabulation.
and I want to distinguish duplicate header names as aliases.

I think this is a departure from the benefits of Splunk.
However, there is an instruction to move from Excel VBA to Splunkn, and I am looking for a way.

I thought that I could cope by loading csv file with SPL's inputcsv command each time.

I'm continuously examining the inputcsv command, inputs.conf, and props.conf.

Please be aware that terms and recognition may not be correct.

Thanks

ketaka

0 Karma

DavidHourani
Super Champion

So you want to fixup the data using VBA and then read it with Splunk using inputcsv ?

The problem I see with inputcsv is that you won't be able to apply logic using props.conf as this only applies to sourcetypes for data that has been indexed. So if you want to apply something to your data before searching that data should be indexed.

0 Karma

ketaka
Explorer

CSV files are output in a fixed form from other departments' systems.
I'm trying to replace the file processing Excel with Splunk.

I see, props.conf works only for indexed data..

It turns out that we need to think about different means.

Thank you for your reply many times.


(The following may be unnecessary content, sorry.

The summary of what you want to do is to load three types of standard CSV files, and output one CSV file after processing.
At that time, it is necessary to delete previous data and perform new processing.
I think this process is not suitable for Splunk, which indexes and accumulates data.
 If you look only at my work on the project, it may not be worthwhile to use Splunk.

0 Karma

denzelchung
Path Finder
0 Karma

ketaka
Explorer

Thank you for your quick reply

I saw the posted post.
However, it does not work as expected.

It seems that my study is not enough to understand.
The relationship between the inputcsv command and props.conf and transforms.conf can not be understood.
I will continue to investigate.

The set contents and commands are described below.

Thank you

  • input csv file path
    /Splunk/var/run/splunk/csv/inputtest.csv

  • use spl
    |inputcsv inputtest.csv

conf file settings

  • transforms.conf /Splunk/etc/system/local/transforms.conf

~~~
[inputtest]
DELIMS = ","
FIELDS = "Sys","Usr","NW"
~~~

  • props.conf /Splunk/etc/system/local/props.conf

~~~
[inputtest]
KV_MODE = none
filename = inputtest.csv
REPORT-testHeader = inputtest
~~~

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...