topic Re: How to ingest Jan month data into Splunk? in Getting Data In

How to ingest Jan month data into Splunk?

Anud — Thu, 04 May 2023 12:15:20 GMT

Hi Team,

Please suggest me to ingest the Jan month data into Splunk.
Those files are CSV files and its contains 18gb size and total 4 days data has to sent to Splunk index.
please let us know the possibilities to ingest old data.

Thanks in advance!!

Re: How to ingest Jan month data into Splunk?

gcusello — Thu, 04 May 2023 12:42:02 GMT

Hi @Anud,

I suppose that the 4 GB of data aren't in the same file!

Anyway, you have to follow the normal procedure to ingest csv files documented at https://docs.splunk.com/Documentation/Splunk/9.0.4/Data/Monitorfilesanddirectorieswithinputs.conf

There are many video (e.g. https://www.youtube.com/watch?v=3kx0OGKy_XU) that describes this process.

Ciao.

Giuseppe

Re: How to ingest Jan month data into Splunk?

Anud — Thu, 04 May 2023 13:02:26 GMT

Thanks for the quick response!! Here each day we have 18GB file size, so is it possible for normal process way to ingest. That data is from Jan month and indexed time stamp will be today date or else Jan month. how it looks, please let us know.

Re: How to ingest Jan month data into Splunk?

gcusello — Thu, 04 May 2023 13:29:14 GMT

Hi @Anud,

I'm not sure that's possible to read a 18GB csv file!

And it's also difficoult to manage a file of thst dimensions, I hint to find a different way to write tis file, e.g. applying a rotation policy.

Anyway, if possible, you can assign the timestamp based on one field in csv as usual.

If you don't have a date/time field in the csv how you define which rows must be indexed with the today's date and which one with the last month?

Ciao.

Giuseppe