Getting Data In

.csv file input indexed and available in splunk but the data is invisible

saikatr
Path Finder

I have a data input for .csv files on splunk, these files are created on a daily basis and named with the day's datestamp as filename, such as _20150521120829589.csv.

I can see that the files are being indexed and its turning up events on search. but when I try to view the contents of the file (say for example using "<all the required search query here> | table Record_Type Action Client_Id Sku-Id" it gives me empty tables, even though it shows the value for statistics as 444,942.

Two things stand out when i check the splunkd.log

1.Breaking event because limit of 256 has been exceeded
2.. Too many events (100K) with the same timestamp: incrementing timestamps 1 second(s) into the future to insure retrievability

I took a look at the csv file and the number of rows is above 111k. Can anyone please advise how I make the data visible (as it seems available)?

0 Karma
1 Solution

woodcock
Esteemed Legend

You have to tell Splunk if and how extract fields and it does not seem like you have done this.

You need to modify props.conf with something like this:

[YourSourcetypeHere]
INDEXED_EXTRACTIONS = CSV
HEADER_FIELD_LINE_NUMBER = 1

Read all about it here:
http://docs.splunk.com/Documentation/Splunk/latest/Data/Extractfieldsfromfileheadersatindextime

The errors are related to Splunk not being able to find timestamps. You also need to tell Splunk how to find the timestamp in each event. Read all about it here:

http://docs.splunk.com/Documentation/Splunk/latest/Data/Configuretimestamprecognition

View solution in original post

woodcock
Esteemed Legend

You have to tell Splunk if and how extract fields and it does not seem like you have done this.

You need to modify props.conf with something like this:

[YourSourcetypeHere]
INDEXED_EXTRACTIONS = CSV
HEADER_FIELD_LINE_NUMBER = 1

Read all about it here:
http://docs.splunk.com/Documentation/Splunk/latest/Data/Extractfieldsfromfileheadersatindextime

The errors are related to Splunk not being able to find timestamps. You also need to tell Splunk how to find the timestamp in each event. Read all about it here:

http://docs.splunk.com/Documentation/Splunk/latest/Data/Configuretimestamprecognition

nawazns5038
Builder

Too many events (100K) with the same timestamp: incrementing timestamps 1 second(s) into the future to insure retrievability

 

Even I am having the same error but my data does not have a timestamp in the event, so I kept it to DATETIME_CONFIG=CURRENT

How do I overcome this error ? 

 

 

0 Karma

woodcock
Esteemed Legend

Also, this configuration needs to be sent out to your forwarder and the Splunk instance restarted before it will work.

0 Karma
Get Updates on the Splunk Community!

[Puzzles] Solve, Learn, Repeat: Dynamic formatting from XML events

This challenge was first posted on Slack #puzzles channelFor a previous puzzle, I needed a set of fixed-length ...

Enter the Agentic Era with Splunk AI Assistant for SPL 1.4

  &#x1f680; Your data just got a serious AI upgrade — are you ready? Say hello to the Agentic Era with the ...

Stronger Security with Federated Search for S3, GCP SQL & Australian Threat ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...