Archive

Can't get all rows from CSV

New Member

I monitor folder on one server with SplunkUniversalForwarder installed. the configuration of input.conf as below:
[monitor://E:\A530\archive\Powershell_User]

  sourcetype = Powershell_User

  crcSalt = <SOURCE>

  disabled = false

  followTail = 0

then I do search in search head with Splunk query:
source="E:\A530\archive\PowershellUser\20180614getaduserallforSplunk.csv"

the total rows of this CSV file are 7168, but only return 5139 rows.

the CSV is well formatted

How can I get all rows in Splunk query?

Thanks

Tags (1)
0 Karma

Esteemed Legend

You are pretty much out of luck. You can get the date from your filename but the events must have a timestamp in them and yours do not. See here:

https://www.splunk.com/blog/2009/12/02/configure-splunk-to-pull-a-date-out-of-a-non-standard-filenam...

You can use DATETIME_CONFIG = CURRENT to timestamp with the index time.

0 Karma

Esteemed Legend

Are you ABSOLUTELY sure that it is well-formatted? Splunk will abort the import (possibly without indicating an error) if it hits something that is wonky. The best way to check is using the Lookup Editor App (https://splunkbase.splunk.com/app/1724/). You simply upload the lookup file and it checks for you and you will see the error straight away because it will complain. It will even show you what kind of error and on what line!

0 Karma

SplunkTrust
SplunkTrust

Do you have a timestamp column in csv and what's the timerange you are using in search?

0 Karma

New Member

Hi nair,

I am fresh to use SPLUNK, I use all time range in search.

How can i find timestamp information?

thanks.

0 Karma

SplunkTrust
SplunkTrust

If you are selecting "All Time" timestamp does not really matter. Are you uploading it to the same index or do you have some props to split it? also try index=* source="E:\\A530\\archive\\Powershell_User\\20180614_get_aduser_all_for_Splunk.csv" Is it possible to identify one of the rows which is missing and find the difference of that from other rows?

0 Karma

New Member

Hi Nair,
I know why only part of rows are returned. Splunk index the CSV file with timestamp but there is no dedicated filed for that.
Splunk extract the timestamp of CSV from modified time. the problem is some rows can added the timestamp with modified time.

do you know how to extract timestamp based on file name? we have 20180614 within file name.

0 Karma