Getting Data In

.csv file taking more time for indexing

New Member

Hi All ,
I am facing one issue for indexing.

I have .csv file from external resource and this .csv file size is 11236KB.

also configured data (access log) in data input.

want to generate report for AD Group details.

In .csv file and data(accesslog) , one field (user_id) is common so when we trying to generate report so .csv file is taking more time indexing and getting error fail to reopen lookup (.csv ) file.

Can you please help me on this ?

Labels (1)
0 Karma

SplunkTrust
SplunkTrust

There appears to be some confusion over indexes and lookups. Let's step back a bit -

A lookup, which is a csv file in the lookups directory, can be used like | inputlookup <lookupname> to just "read the whole lookup in", or can be used as a lookup my search here | lookup <lookupname> <search fields> OUPUT <new fields> to augment existing data by lookup up a key value in your lookup and returning other data out of that matching lookup row into that event.

Indexed data is instead data ingested in an input. You access that with the regular search command (and the base search, the first one which doesn't need the word search in it.).

With that in mind, could you please describe again what it is you are doing, what's taking so long, and provide the actual whole search of your search? (The search you posted below starts in the middle with a 'dedup'. That's not how a search can start, so it must be a bad copy/paste or something.)

And please paste in searches as "code" using the little code button above.

0 Karma

SplunkTrust
SplunkTrust

What is your search?

---
If this reply helps you, an upvote would be appreciated.
0 Karma

New Member

-

dedup user_id | sort department,user_id | where bytes_in >0 |stats values("user_id") as User,values("dest_domain") as Application,values("bytes_in") as Bandwidth_used by department| rename department AS "AD Group"

0 Karma

SplunkTrust
SplunkTrust

Where are you accessing the CSV file?

---
If this reply helps you, an upvote would be appreciated.
0 Karma

New Member

i am accessing field name (user_id , department) from .csv file and .csv file is available in lookup folder.

0 Karma

SplunkTrust
SplunkTrust

I see no lookup or inputlookup commands in your query. How are you getting fields from the lookup file?

---
If this reply helps you, an upvote would be appreciated.
0 Karma