.csv file taking more time for indexing

[pragycho]

Hi All ,
I am facing one issue for indexing.

I have .csv file from external resource and this .csv file size is 11236KB.

also configured data (access log) in data input.

want to generate report for AD Group details.

In .csv file and data(accesslog) , one field (user_id) is common so when we trying to generate report so .csv file is taking more time indexing and getting error fail to reopen lookup (.csv ) file.

Can you please help me on this ?

[Richfez]

There appears to be some confusion over indexes and lookups. Let's step back a bit -

A lookup, which is a csv file in the lookups directory, can be used like | inputlookup <lookupname> to just "read the whole lookup in", or can be used as a lookup my search here | lookup <lookupname> <search fields> OUPUT <new fields> to augment existing data by lookup up a key value in your lookup and returning other data out of that matching lookup row into that event.

Indexed data is instead data ingested in an input. You access that with the regular search command (and the base search, the first one which doesn't need the word search in it.).

With that in mind, could you please describe again what it is you are doing, what's taking so long, and provide the actual whole search of your search? (The search you posted below starts in the middle with a 'dedup'. That's not how a search can start, so it must be a bad copy/paste or something.)

And please paste in searches as "code" using the little code button above.

[richgalloway]

What is your search?

[pragycho]

-

dedup user_id | sort department,user_id | where bytes_in >0 |stats values("user_id") as User,values("dest_domain") as Application,values("bytes_in") as Bandwidth_used by department| rename department AS "AD Group"

[richgalloway]

Where are you accessing the CSV file?

[pragycho]

i am accessing field name (user_id , department) from .csv file and .csv file is available in lookup folder.

[richgalloway]

I see no lookup or inputlookup commands in your query. How are you getting fields from the lookup file?