Using outputlookup and inputlookup in the same search, why am I getting "The lookup table 'Results.csv' is invalid"?

[SrinivasaC]

Hi ,

Below is my search:

< base-search > | outputlookup Results.csv | search inputlookup Results.csv | xyseries col1, col2, col3

I'm writing my result into a lookup file results.csv. Results contains 3,60,00 records. It's taking time to write data into the lookup file, so when I use inputlookup file immediately after outputlookup, it's giving error as "The lookup table 'Results.csv' is invalid".

When I check in the lookup definitions tab, I can see the Results.csv file.

How do I avoid a delay in writing data into a CSV file so that my .csv file will available for the rest of the search?
Or do we have any options other than using input/outputlookup files?.

Thanks in advance..

[rharrisssi]

Are you using Windows? Windows has issues with file locking.

[richgalloway]

Why are you writing to Results.csv? What is the benefit?

[SrinivasaC]

I'm retrieving 2/3/4/5 weeks of data into .csv files and performing operations/calculations on it, to improve performance.

I tried to get data directly from index and its taking lot of time and sometimes its got struck.

Here my requirement is to get data for last 5 weeks of data on selected date, ex: user selects 29-sep, I have to retrieve data for 29-sep, 22-Sep, 15-sep,8-sep and 1-sep dates and pass on to splunk dashboard.
if user can select 2, 3 , 4 & 5 weeks. So here I'm storing each week's data into lookup files(i.e. .csv files) and doing some calculation on it and storing again data into summarize index. This operations we are doing on schedule basis not realtime. user can only select last 6 days with 5 weeks data.

[richgalloway]

Have you considered putting the last 5 weeks of data into a summary index once per day/week? That will improve performance by having the data ready instead of searching each time a user selects a date.

[SrinivasaC]

Yes.. its summarized index only..

[richgalloway]

I meant write to a summary index instead of the CSV file. If that's not feasible, HiroshiSatoh's suggestion to use a subsearch should work.

[SrinivasaC]

The index I'm using is summarized index only it contains 5 weeks data.
I tried HiroshiSatoh's suggestion also, it didn't help...

[HiroshiSatoh]

This is only to keyword search.

| Search inputlookup Results.csv | ⇒Search ”inputlookup” AND "Results.csv"

Sub search If you want to do something is required.

ex.)
| outputlookup Results.csv | append [|inputlookup Results.csv] | xyseries col1, col2, col3
| outputlookup Results.csv | search [|inputlookup Results.csv|table x] | xyseries col1, col2, col3

[HiroshiSatoh]

The search is to work without any problems. Can you put a search statement that does not work?
⇒< base-search > | outputlookup Results.csv | append [|inputlookup Results.csv]

Or, Do not mistake the place to be passed to the search statement from the dashboard?