Does outputlookup append or overwrite?

[hulahoop]

Does the outputlookup command overwrite or append to the existing specified lookup file? The documentation does not clarify: http://www.splunk.com/base/Documentation/latest/SearchReference/Outputlookup.

[inventsekar]

i was searching for the similar issue and after reading this solution i assumed the current Splunk version also works similar to this solution. Then i was reading the docs and learnt the new options available, so i thought to update other readers who may face similar situation like myself.

Until Splunk 6.5, the scheduled reports never had "Write to a CSV lookup file" option. Ref:

https://docs.splunk.com/Documentation/Splunk/6.5.0/Report/Schedulereports

From Splunk 6.6, the scheduled reports are having a "Write to a CSV lookup file" option. Ref: 

https://docs.splunk.com/Documentation/Splunk/6.6.0/Report/Schedulereports#Define_a_Write_to_a_CSV_Lo...

so, if we are using Splunk 6.6 or newer versions, then, simply with scheduled reports, we can "append" the lookup files. thanks. 

(PS - i have given around 500+ karma points so far, received badge for that, if an answer helped you, a karma point would be nice!. we all should start "Learn, Give Back, Have Fun")

[spunk311z]

in my use cases it is better (safer) to export the results as csv (by hand, via the splunk results gui, top right download button) and then use the AWESOME Lookup editor splunk app to manually copy / paste the data i want to append. (i open the csv in excel to copy fields)

granted this is not automated, but it keeps me from making errors or accidentally overwriting prior data in the lookup.

app: https://splunkbase.splunk.com/app/1724/

[gkanapathy · Accepted solution]

It will overwrite. If you want to append, you should first do an ... | inputlookup append=true myoldfile, and then probably some kind of dedup depending on the specifics of the lookup, then the outputlookup myoldfile, e.g.,

stats count by host,hostip | fields - count | inputlookup append=true hostiplookup | dedup host | outputlookup hostiplookup

[inventsekar]

Are you guys sure of this method with tstats? 

| tstats count where (index=test* earliest="2/1/2020:00:00:00" latest="3/1/2020:00:00:00") BY _time span=1d 
| inputlookup append=true testlookup.cvs 
| outputlookup testlookup.csv

[bhawkins1]

In splunk 6.x the above did not work until I change | inputlookup x to append [| inputlookup x]. To clarify, this is useful for cases where you want to append data to the csv file without making duplicate "keys". Without the extra dedup, splunk will basically just open the file in append mode ( 'a' ) or write mode ( 'w' ).

[dadi]

I'm not sure if you are aware of this issue(splunk 5), but when I've outputlookup with append=true, I wasn't managed to write more than 1198 new records. This solution make the append=true to be unnecessary and work around this bug. Thanks!

[gkanapathy]

technically i guess this prepends, not appends, but that's more probably what you want anyway, especially if you're constructing a time-based lookup.

[jwestberg]

It will overwrite.

Because of this, the |outputlookup command is well suited to being used in scheduled saved searches, keeping a lookup table up to date with each run.