import csv file and only store diffs from last import of same file (autoruns data)

[johnmccash]

I'm interested in storing csv output from the sysinternals autoruns tool in Splunk. But I will be pulling in from a rather large number of hosts, and most of the data will be the same from dump to dump. What I'm really interested in are the additions, removals, and changes. Is there any way to import the csv but have Splunk only record lines that are added, removed, or changed, from the last time the autoruns dump from a particular system was imported?

[mhorch]

This is what i use for my threat hunting.

<#
Collect AutoRuns Records
This script is used to gather autoruns data for threat hunting.

>

.\autorunsc64.exe -accepteula -a * -s -h -m -ct -nobanner > autoruns.csv

$directory = "C:\"
$x1 = Get-ChildItem -File autoruns.csv -ErrorAction SilentlyContinue | select Name, FullName

$csvExportPath = Join-Path $directory $x1.Name
$createCSV = "$csvExportPath"+".csv"
$renamedReferenceCSV = "$directory"+"ReferenceAutoRuns.csv"
$renamedDifferenceCSV = "$directory"+"DifferenceAutoRuns.csv"
$deltaFile = "$directory"+"DeltaAutoRuns.csv"

if (!(Test-Path $renamedReferenceCSV)){Rename-Item $x1.FullName -NewName "ReferenceAutoRuns.csv"
Copy-Item $renamedReferenceCSV -Destination $deltaFile

}else{

if (!(Test-Path $renamedDifferenceCSV) -and (Test-Path $renamedReferenceCSV)){Rename-Item $x1.FullName -NewName "DifferenceAutoRuns.csv"

if ((Test-Path $renamedReferenceCSV) -and (Test-Path $renamedDifferenceCSV))
{
$importReferenceCSV = Import-Csv -Delimiter "t" $renamedReferenceCSV -ErrorAction SilentlyContinue
$importDifferenceCSV = Import-Csv -Delimiter "t" $renamedDifferenceCSV -ErrorAction SilentlyContinue

Compare-Object -ReferenceObject $importReferenceCSV -DifferenceObject $importDifferenceCSV |
Select-Object -ExpandProperty InputObject|
Export-Csv $deltaFile -NoTypeInformation
Start-Sleep 10
#Clean Up Old Files
Remove-Item $renamedDifferenceCSV

}

}
}

[johnmccash]

Surely somebody out there has done this? It's a perfectly reasonable use case...
John

[johnmccash]

Just freshening this up, so maybe somebody will answer it...

[renjith_nair]

http://stackoverflow.com/questions/11108667/comparing-two-csv-files-and-getting-difference

[somesoni2]

You can write a script which will compare last two dump file (csv) from the host and log the changes. I believe there should be scripts like this already available on web.

[johnmccash]

Can you point me to one? I did look, but haven't found anything.
Thanks