How to write such an SPL search statement
two quetions:
1、on the Splunk Enterpirse ,how to remove duplicate data and count it . as the picture shows
2、I have some keywords in the csv file. Each column has a field.
I extracted a "uri" field in the splunk and I would like to compare and CSV files.
If the value of uri matches a value in the first column of the csv file. Get the field name "success" for csv. The If you match a value in the second column of the csv file, get the second field name "fild" .. and so on
First you have to convert your matrix CSV into a Splunk lookup, like this:
| makeresults
| eval raw="complete,false,NULL login,offline,empty finished,faild,nothing online,no,zero ok,not,nil"
| makemv raw
| mvexpand raw
| rename raw AS _raw
| rex "^(?<success>[^,]+),(?<faild>[^,]+),(?<none>.*)$"
| table faild none success
| rename COMMENT AS "Everything above would be replaced by '|inputlookup <YourExistingLookupNameHere>' for you"
| transpose
| eval input=","
| rename "row *" AS row* column AS output
| foreach row* [eval input = input . "*" . <<FIELD>> . ","]
| fields - row*
| makemv delim="," input
| mvexpand input
| table input output
| outputlookup MyNewWildcardLookup
You need to make sure that you specify match_type = WILDCARD(input)
for MyNewWildcardLookup
in transforms.conf
.
Then you use it like this:
... | lookup MyNewWildcardLookup input AS uri
This will give each event that matches anything a value for output
.
1) ... | stats count by field1 field2
2) ... | inputlookup fileName.csv uriFieldNameInData
1) Add this to search - ` | stats count by field1 field2
2) You can avoid lookups and achieve this using case statement
| eval status=case(uri="complete" OR uri="login" OR uri="finished" OR uri="online" OR url="ok","success",uri="false" OR uri="offline" OR uri="faild" OR uri="no" OR url="not","faild",isnull(uri) OR uri="empty" OR uri="nothing" OR uri="zero" OR url="nil","faild")
No, I have a lot of keywords, not just these five.
If this is not possible, I can divide each column by multiple CSVs and create multiple alerts. For example, "success" this column as a CSV. If the contents of the match and match, the alert
Reorganize the lookup file this way to have 2 fields - uri and status
uri,status
complete,success
login,success
finished,success
online,success
ok,success
false,faild
offline,faild
faild,faild
no,faild
not,faild
NULL,none
empty,none
nothing,none
zero,none
nil,none
Then do the mapping
<your search> | lookup lookupname.csv uri as uri OUTPUT status
Hi
To remove duplicate data : use
............ |dedup field1 field2