topic Re: adding a field based on other fields in Splunk Search

adding a field based on other fields

wsw70 — Mon, 28 Sep 2020 10:10:23 GMT

Hello,

I am trying to use splunk to parse nessus results. I have managed to have them loaded, parsed and I get the fields I wanted. The file general structure is

date foo bar MS11-049: Vulnerability in the Microsoft XML Editor Could...
date foo bar Microsoft: Vulnerabilities in GDI+ Could Allow...
date foo bar Firefox 6 Multiple Vulnerabilities...
date foo bar Adobe Reader < 10.1...
date foo bar Adobe Flash Multiple...

date is the timestamp, foo and bar are some info and the remaining of the line is the name of the vulnerability. date, foo, bar and the vulnerability are available as fields N_date, N_foo, N_bar and N_vuln.

I would like to create a new field N_family which would give the product family. In the case above this would be Microsoft (matching MS and Microsoft), Firefox (matching Firefox) and Adobe (matching Adobe).

Is it possible to add such a new field to the existing list of fields extracted from the event?
Is it possible to do it at indexing time?

I will be trying to do the same with IP addresses which match a given site (I found some useful information here and hope that the mechanism above would be applicable as well, if it exists).

Thank you for any pointers!

Re: adding a field based on other fields

MHibbin — Thu, 01 Dec 2011 16:49:33 GMT

Right I know it is not at index time, but this is the what I think realistically you are best off doing... using a csv file as a lookup. So your first step should be to extract the product family members... you could do something like...

source="<your_source_type>" | rex field=N_vuln "(?P<N_group>\w+[^\d\:\-\s])" | where isnotnull(N_group) | stats count by N_group

This will show you a list of the N_group members....

N_group        count
Adobe           2
Firefox         1
MS              1
Microsoft       1

Then you should have a manually (or it can be scripted if you like) created and maintained csv (this example I use family.csv and put it in $SPLUNK_HOME/etc/apps/search/lookups) which has the family and members, something like...

group,family
MS,Microsoft
Microsoft,Microsoft
Adobe,Adobe
Firefox,Firefox

Then modify the command to include the lookup... such as...

source="/var/tmp/test1.log" | rex field=N_vuln "(?P<N_group>\w+[^\d\:\-])" | where isnotnull(N_group) | stats count by N_group | lookup family.csv group as N_group OUTPUT family as N_family

And then you should have the N_family field you desire.

Regards,

P.S Please note, that you don't need to keep the "stats" command as this is just for demo to show your values are working.

P.P.S... I think this is better, as it is not a static format, such as IP addresses. So you are not committing any changes which could end being faulty to the index, which could require cleaning the index. You could probably save this as saved search to call on, so you don't have to have the whole string.

Re: adding a field based on other fields

wsw70 — Fri, 02 Dec 2011 08:28:16 GMT

Thanks a lot for the very complete answer. I will test it right away 🙂