topic Re: Field extraction and search issue in Splunk Search

Field extraction and search issue

mbasharat — Thu, 05 Nov 2020 20:05:29 GMT

Hi,

I am dealing with an issue because data changed from my source. I was using a lookup as below to search only on the hosts that are in my lookup. This field name NETBIOS was always coming as UNKNOWN\samplehost so I did a simple eval and added UNKNOWN\ with host name in lookup query and that worked great.

Now, the data has changed in a way that I am seeing domain coming from data source in netbiosName field e.g.

ABC\host1

XYZ\host2

How can I structure a search where I can filter upfront as in search above regardless of the domain value that come in? I can get rid of the "\" but this means that I will have to eval or rex before everything and then do a match which takes a toll on query performance. My query was taking only about 10 seconds for ~5k hosts matching from lookup to index but aforementioned way cause it to run for ~ 20 mins because it has to go thru all hosts and then do a match on the ones in lookup. Thank in-advance!!!

Re: Field extraction and search issue

bowesmana — Thu, 05 Nov 2020 21:11:19 GMT

@mbasharat

You could change the "UNKNOWN" to "*" in the subsearch, but not sure what that would do for performance with the leading wildcard search.

Alternatively, you can create a new calculated field definition, where you remove the domain from the start of the netbios name and create a new field, say, netbiosHost.

Then in your subsearch, you return the field

index=source1sample sourcetype="samplesourcetype" [ | inputlookup sample.csv | table sample_netbios | eval sample_netbios=upper(sample_netbios) | rename sample_netbios as netbiosHost ]

Create your calculated field like is set up using the replace shown in this query

| makeresults | eval netbiosName="ABC\HOST" | eval netbiosHost=replace(netbiosName,"[^\\\]*\\\(.*)","\1")

Then you don't need to do the search for all hosts followed by eval as Splunk is doing the calculated field for you.

Re: Field extraction and search issue

mbasharat — Fri, 06 Nov 2020 13:57:17 GMT

Hi bowesmana,

How to consolidate an additional field for netbiosDomain in your below eval please?

| makeresults | eval netbiosName="ABC\HOST" | eval netbiosHost=replace(netbiosName,"[^\\\]*\\\(.*)","\1") | eval netbiosDomain=???

I have also tried below and this one works as well. However, which one will be a better approach?

| eval Split=split(netbiosName,"\\") | eval SplitHost=mvindex(Split,1) | eval SplitDomain=mvindex(Split,0)

By using * in place of UNKNOWN as you suggest produces below error:

Regex: UTF-8 error: isolated byte with 0x80 bit set.

Can you provide a RegEx option to extract both Domain and Host that I can add in props.conf and have the fields extracted at the backend so I can do my searches and calculations as normal?

Re: Field extraction and search issue

bowesmana — Sun, 08 Nov 2020 22:57:36 GMT

@mbasharat

You will need to set up a new field transformation with

(?<netbiosDomain>[^\\]+)\\(?<netbiosHost>.*)

and your source key as netbiosName. Then create your field extraction to use this transformation. The should give you both fields

Re: Field extraction and search issue

mbasharat — Tue, 10 Nov 2020 16:08:58 GMT

Hi @ bowesmana,

Below worked,

| makeresults | eval netbiosName="ABC\HOST" | rex fieldName=netbiosName "(?<NETBIOS_Domain>\w+)\\\(?<NETBIOS_Host>\w+)"

I ended up using below evals in props,

| makeresults | eval netbiosName="ABC\HOST" | eval splitHost=mvindex(split(netbiosName,"\\"),1) | eval splitDomain=mvindex(split(netbiosName,"\\"),0)