Field extraction and search issue

[mbasharat]

Hi,

I am dealing with an issue because data changed from my source. I was using a lookup as below to search only on the hosts that are in my lookup. This field name NETBIOS was always coming as UNKNOWN\samplehost so I did a simple eval and added UNKNOWN\ with host name in lookup query and that worked great.

index=source1sample sourcetype="samplesourcetype"

[| inputlookup sample.csv | table sample_netbios | eval sample_netbios=upper(sample_netbios) | rename sample_netbios as netbiosName | eval netbiosName="UNKNOWN"."\\". netbiosName]

Now, the data has changed in a way that I am seeing domain coming from data source in netbiosName field e.g.

ABC\host1

XYZ\host2

How can I structure a search where I can filter upfront as in search above regardless of the domain value that come in? I can get rid of the "\" but this means that I will have to eval or rex before everything and then do a match which takes a toll on query performance. My query was taking only about 10 seconds for ~5k hosts matching from lookup to index but aforementioned way cause it to run for ~ 20 mins because it has to go thru all hosts and then do a match on the ones in lookup. Thank in-advance!!!

[bowesmana]

@mbasharat 

You could change the "UNKNOWN" to "*" in the subsearch, but not sure what that would do for performance with the leading wildcard search.

Alternatively, you can create a new calculated field definition, where you remove the domain from the start of the netbios name and create a new field, say, netbiosHost.

Then in your subsearch, you return the field

index=source1sample sourcetype="samplesourcetype" [
  | inputlookup sample.csv 
  | table sample_netbios 
  | eval sample_netbios=upper(sample_netbios) 
  | rename sample_netbios as netbiosHost
] 

 Create your calculated field like is set up using the replace shown in this query

| makeresults
| eval netbiosName="ABC\HOST"
| eval netbiosHost=replace(netbiosName,"[^\\\]*\\\(.*)","\1")

Then you  don't need to do the search for all hosts followed by eval as Splunk is doing the calculated field for you.

[mbasharat]

Hi bowesmana,

How to consolidate an additional field for netbiosDomain in your below eval please?

| makeresults
| eval netbiosName="ABC\HOST"
| eval netbiosHost=replace(netbiosName,"[^\\\]*\\\(.*)","\1")
| eval netbiosDomain=???

I have also tried below and this one works as well. However, which one will be a better approach?

| eval Split=split(netbiosName,"\\")
| eval SplitHost=mvindex(Split,1)
| eval SplitDomain=mvindex(Split,0) 

By using * in place of UNKNOWN as you suggest produces below error:

Regex: UTF-8 error: isolated byte with 0x80 bit set.

Can you provide a RegEx option to extract both Domain and Host that I can add in props.conf and have the fields extracted at the backend so I can do my searches and calculations as normal?

[bowesmana · Accepted solution]

@mbasharat 

You will need to set up a new field transformation with

(?<netbiosDomain>[^\\]+)\\(?<netbiosHost>.*)

 and your source key as netbiosName. Then create your field extraction to use this transformation. The should give you both fields

[mbasharat]

Hi @ bowesmana,

Below worked,

| makeresults
| eval netbiosName="ABC\HOST"
| rex fieldName=netbiosName "(?<NETBIOS_Domain>\w+)\\\(?<NETBIOS_Host>\w+)"

I ended up using below evals in props,

| makeresults
| eval netbiosName="ABC\HOST"
| eval splitHost=mvindex(split(netbiosName,"\\"),1)
| eval splitDomain=mvindex(split(netbiosName,"\\"),0)