Splunk Search

In notepad editor the field offset and its size is known , how to extract fields based upon offset ?

yogeshpunia05
Explorer

In notepad editor the field offset and its size is known , how to extract fields based upon offset ? AS log pattern is not same and may contain spaces instead of data , this is creating problem for extraction , I tried putting delimiter but when two fields are missing only one pipeline added, this disturb the extraction . 

example
line1: value1 value2 value3   value4 value5 
lin2:    value1                                value4  value5
line3   value1  value2                                value5 

using | create data like this : 

line1 value1|value2|value3|value4|value5
line2 value1|value4|value5
line3  value1|value2|value5 


Kindly suggest how we can index or extract fields properly so that 
data appears like : 

line1: value1 value2 value3   value4 value5 
lin2:    value1       na     na          value4  value5
line3   value1  value2       na     na          value5 

Labels (2)
0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

If you know how wide each field is, say 10 characters, with 1 space between, then something like:

| rex "^.{6} (?<value1>.{10}) (?<value2>.{10}) (?<value3>.{10}) (?<value4>.{10}) (?<value5>.{10})"

If the first column or the fields are different widths, modify the counts accordingly. 

You may want to trim the values extracted also to remove the trailing blanks, depending on your needs.

0 Karma

rnowitzki
Builder

Hi @yogeshpunia05,

Do you know how much space/blanks is between the fields when there is data and when there is not?

It looks like there is 1 between the values, when there are indeed values, like: value1<1 blank>value2

So, when values are missing, how many blanks are between the other values?
value1<? blanks>value3

BR
Ralph

--
Karma and/or Solution tagging appreciated.
0 Karma

yogeshpunia05
Explorer

Hi @rnowitzki spaces are based on base of field value , and can change as in original use case there can be multiple values that can go blank thus changing the count of spaces. 

0 Karma

rnowitzki
Builder

Yeah, I assumed that. But can you tell by the amount of spaces how many fields are missing? Is 1 field replaced by 1 blank (or tab?) when it's missing?

It can be probably done with a RegEx, but we would need to know how many spaces are in the line if 1 field is missing.

It would also be interesting, if there is a set amount of fields. Like if it is maximum 5 fields as in your examples, or if it can be more.

Thanks
Ralph

--
Karma and/or Solution tagging appreciated.
0 Karma
Get Updates on the Splunk Community!

Building Reliable Asset and Identity Frameworks in Splunk ES

 Accurate asset and identity resolution is the backbone of security operations. Without it, alerts are ...

Cloud Monitoring Console - Unlocking Greater Visibility in SVC Usage Reporting

For Splunk Cloud customers, understanding and optimizing Splunk Virtual Compute (SVC) usage and resource ...

Automatic Discovery Part 3: Practical Use Cases

If you’ve enabled Automatic Discovery in your install of the Splunk Distribution of the OpenTelemetry ...