Splunk Enterprise

How to extract field with variable field

michael_wong
Path Finder

In transforms.conf I can use DELIMS to extract the field by fixed format.

My question is, if one of the field is changeable, how can we resolve that?

[REPORT-DP-fields]
DELIMS = " "
FIELDS = "Month","Day","Time","Mgmt_IP","Device","Date","DP_Time","Severity","Radware_ID","attack_category","attack_name","protocol","src_ip","src_port","dst_ip","dst_port","physical_port","Context","policy","status","packet_number","bandwidth","vlan","mpls_rd","mpls_tag","risk","action","id"

Here is the sample

In below sample 1, the fifth field are Device (split by space), but in sample 2, it's in sixth field

Raw data as below

#1.Mar 11111111 11111111:11111111:11111111 11111111.111111111111.11111111.111111111111 DefensePro: 11111111-11111111-1111111111111111 11111111:11111111:1111 WARNING 111111111111 Traffic-Filters "get_all" TCP 111111.1111.111111.1111 111111 111111.1111.111111.11 1111111111 11 Regular "match_all" sampled 11 1111 N/A 11 N/A high forward FFFFFFFF-FFFF-FFFF-11FDB-1111111111F11BD1111D

#2. Mar 1111 1111:1111:1111 logs-rv-sygdc-snat.systems GBWDC111111YI111111.systems.uk.hsbc/GBWDC111111YI111111.systems.uk.hsbc DefensePro: 1111-1111-11111111 1111:1111:1111 WARNING 111111 Anomalies "Invalid TCP Flags" IP 11.11.11.11 11 11.11.11.11 11 11 Regular "Packet Anomalies" occur 11 11 N/A 11 N/A low drop FFFFFFFF-FFFF-FFFF-111111E-11111111D11D111111DD

Thanks,

Michael

Labels (1)
0 Karma

scelikok
SplunkTrust
SplunkTrust

Hi @michael_wong,

You can use host based transforms to achieve this. Define new transform setting for second sample like DP_six fieldsDevice . And call this transform using host stanza.

On props.conf

[host::host_with_sixfield]
REPORT-DP_six fieldsDevice

 

If this reply helps you an upvote is appreciated.
0 Karma

michael_wong
Path Finder

Hi @scelikok,

Thanks for your answer. Can you tell more about how to make priority of the DP_six fieldsDevice?

I have made the change, but looks it didn't take effect. If two report defined in transform.conf, which one will take effect?

0 Karma

isoutamo
SplunkTrust
SplunkTrust

Here is defined precedences over source, host, sourcetype https://docs.splunk.com/Documentation/Splunk/latest/Admin/Propsconf.

Can you share your configurations, so we can easier help you.

r. Ismo

0 Karma

isoutamo
SplunkTrust
SplunkTrust
Are those in some log file/feed or are they from different source/logs?
0 Karma

michael_wong
Path Finder

No, they are same source, but have a bit difference since configuration inconsistent

0 Karma
Get Updates on the Splunk Community!

Announcing General Availability of Splunk Incident Intelligence!

Digital transformation is real! Across industries, companies big and small are going through rapid digital ...

Splunk Training for All: Meet Aspiring Cybersecurity Analyst, Marc Alicea

Splunk Education believes in the value of training and certification in today’s rapidly-changing data-driven ...

The Splunk Success Framework: Your Guide to Successful Splunk Implementations

Splunk Lantern is a customer success center that provides advice from Splunk experts on valuable data ...