How to extract field with variable field

[michael_wong]

In transforms.conf I can use DELIMS to extract the field by fixed format.

My question is, if one of the field is changeable, how can we resolve that?

[REPORT-DP-fields]
DELIMS = " "
FIELDS = "Month","Day","Time","Mgmt_IP","Device","Date","DP_Time","Severity","Radware_ID","attack_category","attack_name","protocol","src_ip","src_port","dst_ip","dst_port","physical_port","Context","policy","status","packet_number","bandwidth","vlan","mpls_rd","mpls_tag","risk","action","id"

Here is the sample

In below sample 1, the fifth field are Device (split by space), but in sample 2, it's in sixth field

Raw data as below

#1.Mar 11111111 11111111:11111111:11111111 11111111.111111111111.11111111.111111111111 DefensePro: 11111111-11111111-1111111111111111 11111111:11111111:1111 WARNING 111111111111 Traffic-Filters "get_all" TCP 111111.1111.111111.1111 111111 111111.1111.111111.11 1111111111 11 Regular "match_all" sampled 11 1111 N/A 11 N/A high forward FFFFFFFF-FFFF-FFFF-11FDB-1111111111F11BD1111D

#2. Mar 1111 1111:1111:1111 logs-rv-sygdc-snat.systems GBWDC111111YI111111.systems.uk.hsbc/GBWDC111111YI111111.systems.uk.hsbc DefensePro: 1111-1111-11111111 1111:1111:1111 WARNING 111111 Anomalies "Invalid TCP Flags" IP 11.11.11.11 11 11.11.11.11 11 11 Regular "Packet Anomalies" occur 11 11 N/A 11 N/A low drop FFFFFFFF-FFFF-FFFF-111111E-11111111D11D111111DD

Thanks,

Michael

[scelikok]

Hi @michael_wong,

You can use host based transforms to achieve this. Define new transform setting for second sample like DP_six fieldsDevice . And call this transform using host stanza.

On props.conf

[host::host_with_sixfield]
REPORT-DP_six fieldsDevice

[michael_wong]

Hi @scelikok,

Thanks for your answer. Can you tell more about how to make priority of the DP_six fieldsDevice?

I have made the change, but looks it didn't take effect. If two report defined in transform.conf, which one will take effect?

[isoutamo]

Here is defined precedences over source, host, sourcetype https://docs.splunk.com/Documentation/Splunk/latest/Admin/Propsconf.

Can you share your configurations, so we can easier help you.

r. Ismo

[isoutamo]

Are those in some log file/feed or are they from different source/logs?

[michael_wong]

No, they are same source, but have a bit difference since configuration inconsistent