Hi All,
Requesting your help with
Log Example.
I have 54 fields separated by comma
The field data is variable and cannot be controlled.
Hence like grep cannot use match commands
Lastly I have zero knowledge on programming
2017/07/06,ab2,ag4,baa24,3456,34567,Server52,10.2.5.24,127.24.57.25,tcp,rst,,,,,,,,,......,,,vsys4,zone5,vpn
2017/07/06,ab3,ag4,baa24,3456,34567,Server52,10.2.5.24,127.24.57.25,tcp,rst,,,,,,,,,......,,,ssys4,Aone5,vpn
2017/07/06,ab4,ag4,baa24,3456,34567,Server52,10.2.5.24,127.24.57.25,tcp,rst,,,,,,,,,......,,,ssss4,zone5,vpn
2017/07/06,ab2,ag4,baa24,3456,34567,Server52,10.2.5.24,127.24.57.25,tcp,rst,,,,,,,,,......,,,vsys4,zone5,vpn
2017/07/06,ab3,ag4,baa24,3456,34567,Server52,10.2.5.24,127.24.57.25,tcp,rst,,,,,,,,,......,,,ssys4,Aone5,vpn
2017/07/06,ab3,ag4,baa24,3456,34567,Server52,10.2.5.24,127.24.57.25,tcp,rst,,,,,,,,,......,,,ssys4,Aone5,vpn
2017/07/06,ab4,ag4,baa24,3456,34567,Server52,10.2.5.24,127.24.57.25,tcp,rst,,,,,,,,,......,,,ssss4,zone5,vpn
2017/07/06,ab2,ag4,baa24,3456,34567,Server52,10.2.5.24,127.24.57.25,tcp,rst,,,,,,,,,......,,,vsys4,zone5,vpn
2017/07/05,ab2,ag4,baa24,3456,34567,Server52,10.2.5.24,127.24.57.25,tcp,rst,,,,,,,,,......,,,vsys4,zone5,vpn
2017/07/05,ab3,ag4,baa24,3456,34567,Server52,10.2.5.24,127.24.57.25,tcp,rst,,,,,,,,,......,,,ssys4,Aone5,vpn
2017/07/05,ab4,ag4,baa24,3456,34567,Server52,10.2.5.24,127.24.57.25,tcp,rst,,,,,,,,,......,,,ssss4,zone5,vpn
2017/07/05,ab2,ag4,baa24,3456,34567,Server52,10.2.5.24,127.24.57.25,tcp,rst,,,,,,,,,......,,,vsys4,zone5,vpn
2017/07/05,ab3,ag4,baa24,3456,34567,Server52,10.2.5.24,127.24.57.25,tcp,rst,,,,,,,,,......,,,ssys4,Aone5,vpn
2017/07/05,ab3,ag4,baa24,3456,34567,Server52,10.2.5.24,127.24.57.25,tcp,rst,,,,,,,,,......,,,ssys4,Aone5,vpn
2017/07/05,ab4,ag4,baa24,3456,34567,Server52,10.2.5.24,127.24.57.25,tcp,rst,,,,,,,,,......,,,ssss4,zone5,vpn
2017/07/05,ab2,ag4,baa24,3456,34567,Server52,10.2.5.24,127.24.57.25,tcp,rst,,,,,,,,,......,,,vsys4,zone5,vpn
I want to get hit counts for similar logs when fields 2,8,9,52 and 53 are same
Keep in mind that these field names cannot be used in command as it will change continuously
For example in field 52 if the name is vsys5 next log may ssys4 for ssss4
NO=========Fields
1=========2017/07/06,
2========= ab2,
3========= ag4,
4========= baa24,
5========= 3456,
6========= 34567,
7========= Server52,
8========= 10.2.5.24,
9========= 127.24.57.25,
10======== tcp,
11======== rst,
! ========= ****
! ========= ****
! ========= ****
! ========= ****
! ========= ****
52======== or or
53======== zone5,
54======== vpn
The result should group together similar logs having fields 2, 8 ,9,53 and 54 and show the total hit counts
The commands should not use any input from logs like vsys, zone as it keeps on changing and these field names are not constant
The command may be based on "," (comma) and OR index number
I should be able to capture the output based on position, whatever be in index 2,8,9,52,53 ........or so on
index2=====index8========index9=====index52=========index53
Desired result will be as below.
Field2====Field 2========== Field 3======Field 4==========Field 5=============hit count
ab2======10.2.5.24======127.24.57.25====vsys4==========zone5===================6
ab3======10.2.5.24======127.24.57.25====ssys4==========zone5===================6
ab3======10.2.5.24======127.24.57.25====ssss4==========zone5===================4
Kindly keep in mind that I have 54 fields separated by comma.
I tried with the below command. But I am not able to get output after field 13.
-----------------------------------------------------------------------------------------------------------------
sourcetype=************** | rex "^(?<index1>[^\,]+)\,(?<index2>[^\,]+)\,(?<index3>[^\,]+)\,(?<index4>[^\,]+)\,(?<index5>[^\,]+)\,(?<index6>[^\,]+)\,(?<index7>[^\,]+)\,(?<index8>[^\,]+)\,(?<index9>[^\,]+)\,(?<index10>[^\,]+)\,(?<index11>[^\,]+)\,(?<index12>[^\,]+)\,(?<index13>[^\s]+)" | stats count as hitcount by index12 index13
Apart from that as I do not have necessary privilege I am unable to upload screenshots.
Hi Dal Jeanis,
Thanks for you advise but your answer is based on the scenario that after 9th field doesn't matter or used field name "src="
The command may be based on "," (comma) and OR index number
I should be able to capture the output based on position, whatever be in index 2,8,9,52,53 ........or so on as I wish
***Keep in mind that these field names cannot be used in command as it will change continuously
For example in field 52 if the name is "vsys5" next log may "ssys4" for "ssss4" **
---------------------------------------------------------------------------------------------------------------------***
The question was modified after input from Dal Jeanis
... View more