I am attempting to use SEDCMD on ingest to eliminate extra "data" from my logs (and license). This will be running on Heavy Forwarder. Turns out SEDCMD only works on _raw during ingest which is complicated with the Palo TA as it separates CONFIG, THREAT, TRAFFIC, etc.. into their own sourcetypes, so I have to operate off of sourcetype=pan:log which looks like this:
Jan 7 10:19:47 palohost 1,2021/06/07: 15:19:46,011901036309,GLOBALPROTECT,...
Jan 7 10:19:47 palohost 1,2021/06/07: 15:19:46,011901036309,HIPMATCH,...
Jan 7 10:19:47 palohost 1,2021/06/07: 15:19:46,011901036309,CONFIG,...
Jan 7 10:19:47 palohost 1,2021/06/07: 15:19:46,011901036309,THREAT,...
Jan 7 10:19:47 palohost 1,2021/06/07: 15:19:46,011901036309,TRAFFIC,...
I need to perform the SEDCMD only on lines with a TRAFFIC in the 4th field, which I can identify just fine with:
^(?:[^,]*,){3}TRAFFIC
The challenge begins here. I need to capture the first field, which is from start of the line up to the first comma. So for this line:
Jan 7 10:19:47 palohost 1,2021/06/07: 15:19:46,011901036309,TRAFFIC,...
I only want to capture
Jan 7 10:19:47 palohost 1
Any advise?
Can you please try this?
SEDCMD-a = s/[^,]+(.*TRAFFIC,)/\1 /g
Screen:
Thanks
KV
▄︻̷̿┻̿═━一
If any of my reply helps you to solve the problem Or gain knowledge, an upvote would be appreciated.
Can you please try this?
[YOUR_SOURCE_TYPE]
SEDCMD-a = s/,(.*)TRAFFIC,(.*)$//g
.
.
.
Thanks
KV
▄︻̷̿┻̿═━一
If any of my reply helps you to solve the problem Or gain knowledge, an upvote would be appreciated.
that captures everything afterwards
thanks
@kamlesh_vaghela wrote:What is your expected OP from above screen?
KV
" I only want to capture
Jan 7 10:19:47 palohost 1
"
which would then be deleted in sedcmd
SEDCMD-a = s/,(.*)TRAFFIC,(.*)$//g
will give you below results.
Note. This configuration will work with new coming event only.
I have used this sample event.
Jan 7 10:19:47 palohost 1,2021/06/07: 15:19:46,011901036309,GLOBALPROTECT,...
Jan 7 10:19:47 palohost 1,2021/06/07: 15:19:46,011901036309,HIPMATCH,...
Jan 7 10:19:47 palohost 1,2021/06/07: 15:19:46,011901036309,CONFIG,...
Jan 7 10:19:47 palohost 1,2021/06/07: 15:19:46,011901036309,THREAT,...
Jan 7 10:19:47 palohost 1,2021/06/07: 15:19:46,011901036309,TRAFFIC,...
Thanks
KV
▄︻̷̿┻̿═━一
If any of my reply helps you to solve the problem Or gain knowledge, an upvote would be appreciated.
Thats pretty much the same result as the last and its trips everything but the first field. Results with your SEDCMD
5/28/21 3:32:29.100 PM | May 28 15:32:30 palohost 1
|
Source log line:
May 28 15:32:30 palohost 1,2021/05/28: 20:32:29,011901036309,TRAFFIC,end,2305,2021/05/28 20:32:29,<someip>,<someip>,0.0.0.0,0.0.0.0,<rule>,<user>,,<protocol>,<virtualsystem>,EXTTUNNEL,EXTINSIDE,tunnel,ethernet0/0,,,150000,,50000,443,0,0,,tcp,allow,143734,112711,31023,340,2021/05/28 20:30:59,75,<classification>,,,,,,,211,129,<action>,123,456,0,0,,palohost,from-policy,,,0,,0,,N/A,0,0,0,0,<some guid>,0,0,,,,,,,
Here is what I want it to look like without the first field.
5/28/21 3:32:29.100 PM | ,2021/05/28: 20:32:29,011901036309,TRAFFIC,end,2305,2021/05/28 20:32:29,<someip>,<someip>,0.0.0.0,0.0.0.0,<rule>,<user>,,<protocol>,<virtualsystem>,EXTTUNNEL,EXTINSIDE,tunnel,ethernet0/0,,,150000,,50000,443,0,0,,tcp,allow,143734,112711,31023,340,2021/05/28 20:30:59,75,<classification>,,,,,,,211,129,<action>,123,456,0,0,,palohost,from-policy,,,0,,0,,N/A,0,0,0,0,<some guid>,0,0,,,,,,,
|
Can you please try this?
SEDCMD-a = s/[^,]+(.*TRAFFIC,)/\1 /g
Screen:
Thanks
KV
▄︻̷̿┻̿═━一
If any of my reply helps you to solve the problem Or gain knowledge, an upvote would be appreciated.