All Apps and Add-ons

Is there a way to check the field extraction regexes for a particular sourcetype?

Contributor

Is there a way to check the field extraction regexes for a particular sourcetype (say a sourcetype from the TA for AWS - aws:cloudwatchlogs:vpcflow?
I have reasons to believe that a few of the extractions are not working properly and extracting junk/irrelevant data. Is there a way to modify the extractions for the sourcetype of the add on?
I really would prefer not to create a new sourcetype with custom extractions as much as possible.

0 Karma
1 Solution

SplunkTrust
SplunkTrust

Yes! there is a way!

download the app (or TA) and navigate to .../default/ directory. open props.conf with your text editor and find your sourcetype [stanza]
in your case - this how it looks like:

[aws:cloudwatchlogs:vpcflow]
SHOULD_LINEMERGE = false
EXTRACT-all=^\s*(\d{4}-\d{2}-\d{2}.\d{2}:\d{2}:\d{2}[.\d\w]*)?\s*(?P<version>[^\s]+)\s+(?P<account_id>[^\s]+)\s+(?P<interface_id>[^\s]+)\s+(?P<src_ip>[^\s]+)\s+(?P<dest_ip>[^\s]+)\s+(?P<src_port>[^\s]+)\s+(?P<dest_port>[^\s]+)\s+(?P<protocol_code>[^\s]+)\s+(?P<packets>[^\s]+)\s+(?P<bytes>[^\s]+)\s+(?P<start_time>[^\s]+)\s+(?P<end_time>[^\s]+)\s+(?P<vpcflow_action>[^\s]+)\s+(?P<log_status>[^\s]+)

EVAL-duration=end_time-start_time
FIELDALIAS-src_ip_as_src = src_ip as src
FIELDALIAS-dest_ip_as_dest = dest_ip as dest

LOOKUP-protocol=vpcflow_protocol_code_lookup protocol_code OUTPUT protocol protocol_full_name
LOOKUP-action=vpcflow_action_lookup vpcflow_action OUTPUT action
REPORT-extract-region = extract-region


# unify account ID field
FIELDALIAS-aws-account-id = account_id as aws_account_id

if you would like to modify, create local directory under the app directory and create a new props.conf file in it.
write your relevant configurations and save. restart or debug refresh and you are all set.

NOTE: pay attention to references to transforms if any. (there are noon in this particular sourcetype) also pay attention to lookups as they are referred in this sourcetype props.

hope it helps

View solution in original post

0 Karma

SplunkTrust
SplunkTrust

Yes! there is a way!

download the app (or TA) and navigate to .../default/ directory. open props.conf with your text editor and find your sourcetype [stanza]
in your case - this how it looks like:

[aws:cloudwatchlogs:vpcflow]
SHOULD_LINEMERGE = false
EXTRACT-all=^\s*(\d{4}-\d{2}-\d{2}.\d{2}:\d{2}:\d{2}[.\d\w]*)?\s*(?P<version>[^\s]+)\s+(?P<account_id>[^\s]+)\s+(?P<interface_id>[^\s]+)\s+(?P<src_ip>[^\s]+)\s+(?P<dest_ip>[^\s]+)\s+(?P<src_port>[^\s]+)\s+(?P<dest_port>[^\s]+)\s+(?P<protocol_code>[^\s]+)\s+(?P<packets>[^\s]+)\s+(?P<bytes>[^\s]+)\s+(?P<start_time>[^\s]+)\s+(?P<end_time>[^\s]+)\s+(?P<vpcflow_action>[^\s]+)\s+(?P<log_status>[^\s]+)

EVAL-duration=end_time-start_time
FIELDALIAS-src_ip_as_src = src_ip as src
FIELDALIAS-dest_ip_as_dest = dest_ip as dest

LOOKUP-protocol=vpcflow_protocol_code_lookup protocol_code OUTPUT protocol protocol_full_name
LOOKUP-action=vpcflow_action_lookup vpcflow_action OUTPUT action
REPORT-extract-region = extract-region


# unify account ID field
FIELDALIAS-aws-account-id = account_id as aws_account_id

if you would like to modify, create local directory under the app directory and create a new props.conf file in it.
write your relevant configurations and save. restart or debug refresh and you are all set.

NOTE: pay attention to references to transforms if any. (there are noon in this particular sourcetype) also pay attention to lookups as they are referred in this sourcetype props.

hope it helps

View solution in original post

0 Karma

Contributor

oh. yes..figured it out. this had completely slipped my mind. Thanks

0 Karma