Fields Extraction in Splunk

Alankrit — Wed, 18 Sep 2024 17:38:23 GMT

We are using v9 format of logs in splunk. It is working fine and we are able to see logs in splunk as expected.

We added 4 more fields in transform.conf and test the addon in splunk. Then additional fields taking the value

of s3_filename, bucket name and prefix which are added at the end which is not correct behavior.

We are looking for solution with that we should be able to parse correct value in correct field and the additional fields should have null values if there is no values for them in logs.

transform.conf [proxylogs_fields] DELIMS = "," FIELDS = Timestamp,policy_identities,src,src_translated_ip,dest,content_type,action,url,http_referrer,http_user_agent,status,requestSize,responseSize,responseBodySize,sha256,category,av_detection,pua,amp_disposition,amp_malwarename,amp_score,policy_identity_type,blocked_category,identities,identity_type,request_method,dlp_status,certificate_errors,filename,rulesetID,ruleID,destinationListID,isolateAction,fileAction,warnStatus,forwarding_method,Producer,test_feild1,test_field2,test_field3,test_field4,s3_filename,aws_bucket_name,aws_prefix props.conf [cisco:cloud_security:proxy] REPORT-proxylogs-fields = proxylogs_fields,extract_url_domain LINE_BREAKER = ([\r\n]+) # EVENT_BREAKER = ([\r\n]+) # EVENT_BREAKER_ENABLE = true SHOULD_LINEMERGE = false CHARSET = AUTO disabled = false TRUNCATE = 1000000 MAX_EVENTS = 1000000 EVAL-product = "Cisco Secure Access and Umbrella" EVAL-vendor = "Cisco" EVAL-vendor_product = "Cisco Secure Access/Umbrella" MAX_TIMESTAMP_LOOKAHEAD = 22 NO_BINARY_CHECK = true TIME_PREFIX = ^ TIME_FORMAT = "%Y-%m-%d %H:%M:%S" TZ = UTC FIELDALIAS-bytes_in = requestSize as bytes_in FIELDALIAS-bytes_out = responseSize as bytes_out EVAL-action = lower(action) EVAL-app = "Cisco Cloud Security" FIELDALIAS-http_content_type = content_type as http_content_type EVAL-http_user_agent_length = len(http_user_agent) EVAL-url_length = len(url) EVAL-dest = if(isnotnull(dest),dest,url_domain) EVAL-bytes = requestSize + responseSize

Re: Fields Extraction in Splunk

dural_yyz — Wed, 18 Sep 2024 15:01:30 GMT

I think I understand what you are asking about but without sample ingested data and the new output sample it is harder to decipher what is going wrong.

topic Re: Fields Extraction in Splunk in Splunk Search

Fields Extraction in Splunk

Re: Fields Extraction in Splunk