Deployment Architecture

Unable to remove header from CSV file pulling from S3

rajasha
Explorer

Can someone help here please. I'm trying to remove the header which is currently adding as header as a events in the parsing which needs to remove. 

Also time stamp is not correct. Below is config from props.conf

KV_MODE = auto
SHOULD_LINEMERGE = false
EVENT_BREAKER_ENABLE = true
DATETIME_CONFIG = NONE
CHARSET=UTF-8
INDEXED_EXTRACTIONS=CSV
HEADER_FIELD_LINE_NUMBER=1
TIMESTAMP_FIELDS=Date,Time
LINE_BREAKER = ([\r\n]+)
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
disabled = false
pulldown_type = 1
TIMESTAMP_FIELDS = Date,Time
FIELD_DELIMITER = ,
FIELD_QUOTE = "
CHECK_FOR_HEADER = true

@splunk @BSplunk 

Labels (3)
0 Karma

rajasha
Explorer

Thanks for your reply. Here you go. 

Header:

"Date","Time","Action","Category Name","Localized Country","Policy Name","User","Workstation","Domain","Protocol","Query","URL - Full","Cloud App Name","Cloud App Category","Connection IP","Connection IP Country","Destination IP","Destination IP Country","Source IP","Analytic Name","Threat Type","Full MIME Type","Referrer URL - Full","Referrer Query","Browser Type","Operating System","Bytes Sent","Bytes Received","Bandwidth","Authentication Method","Classification Type","HTTP Status Code","Port","TLS Version (Downstream)","Request Method"

Events:

"12/02/2021","15:20:03","Allowed","Information Technology","ie","##DEFAULT_Policy","paneer@gmail.com","dc-dc4","cloudsink.net","None","None","lfodown01-b.cloudsink.net:443/","None","None","52.48.70.94","Ireland","54.183.120.141","United States","10.10.75.16","None","None","None","None","None","Unknown","Unknown","649","None","","Endpoint (Proxy Connect)","Static Classification","None","443","None","Connect"

 

time stamp in the event logs, is not matching with the search head results. 

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Please share a sample header and events.  Also, what is incorrect about the timestamp?

---
If this reply helps you, Karma would be appreciated.

rajasha
Explorer

Thanks for your reply. Here you go. 

Header:

"Date","Time","Action","Category Name","Localized Country","Policy Name","User","Workstation","Domain","Protocol","Query","URL - Full","Cloud App Name","Cloud App Category","Connection IP","Connection IP Country","Destination IP","Destination IP Country","Source IP","Analytic Name","Threat Type","Full MIME Type","Referrer URL - Full","Referrer Query","Browser Type","Operating System","Bytes Sent","Bytes Received","Bandwidth","Authentication Method","Classification Type","HTTP Status Code","Port","TLS Version (Downstream)","Request Method"

Events:

"12/02/2021","15:20:03","Allowed","Information Technology","ie","##DEFAULT_Policy","paneer@gmail.com","dc-dc4","cloudsink.net","None","None","lfodown01-b.cloudsink.net:443/","None","None","52.48.70.94","Ireland","54.183.120.141","United States","10.10.75.16","None","None","None","None","None","Unknown","Unknown","649","None","","Endpoint (Proxy Connect)","Static Classification","None","443","None","Connect"

 

time stamp in the event logs, is not matching with the search head results. 

0 Karma
Get Updates on the Splunk Community!

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...

Tech Talk | Elevating Digital Service Excellence: The Synergy of Splunk RUM & APM

Elevating Digital Service Excellence: The Synergy of Real User Monitoring and Application Performance ...

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...