Deployment Architecture

Unable to remove header from CSV file pulling from S3

rajasha
Explorer

Can someone help here please. I'm trying to remove the header which is currently adding as header as a events in the parsing which needs to remove. 

Also time stamp is not correct. Below is config from props.conf

KV_MODE = auto
SHOULD_LINEMERGE = false
EVENT_BREAKER_ENABLE = true
DATETIME_CONFIG = NONE
CHARSET=UTF-8
INDEXED_EXTRACTIONS=CSV
HEADER_FIELD_LINE_NUMBER=1
TIMESTAMP_FIELDS=Date,Time
LINE_BREAKER = ([\r\n]+)
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
disabled = false
pulldown_type = 1
TIMESTAMP_FIELDS = Date,Time
FIELD_DELIMITER = ,
FIELD_QUOTE = "
CHECK_FOR_HEADER = true

@splunk @BSplunk 

Labels (3)
0 Karma

rajasha
Explorer

Thanks for your reply. Here you go. 

Header:

"Date","Time","Action","Category Name","Localized Country","Policy Name","User","Workstation","Domain","Protocol","Query","URL - Full","Cloud App Name","Cloud App Category","Connection IP","Connection IP Country","Destination IP","Destination IP Country","Source IP","Analytic Name","Threat Type","Full MIME Type","Referrer URL - Full","Referrer Query","Browser Type","Operating System","Bytes Sent","Bytes Received","Bandwidth","Authentication Method","Classification Type","HTTP Status Code","Port","TLS Version (Downstream)","Request Method"

Events:

"12/02/2021","15:20:03","Allowed","Information Technology","ie","##DEFAULT_Policy","paneer@gmail.com","dc-dc4","cloudsink.net","None","None","lfodown01-b.cloudsink.net:443/","None","None","52.48.70.94","Ireland","54.183.120.141","United States","10.10.75.16","None","None","None","None","None","Unknown","Unknown","649","None","","Endpoint (Proxy Connect)","Static Classification","None","443","None","Connect"

 

time stamp in the event logs, is not matching with the search head results. 

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Please share a sample header and events.  Also, what is incorrect about the timestamp?

---
If this reply helps you, an upvote would be appreciated.

rajasha
Explorer

Thanks for your reply. Here you go. 

Header:

"Date","Time","Action","Category Name","Localized Country","Policy Name","User","Workstation","Domain","Protocol","Query","URL - Full","Cloud App Name","Cloud App Category","Connection IP","Connection IP Country","Destination IP","Destination IP Country","Source IP","Analytic Name","Threat Type","Full MIME Type","Referrer URL - Full","Referrer Query","Browser Type","Operating System","Bytes Sent","Bytes Received","Bandwidth","Authentication Method","Classification Type","HTTP Status Code","Port","TLS Version (Downstream)","Request Method"

Events:

"12/02/2021","15:20:03","Allowed","Information Technology","ie","##DEFAULT_Policy","paneer@gmail.com","dc-dc4","cloudsink.net","None","None","lfodown01-b.cloudsink.net:443/","None","None","52.48.70.94","Ireland","54.183.120.141","United States","10.10.75.16","None","None","None","None","None","Unknown","Unknown","649","None","","Endpoint (Proxy Connect)","Static Classification","None","443","None","Connect"

 

time stamp in the event logs, is not matching with the search head results. 

0 Karma
.conf21 CFS Extended through 5/20!

Don't miss your chance
to share your Splunk
wisdom in-person or
virtually at .conf21!

Call for Speakers has
been extended through
Thursday, 5/20!