Deployment Architecture

Unable to remove header from CSV file pulling from S3

rajasha
Explorer

Can someone help here please. I'm trying to remove the header which is currently adding as header as a events in the parsing which needs to remove. 

Also time stamp is not correct. Below is config from props.conf

KV_MODE = auto
SHOULD_LINEMERGE = false
EVENT_BREAKER_ENABLE = true
DATETIME_CONFIG = NONE
CHARSET=UTF-8
INDEXED_EXTRACTIONS=CSV
HEADER_FIELD_LINE_NUMBER=1
TIMESTAMP_FIELDS=Date,Time
LINE_BREAKER = ([\r\n]+)
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
disabled = false
pulldown_type = 1
TIMESTAMP_FIELDS = Date,Time
FIELD_DELIMITER = ,
FIELD_QUOTE = "
CHECK_FOR_HEADER = true

@splunk @Anonymous 

Labels (2)
0 Karma

rajasha
Explorer

Thanks for your reply. Here you go. 

Header:

"Date","Time","Action","Category Name","Localized Country","Policy Name","User","Workstation","Domain","Protocol","Query","URL - Full","Cloud App Name","Cloud App Category","Connection IP","Connection IP Country","Destination IP","Destination IP Country","Source IP","Analytic Name","Threat Type","Full MIME Type","Referrer URL - Full","Referrer Query","Browser Type","Operating System","Bytes Sent","Bytes Received","Bandwidth","Authentication Method","Classification Type","HTTP Status Code","Port","TLS Version (Downstream)","Request Method"

Events:

"12/02/2021","15:20:03","Allowed","Information Technology","ie","##DEFAULT_Policy","paneer@gmail.com","dc-dc4","cloudsink.net","None","None","lfodown01-b.cloudsink.net:443/","None","None","52.48.70.94","Ireland","54.183.120.141","United States","10.10.75.16","None","None","None","None","None","Unknown","Unknown","649","None","","Endpoint (Proxy Connect)","Static Classification","None","443","None","Connect"

 

time stamp in the event logs, is not matching with the search head results. 

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Please share a sample header and events.  Also, what is incorrect about the timestamp?

---
If this reply helps you, Karma would be appreciated.

rajasha
Explorer

Thanks for your reply. Here you go. 

Header:

"Date","Time","Action","Category Name","Localized Country","Policy Name","User","Workstation","Domain","Protocol","Query","URL - Full","Cloud App Name","Cloud App Category","Connection IP","Connection IP Country","Destination IP","Destination IP Country","Source IP","Analytic Name","Threat Type","Full MIME Type","Referrer URL - Full","Referrer Query","Browser Type","Operating System","Bytes Sent","Bytes Received","Bandwidth","Authentication Method","Classification Type","HTTP Status Code","Port","TLS Version (Downstream)","Request Method"

Events:

"12/02/2021","15:20:03","Allowed","Information Technology","ie","##DEFAULT_Policy","paneer@gmail.com","dc-dc4","cloudsink.net","None","None","lfodown01-b.cloudsink.net:443/","None","None","52.48.70.94","Ireland","54.183.120.141","United States","10.10.75.16","None","None","None","None","None","Unknown","Unknown","649","None","","Endpoint (Proxy Connect)","Static Classification","None","443","None","Connect"

 

time stamp in the event logs, is not matching with the search head results. 

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Index This | What travels the world but is also stuck in place?

April 2026 Edition  Hayyy Splunk Education Enthusiasts and the Eternally Curious!   We’re back with this ...

Discover New Use Cases: Unlock Greater Value from Your Existing Splunk Data

Realizing the full potential of your Splunk investment requires more than just understanding current usage; it ...

Continue Your Journey: Join Session 2 of the Data Management and Federation Bootcamp ...

As data volumes continue to grow and environments become more distributed, managing and optimizing data ...