Deployment Architecture

Unable to remove header from CSV file pulling from S3

rajasha
Explorer

Can someone help here please. I'm trying to remove the header which is currently adding as header as a events in the parsing which needs to remove. 

Also time stamp is not correct. Below is config from props.conf

KV_MODE = auto
SHOULD_LINEMERGE = false
EVENT_BREAKER_ENABLE = true
DATETIME_CONFIG = NONE
CHARSET=UTF-8
INDEXED_EXTRACTIONS=CSV
HEADER_FIELD_LINE_NUMBER=1
TIMESTAMP_FIELDS=Date,Time
LINE_BREAKER = ([\r\n]+)
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
disabled = false
pulldown_type = 1
TIMESTAMP_FIELDS = Date,Time
FIELD_DELIMITER = ,
FIELD_QUOTE = "
CHECK_FOR_HEADER = true

@splunk @Anonymous 

Labels (2)
0 Karma

rajasha
Explorer

Thanks for your reply. Here you go. 

Header:

"Date","Time","Action","Category Name","Localized Country","Policy Name","User","Workstation","Domain","Protocol","Query","URL - Full","Cloud App Name","Cloud App Category","Connection IP","Connection IP Country","Destination IP","Destination IP Country","Source IP","Analytic Name","Threat Type","Full MIME Type","Referrer URL - Full","Referrer Query","Browser Type","Operating System","Bytes Sent","Bytes Received","Bandwidth","Authentication Method","Classification Type","HTTP Status Code","Port","TLS Version (Downstream)","Request Method"

Events:

"12/02/2021","15:20:03","Allowed","Information Technology","ie","##DEFAULT_Policy","paneer@gmail.com","dc-dc4","cloudsink.net","None","None","lfodown01-b.cloudsink.net:443/","None","None","52.48.70.94","Ireland","54.183.120.141","United States","10.10.75.16","None","None","None","None","None","Unknown","Unknown","649","None","","Endpoint (Proxy Connect)","Static Classification","None","443","None","Connect"

 

time stamp in the event logs, is not matching with the search head results. 

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Please share a sample header and events.  Also, what is incorrect about the timestamp?

---
If this reply helps you, Karma would be appreciated.

rajasha
Explorer

Thanks for your reply. Here you go. 

Header:

"Date","Time","Action","Category Name","Localized Country","Policy Name","User","Workstation","Domain","Protocol","Query","URL - Full","Cloud App Name","Cloud App Category","Connection IP","Connection IP Country","Destination IP","Destination IP Country","Source IP","Analytic Name","Threat Type","Full MIME Type","Referrer URL - Full","Referrer Query","Browser Type","Operating System","Bytes Sent","Bytes Received","Bandwidth","Authentication Method","Classification Type","HTTP Status Code","Port","TLS Version (Downstream)","Request Method"

Events:

"12/02/2021","15:20:03","Allowed","Information Technology","ie","##DEFAULT_Policy","paneer@gmail.com","dc-dc4","cloudsink.net","None","None","lfodown01-b.cloudsink.net:443/","None","None","52.48.70.94","Ireland","54.183.120.141","United States","10.10.75.16","None","None","None","None","None","Unknown","Unknown","649","None","","Endpoint (Proxy Connect)","Static Classification","None","443","None","Connect"

 

time stamp in the event logs, is not matching with the search head results. 

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Announcing Modern Navigation: A New Era of Splunk User Experience

We are excited to introduce the Modern Navigation feature in the Splunk Platform, available to both cloud and ...

Observability Simplified: Combining User Experience, Application Performance & ...

Tech Talk Observability Simplified: Combining User Experience, Application Performance & Network ...

Event Series May & June: From Network Visibility to Service Intelligence

Unifying the Network: Moving from Alert Noise to Service Intelligence with Splunk ITSI In today’s hybrid ...