Getting Data In

Extract fields from CSV log file without header

kvnpichon
Path Finder

Hello,

I have a CSV file in this form :

 

2021-08-30 15:45:32;MOZILLA;j.dupont;FR6741557ERF;1.1.1.1;CONNEXION;;
2021-08-30 15:45:24;MOZILLA;j.dupont;FR6741557ERF;1.1.1.1;STATUS;;BDD
2021-08-30 15:45:16;MOZILLA;j.dupontFR6741557ERF;1.1.1.1;START;App_start;WEB

 

Corresponding to these 8 fields : date,application,user,host,ip,type,detail,module

I have 2 questions :

  1. How can I extract these fields ?
  2. How can I extract field at search-time (to be able to be retroactive on old logs) ?

This my actuals props.conf and transforms.conf deployed on Search Head + Indexers and the inputs.conf file on the Universal Forwarder :

props.conf

 

[csvlogs]
disabled = false
TIME_PREFIX = ^
TIME_FORMAT = %Y-%m-%d %H:%M:%S
MAX_TIMESTAMP_LOOKAHEAD = 19
LINE_BREAKER = ([\r\n]+)
SHOULD_LINEMERGE = false
KV_MODE = none
REPORT-fieldsextraction = logs_fields

 

transforms.conf

 

[logs_fields]
DELIMS = ";"
FIELDS = date,application,user,hostname,ip,type,detail,module
KEEP_EMPTY_VALS = true

 

inputs.conf

 

[Monitor://D:\repository\logs.csv]
disabled = false
sourcetype=csvlogs
index=logs_index1

 

Do you have solutions ?

Labels (1)
0 Karma

kvnpichon
Path Finder

Hi guys, I still didn't find any solution, any body could help me ?

0 Karma

ashvinpandey
Contributor

@kvnpichon This post can help you please take a look:
https://blog.avotrix.com/different-ways-to-remove-headers-in-splunk/ 
Also, If this reply helps you, an upvote would be appreciated.

kvnpichon
Path Finder

Hello @ashvinpandey ,

In fact I have no header line in my log file, the process you sent me allow me to delete the header line but doesn't extract fields from the csv logs file.

0 Karma
Get Updates on the Splunk Community!

Building Reliable Asset and Identity Frameworks in Splunk ES

 Accurate asset and identity resolution is the backbone of security operations. Without it, alerts are ...

Cloud Monitoring Console - Unlocking Greater Visibility in SVC Usage Reporting

For Splunk Cloud customers, understanding and optimizing Splunk Virtual Compute (SVC) usage and resource ...

Automatic Discovery Part 3: Practical Use Cases

If you’ve enabled Automatic Discovery in your install of the Splunk Distribution of the OpenTelemetry ...