Getting Data In

Extract fields from CSV log file without header

kvnpichon
Path Finder

Hello,

I have a CSV file in this form :

 

2021-08-30 15:45:32;MOZILLA;j.dupont;FR6741557ERF;1.1.1.1;CONNEXION;;
2021-08-30 15:45:24;MOZILLA;j.dupont;FR6741557ERF;1.1.1.1;STATUS;;BDD
2021-08-30 15:45:16;MOZILLA;j.dupontFR6741557ERF;1.1.1.1;START;App_start;WEB

 

Corresponding to these 8 fields : date,application,user,host,ip,type,detail,module

I have 2 questions :

  1. How can I extract these fields ?
  2. How can I extract field at search-time (to be able to be retroactive on old logs) ?

This my actuals props.conf and transforms.conf deployed on Search Head + Indexers and the inputs.conf file on the Universal Forwarder :

props.conf

 

[csvlogs]
disabled = false
TIME_PREFIX = ^
TIME_FORMAT = %Y-%m-%d %H:%M:%S
MAX_TIMESTAMP_LOOKAHEAD = 19
LINE_BREAKER = ([\r\n]+)
SHOULD_LINEMERGE = false
KV_MODE = none
REPORT-fieldsextraction = logs_fields

 

transforms.conf

 

[logs_fields]
DELIMS = ";"
FIELDS = date,application,user,hostname,ip,type,detail,module
KEEP_EMPTY_VALS = true

 

inputs.conf

 

[Monitor://D:\repository\logs.csv]
disabled = false
sourcetype=csvlogs
index=logs_index1

 

Do you have solutions ?

Labels (1)
0 Karma

kvnpichon
Path Finder

Hi guys, I still didn't find any solution, any body could help me ?

0 Karma

ashvinpandey
Contributor

@kvnpichon This post can help you please take a look:
https://blog.avotrix.com/different-ways-to-remove-headers-in-splunk/ 
Also, If this reply helps you, an upvote would be appreciated.

kvnpichon
Path Finder

Hello @ashvinpandey ,

In fact I have no header line in my log file, the process you sent me allow me to delete the header line but doesn't extract fields from the csv logs file.

0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.

Can’t make it to .conf25? Join us online!

Get Updates on the Splunk Community!

Community Content Calendar, September edition

Welcome to another insightful post from our Community Content Calendar! We're thrilled to continue bringing ...

Splunkbase Unveils New App Listing Management Public Preview

Splunkbase Unveils New App Listing Management Public PreviewWe're thrilled to announce the public preview of ...

Leveraging Automated Threat Analysis Across the Splunk Ecosystem

Are you leveraging automation to its fullest potential in your threat detection strategy?Our upcoming Security ...