Getting Data In

Extract fields from CSV log file without header

kvnpichon
Path Finder

Hello,

I have a CSV file in this form :

 

2021-08-30 15:45:32;MOZILLA;j.dupont;FR6741557ERF;1.1.1.1;CONNEXION;;
2021-08-30 15:45:24;MOZILLA;j.dupont;FR6741557ERF;1.1.1.1;STATUS;;BDD
2021-08-30 15:45:16;MOZILLA;j.dupontFR6741557ERF;1.1.1.1;START;App_start;WEB

 

Corresponding to these 8 fields : date,application,user,host,ip,type,detail,module

I have 2 questions :

  1. How can I extract these fields ?
  2. How can I extract field at search-time (to be able to be retroactive on old logs) ?

This my actuals props.conf and transforms.conf deployed on Search Head + Indexers and the inputs.conf file on the Universal Forwarder :

props.conf

 

[csvlogs]
disabled = false
TIME_PREFIX = ^
TIME_FORMAT = %Y-%m-%d %H:%M:%S
MAX_TIMESTAMP_LOOKAHEAD = 19
LINE_BREAKER = ([\r\n]+)
SHOULD_LINEMERGE = false
KV_MODE = none
REPORT-fieldsextraction = logs_fields

 

transforms.conf

 

[logs_fields]
DELIMS = ";"
FIELDS = date,application,user,hostname,ip,type,detail,module
KEEP_EMPTY_VALS = true

 

inputs.conf

 

[Monitor://D:\repository\logs.csv]
disabled = false
sourcetype=csvlogs
index=logs_index1

 

Do you have solutions ?

Labels (1)
0 Karma

kvnpichon
Path Finder

Hi guys, I still didn't find any solution, any body could help me ?

0 Karma

ashvinpandey
Contributor

@kvnpichon This post can help you please take a look:
https://blog.avotrix.com/different-ways-to-remove-headers-in-splunk/ 
Also, If this reply helps you, an upvote would be appreciated.

kvnpichon
Path Finder

Hello @ashvinpandey ,

In fact I have no header line in my log file, the process you sent me allow me to delete the header line but doesn't extract fields from the csv logs file.

0 Karma
Get Updates on the Splunk Community!

Index This | Why do they call it hyper text?

November 2023 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

State of Splunk Careers 2023: Career Resilience and the Continued Value of Splunk

For the past three years, Splunk has partnered with Enterprise Strategy Group to conduct a survey that gauges ...

The Great Resilience Quest: 9th Leaderboard Update

The ninth leaderboard update (11.9-11.22) for The Great Resilience Quest is out >> Kudos to all the ...