Getting Data In

Extract fields from CSV log file without header

kvnpichon
Path Finder

Hello,

I have a CSV file in this form :

 

2021-08-30 15:45:32;MOZILLA;j.dupont;FR6741557ERF;1.1.1.1;CONNEXION;;
2021-08-30 15:45:24;MOZILLA;j.dupont;FR6741557ERF;1.1.1.1;STATUS;;BDD
2021-08-30 15:45:16;MOZILLA;j.dupontFR6741557ERF;1.1.1.1;START;App_start;WEB

 

Corresponding to these 8 fields : date,application,user,host,ip,type,detail,module

I have 2 questions :

  1. How can I extract these fields ?
  2. How can I extract field at search-time (to be able to be retroactive on old logs) ?

This my actuals props.conf and transforms.conf deployed on Search Head + Indexers and the inputs.conf file on the Universal Forwarder :

props.conf

 

[csvlogs]
disabled = false
TIME_PREFIX = ^
TIME_FORMAT = %Y-%m-%d %H:%M:%S
MAX_TIMESTAMP_LOOKAHEAD = 19
LINE_BREAKER = ([\r\n]+)
SHOULD_LINEMERGE = false
KV_MODE = none
REPORT-fieldsextraction = logs_fields

 

transforms.conf

 

[logs_fields]
DELIMS = ";"
FIELDS = date,application,user,hostname,ip,type,detail,module
KEEP_EMPTY_VALS = true

 

inputs.conf

 

[Monitor://D:\repository\logs.csv]
disabled = false
sourcetype=csvlogs
index=logs_index1

 

Do you have solutions ?

Labels (1)
0 Karma

kvnpichon
Path Finder

Hi guys, I still didn't find any solution, any body could help me ?

0 Karma

ashvinpandey
Contributor

@kvnpichon This post can help you please take a look:
https://blog.avotrix.com/different-ways-to-remove-headers-in-splunk/ 
Also, If this reply helps you, an upvote would be appreciated.

kvnpichon
Path Finder

Hello @ashvinpandey ,

In fact I have no header line in my log file, the process you sent me allow me to delete the header line but doesn't extract fields from the csv logs file.

0 Karma
Get Updates on the Splunk Community!

Stay Connected: Your Guide to November Tech Talks, Office Hours, and Webinars!

What are Community Office Hours? Community Office Hours is an interactive 60-minute Zoom series where ...

Index This | When is October more than just the tenth month?

October 2025 Edition  Hayyy Splunk Education Enthusiasts and the Eternally Curious!   We’re back with this ...

Observe and Secure All Apps with Splunk

  Join Us for Our Next Tech Talk: Observe and Secure All Apps with SplunkAs organizations continue to innovate ...