Getting Data In

Extract fields from CSV log file without header

kvnpichon
Path Finder

Hello,

I have a CSV file in this form :

 

2021-08-30 15:45:32;MOZILLA;j.dupont;FR6741557ERF;1.1.1.1;CONNEXION;;
2021-08-30 15:45:24;MOZILLA;j.dupont;FR6741557ERF;1.1.1.1;STATUS;;BDD
2021-08-30 15:45:16;MOZILLA;j.dupontFR6741557ERF;1.1.1.1;START;App_start;WEB

 

Corresponding to these 8 fields : date,application,user,host,ip,type,detail,module

I have 2 questions :

  1. How can I extract these fields ?
  2. How can I extract field at search-time (to be able to be retroactive on old logs) ?

This my actuals props.conf and transforms.conf deployed on Search Head + Indexers and the inputs.conf file on the Universal Forwarder :

props.conf

 

[csvlogs]
disabled = false
TIME_PREFIX = ^
TIME_FORMAT = %Y-%m-%d %H:%M:%S
MAX_TIMESTAMP_LOOKAHEAD = 19
LINE_BREAKER = ([\r\n]+)
SHOULD_LINEMERGE = false
KV_MODE = none
REPORT-fieldsextraction = logs_fields

 

transforms.conf

 

[logs_fields]
DELIMS = ";"
FIELDS = date,application,user,hostname,ip,type,detail,module
KEEP_EMPTY_VALS = true

 

inputs.conf

 

[Monitor://D:\repository\logs.csv]
disabled = false
sourcetype=csvlogs
index=logs_index1

 

Do you have solutions ?

Labels (1)
0 Karma

kvnpichon
Path Finder

Hi guys, I still didn't find any solution, any body could help me ?

0 Karma

ashvinpandey
Contributor

@kvnpichon This post can help you please take a look:
https://blog.avotrix.com/different-ways-to-remove-headers-in-splunk/ 
Also, If this reply helps you, an upvote would be appreciated.

kvnpichon
Path Finder

Hello @ashvinpandey ,

In fact I have no header line in my log file, the process you sent me allow me to delete the header line but doesn't extract fields from the csv logs file.

0 Karma
Get Updates on the Splunk Community!

Take Your Breath Away with Splunk Risk-Based Alerting (RBA)

WATCH NOW!The Splunk Guide to Risk-Based Alerting is here to empower your SOC like never before. Join Haylee ...

SignalFlow: What? Why? How?

What is SignalFlow? Splunk Observability Cloud’s analytics engine, SignalFlow, opens up a world of in-depth ...

Federated Search for Amazon S3 | Key Use Cases to Streamline Compliance Workflows

Modern business operations are supported by data compliance. As regulations evolve, organizations must ...