Splunk Enterprise

Define identification field

leandromatperei
Path Finder

Guys,

I need support, I need to upload these files and process this data, but I need them to be indexed by the _time field, I was unable to use the standard type of pre-defined data. Can you help me?

Another detail, the Valor and Frete fields have commas, and refer to currency, there is a way to treat this data before indexing, so that this data is treated as "." for example ?

Example file: https://www.dropbox.com/s/exh7g1glumxcetr/log.txt?dl=0

Labels (2)
Tags (1)
0 Karma
1 Solution

to4kawa
Ultra Champion

If you have SECCMD do what I'm doing and search in smart mode, you're basically good to go.

View solution in original post

0 Karma

to4kawa
Ultra Champion

How about SEDCMD with props.conf.

0 Karma

leandromatperei
Path Finder

@to4kawa 

I defined it that way within the sourcetype and it almost went, I just couldn't get the same match for some fields: Category and Shipping

 

[ <SOURCETYPE NAME> ]
SHOULD_LINEMERGE=true
LINE_BREAKER=([\r\n]+)
NO_BINARY_CHECK=true
CHARSET=AUTO
disabled=false
TIME_FORMAT=%Y/%m/%d  %H.%.M:%.S.%3N
TIME_PREFIX=_time
SEDCMD=s/=(\d.*?)(?=,[A-Z])/="\1"/g

 

 
 
How can I leave these fields the same way?


 
 
0 Karma

to4kawa
Ultra Champion

If you have SECCMD do what I'm doing and search in smart mode, you're basically good to go.

0 Karma

to4kawa
Ultra Champion

 

index=_internal | head 1 | fields _raw
| eval _raw="IP=189.41.40.129,Produto=\"Test1\",Valor=179,00,Categoria=Banho,Campanha=1,Vendeu=1,MetododeCompra=1,Bandeira=1,Transportadora=2,Frete=18,57,\"_time\"2021/01/25 19:20:37.374\"
IP=201.0.205.197,Produto=\"Test2\",Valor=123,98,Categoria=Jogos,Campanha=1,Vendeu=0,MetododeCompra=0,Bandeira=0,Transportadora=5,Frete=12,58,\"_time\"2021/01/25 19:20:38.977\"
IP=187.125.147.178,Produto=\"Teste3\",Valor=139,90,Categoria=Cozinha,Campanha=1,Vendeu=1,MetododeCompra=1,Bandeira=1,Transportadora=3,Frete=14,27,\"_time\"2021/01/25 19:20:38.977\"
IP=187.115.202.233,Produto=\"Test4\",Valor=139,90,Categoria=Cozinha,Campanha=1,Vendeu=1,MetododeCompra=1,Bandeira=1,Transportadora=2,Frete=14,51,\"_time\"2021/01/25 19:20:39.579\"
IP=187.111.15.221,Produto=\"Test5\",Valor=164,00,Categoria=Banho,Campanha=2,Vendeu=1,MetododeCompra=1,Bandeira=1,Transportadora=1,Frete=16,81,\"_time\"2021/01/25 19:20:40.580\""
| multikv noheader=t
| fields _raw
| rex mode=sed "s/\"_time\"/Time=/ s/=(\d.*?)(?=,[A-Z])/=\"\1\"/g"
| rex "(?<time>\d{4}/.*)"
| eval _time=strptime(time,"%Y/%m/%d %T.%3N")
| kv

 

0 Karma

leandromatperei
Path Finder

Hi @to4kawa 

This is in the query and how can I make this definition directly in sourcetype? I have a thousand line file that follows this same pattern.

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...