Splunk Enterprise

Define identification field

leandromatperei
Path Finder

Guys,

I need support, I need to upload these files and process this data, but I need them to be indexed by the _time field, I was unable to use the standard type of pre-defined data. Can you help me?

Another detail, the Valor and Frete fields have commas, and refer to currency, there is a way to treat this data before indexing, so that this data is treated as "." for example ?

Example file: https://www.dropbox.com/s/exh7g1glumxcetr/log.txt?dl=0

Labels (2)
Tags (1)
0 Karma
1 Solution

to4kawa
Ultra Champion

If you have SECCMD do what I'm doing and search in smart mode, you're basically good to go.

View solution in original post

0 Karma

to4kawa
Ultra Champion

How about SEDCMD with props.conf.

0 Karma

leandromatperei
Path Finder

@to4kawa 

I defined it that way within the sourcetype and it almost went, I just couldn't get the same match for some fields: Category and Shipping

 

[ <SOURCETYPE NAME> ]
SHOULD_LINEMERGE=true
LINE_BREAKER=([\r\n]+)
NO_BINARY_CHECK=true
CHARSET=AUTO
disabled=false
TIME_FORMAT=%Y/%m/%d  %H.%.M:%.S.%3N
TIME_PREFIX=_time
SEDCMD=s/=(\d.*?)(?=,[A-Z])/="\1"/g

 

 
 
How can I leave these fields the same way?


 
 
0 Karma

to4kawa
Ultra Champion

If you have SECCMD do what I'm doing and search in smart mode, you're basically good to go.

0 Karma

to4kawa
Ultra Champion

 

index=_internal | head 1 | fields _raw
| eval _raw="IP=189.41.40.129,Produto=\"Test1\",Valor=179,00,Categoria=Banho,Campanha=1,Vendeu=1,MetododeCompra=1,Bandeira=1,Transportadora=2,Frete=18,57,\"_time\"2021/01/25 19:20:37.374\"
IP=201.0.205.197,Produto=\"Test2\",Valor=123,98,Categoria=Jogos,Campanha=1,Vendeu=0,MetododeCompra=0,Bandeira=0,Transportadora=5,Frete=12,58,\"_time\"2021/01/25 19:20:38.977\"
IP=187.125.147.178,Produto=\"Teste3\",Valor=139,90,Categoria=Cozinha,Campanha=1,Vendeu=1,MetododeCompra=1,Bandeira=1,Transportadora=3,Frete=14,27,\"_time\"2021/01/25 19:20:38.977\"
IP=187.115.202.233,Produto=\"Test4\",Valor=139,90,Categoria=Cozinha,Campanha=1,Vendeu=1,MetododeCompra=1,Bandeira=1,Transportadora=2,Frete=14,51,\"_time\"2021/01/25 19:20:39.579\"
IP=187.111.15.221,Produto=\"Test5\",Valor=164,00,Categoria=Banho,Campanha=2,Vendeu=1,MetododeCompra=1,Bandeira=1,Transportadora=1,Frete=16,81,\"_time\"2021/01/25 19:20:40.580\""
| multikv noheader=t
| fields _raw
| rex mode=sed "s/\"_time\"/Time=/ s/=(\d.*?)(?=,[A-Z])/=\"\1\"/g"
| rex "(?<time>\d{4}/.*)"
| eval _time=strptime(time,"%Y/%m/%d %T.%3N")
| kv

 

0 Karma

leandromatperei
Path Finder

Hi @to4kawa 

This is in the query and how can I make this definition directly in sourcetype? I have a thousand line file that follows this same pattern.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...