Define identification field

[leandromatperei]

Guys,

I need support, I need to upload these files and process this data, but I need them to be indexed by the _time field, I was unable to use the standard type of pre-defined data. Can you help me?

Another detail, the Valor and Frete fields have commas, and refer to currency, there is a way to treat this data before indexing, so that this data is treated as "." for example ?

Example file: https://www.dropbox.com/s/exh7g1glumxcetr/log.txt?dl=0

[to4kawa]

How about SEDCMD with props.conf.

[leandromatperei]

@to4kawa 

I defined it that way within the sourcetype and it almost went, I just couldn't get the same match for some fields: Category and Shipping

[ <SOURCETYPE NAME> ]
SHOULD_LINEMERGE=true
LINE_BREAKER=([\r\n]+)
NO_BINARY_CHECK=true
CHARSET=AUTO
disabled=false
TIME_FORMAT=%Y/%m/%d  %H.%.M:%.S.%3N
TIME_PREFIX=_time
SEDCMD=s/=(\d.*?)(?=,[A-Z])/="\1"/g

How can I leave these fields the same way?

[to4kawa · Accepted solution]

If you have SECCMD do what I'm doing and search in smart mode, you're basically good to go.

[to4kawa]

index=_internal | head 1 | fields _raw
| eval _raw="IP=189.41.40.129,Produto=\"Test1\",Valor=179,00,Categoria=Banho,Campanha=1,Vendeu=1,MetododeCompra=1,Bandeira=1,Transportadora=2,Frete=18,57,\"_time\"2021/01/25 19:20:37.374\"
IP=201.0.205.197,Produto=\"Test2\",Valor=123,98,Categoria=Jogos,Campanha=1,Vendeu=0,MetododeCompra=0,Bandeira=0,Transportadora=5,Frete=12,58,\"_time\"2021/01/25 19:20:38.977\"
IP=187.125.147.178,Produto=\"Teste3\",Valor=139,90,Categoria=Cozinha,Campanha=1,Vendeu=1,MetododeCompra=1,Bandeira=1,Transportadora=3,Frete=14,27,\"_time\"2021/01/25 19:20:38.977\"
IP=187.115.202.233,Produto=\"Test4\",Valor=139,90,Categoria=Cozinha,Campanha=1,Vendeu=1,MetododeCompra=1,Bandeira=1,Transportadora=2,Frete=14,51,\"_time\"2021/01/25 19:20:39.579\"
IP=187.111.15.221,Produto=\"Test5\",Valor=164,00,Categoria=Banho,Campanha=2,Vendeu=1,MetododeCompra=1,Bandeira=1,Transportadora=1,Frete=16,81,\"_time\"2021/01/25 19:20:40.580\""
| multikv noheader=t
| fields _raw
| rex mode=sed "s/\"_time\"/Time=/ s/=(\d.*?)(?=,[A-Z])/=\"\1\"/g"
| rex "(?<time>\d{4}/.*)"
| eval _time=strptime(time,"%Y/%m/%d %T.%3N")
| kv

[leandromatperei]

Hi @to4kawa 

This is in the query and how can I make this definition directly in sourcetype? I have a thousand line file that follows this same pattern.