How do you build a regex to index only specific files?

[sarit_s]

Hello,

i have these 3 stanzas in my transforms.conf file:

[set_f270_header]
REGEX = (^\$\w+\s\d+|^\-\-\-\-\-  header)
FORMAT = sourcetype::f270_header
DEST_KEY = MetaData:Sourcetype

[set_f270_system]
REGEX = (^\w{3}\s+\d+\s\d{2}|^\-\-\-\-\-  System Log)
FORMAT = sourcetype::f270_system
DEST_KEY = MetaData:Sourcetype

[set_f270_joblog]
REGEX = (^\$\w+\s\d+|^\-\-\-\-\-  joblog)
FORMAT = sourcetype::f270_joblog
DEST_KEY = MetaData:Sourcetype

my files names are for example:

037388b4-0f12-410e-a8ab-a795e9244e22.sanitized.joblog

130dab3c-3e62-45a0-aefe-f160c0dd3325_header

73dc67bc-db07-49d5-a12c-a1ed12f54fee_System+Log

Beside them, i have more file types, but I don't want to index them right now.

My problem is that the files are not indexed correctly and I got all the file types in my sourcetype

What am I doing wrong ?

[nickhills · Accepted solution]

It seems to me that you would be better configuring this as 3 stanzas in your inputs.conf, rather than props/transforms.

[monitor://your/path/*_header]
sourcetype=f270_header

[monitor://your/path/*_System+Log]
sourcetype=f270_system

[monitor://your/path/*joblog]
sourcetype=f270_job

[sarit_s]

solved by doing something similar in s3 inputs.conf
add it as whitelist

[sarit_s]

my inputs are from s3
i tries this but it didnt work