Getting Data In
Highlighted

Inputs.conf help

New Member

I am trying to onboard ingest about 30 different log type from a single Source (Linux Server)

Currently the logs are being written and zipped by rsyslog in a particular folder (structure below):
/mnt/log/files/YYYY/MM/DD/hostname/filename.log.gz

What would be the best way to ingest the different logtypes

I was thinking about the following:

[monitor:///mnt/log/files/////]
host_segment=7
whitelist=(.+\access.gz)
index=webserver
sourcetype=access:apache

Not sure why the asterisks are not showing up on the preview;
it should be /mnt/log/files/aster/aster/aster/aster/

I was also thinking to ingest all 30 log types I would need to create 30 different configs?

0 Karma
Highlighted

Re: Inputs.conf help

Legend

If the different log types are going to different indexes, then yes, you will need 30 different configs. But if the only difference will be the sourcetype, you could do an override in props.conf. You still have to configure all the different configs, but it might be easier to do. Take a look at this:

inputs.conf
[monitor:///mnt/log/files/]
whitelist=.gz$
host_segment=7
index=webserver

props.conf
[source::/mnt/log/files/.../access.gz]
sourcetype=access:apache

[source::/mnt/log/files/.../otherthing.gz]
sourcetype=st_otherthing

etc.

0 Karma
Highlighted

Re: Inputs.conf help

New Member

It would most probably be about 4-5 different indexes
Best way to do this would be with the inputs as so..
inputs.conf
[monitor:///mnt/log/files/]
whitelist=.gz$
host_segment=7
index=webserver

[monitor:///mnt/log/files/]
whitelist=.gz$
host_segment=7
index=webapp

All 30 log types would be defined as separate sourcetype
so best way do to this would be with the props.conf as you outlined?
props.conf
[source::/mnt/log/files/.../access.gz]
sourcetype=access:apache

[source::/mnt/log/files/.../error.gz]
sourcetype=st_otherthing

0 Karma
Highlighted

Re: Inputs.conf help

New Member

Also what happens to a gz file that I am not aware of? will that get indexed? and will not have a pre-defined sourcetype?

0 Karma