Consider i have a directory like :
/mydir/file1.log
/mydir/file.2.log
/mydir/message_1234.trc
Now i want to end up with choosen sourcetypes and souce overrriding for getting rid of the 1234.
When I configured this according the docs it doenst work, the source is not overridden.
I am not sure if this is the best approach and if this will work due the loop in destkey :
inputs.conf
[monitor:///mydir]
props.conf
[source::.../file1.log]
sourcetype = mysourcetype1
[source::.../file2.log]
sourcetype = mysourcetype2
[source::.../*.trc]
sourcetype = temp_sourcetype
[temp_sourcetype]
TRANSFORMS-source_fix = sourceoverride
[mysourcetype3]
TRANSFORMS-fix_source = source_clean_roll_1
tranforms.conf
[temp_sourcetype]
DEST_KEY = MetaData:Sourcetype
SOURCE_KEY = MetaData:Source
REGEX = \d+\.trc
FORMAT = sourcetype::mysourcetype3
[source_clean_roll_1]
DEST_KEY = MetaData:Source
SOURCE_KEY = MetaData:Source
REGEX = (.*)(message_)(\d+)(\.trc)
FORMAT = source::$1$2$4
I am indeed checking this but thats a bit difficult :
/mydir/message_1234.trc
it would be handy if
/mydir/message_\d+\.trc
was supported but that becomes a whitelist and that will exclude the *.log so it looks like I am in a lock
Hi Starlette,
It seems you have a bad reference between props and transforms. Your first transforms stanza should be named [sourceoverride]
instead of [temp_sourcetype]
.
UPDATE: After looking a little more closely at what your trying to do; I'm not sure that you can parse your data twice with props -> transforms -> props -> transforms
. I think, and I may be wrong here, that this transforms operations can only occur once, i.e. data comes in, goes through props, pops over to transforms, and goes into the index. You cannot go back for another round of parsing.
UPDATE 2: Well, going out on a limb here. I have not tried this configuration - but it looks like it could work. Sorry if it doesn't.
Wouldn't it be possible do some of the work in inputs.conf, i.e.
inputs.conf
[monitor:///mydir/*.trc]
sourcetype=blahblah
props.conf
[blahblah]
TRANSFORMS-asdf = trc_resourcing
transforms.conf
[trc_resourcing]
DEST_KEY=MetaData:Source
SOURCE_KEY=MetaData:Source
REGEX = (.*)(message_)(\d+)(\.trc)
FORMAT = source::$1$2$4
hope this helps,
Kristian
I am indeed checking this but thats a bit difficult :
/mydir/message_1234.trc
it would be handy if
/mydir/message_\d+.trc
was supported but that becomes a whitelist and that will exclude the *.log so it looks like I am in a lock
Sorry - do you want a different sourcetype for each .trc-file? Hmm, yes. That's what you were trying to do in the first transform...
Will there be a unique sourcetype per file, or are there say a dozen different sourctypes you want the trc-files to be? Could you hardcode it in separate monitor stanzas?
Hi Kristian,...
Then i ended up with 1 sourcetype, ( for more *.trc's)
So thats not ideal as well...thanks
yes, thats what was thinking as well, then it might be not possible to handle this...