Hello,
I have a folder where I have different types of files in it and want to monitor the whole folder as one sourcetype with different props.conf
inputs.conf
[monitor:///mydata/my_folder/ToSplunk/*.(mylogfile|edi.mylogfile|edi)]
index = xyz
_TCP_ROUTING = dev_indexers,qa_indexers
sourcetype = XYZ_SRCTYPE
crcSalt = <SOURCE>
Props.conf
[XYZ_SRCTYPE]
SHOULD_LINEMERGE=false
LINE_BREAKER=(\~|\r\n)ST\*834\*
NO_BINARY_CHECK=true
TRUNCATE=999999
CHARSET=UTF-8
priority = 1
As I said I have different files, I wrote different props.conf for specific log structure to break the events.
[source::/mysource/ToSplunk/*.xml.*.edi]
SHOULD_LINEMERGE=false
LINE_BREAKER=([\r\n\s]+)\<Policy\>[\r\n\s]+
NO_BINARY_CHECK=true
TRUNCATE=999999
CHARSET=UTF-8
priority = 5
[source::/mysource/ToSplunk/*.COMPARE.xml.*.edi]
SHOULD_LINEMERGE=false
LINE_BREAKER=([\r\n\s]+)\<CompareMissing\>[\r\n\s]+
NO_BINARY_CHECK=true
TRUNCATE=999999
CHARSET=UTF-8
priority = 6
[source::/mysource/ToSplunk/*.SBS*.xml.edi]
SHOULD_LINEMERGE=false
LINE_BREAKER=([\r\n\s])+\<Policy\s+
NO_BINARY_CHECK=true
TRUNCATE=999999
CHARSET=UTF-8
priority = 7
[source::/mysource/ToSplunk/*.RCNO*.P.OUT.*]
SHOULD_LINEMERGE=true
LINE_BREAKER=([\r\n]+)
NO_BINARY_CHECK=true
TRUNCATE=999999
CHARSET=UTF-8
priority = 8
The linebreaking in first stanza declared for the sourcetype is working fine, but none of the stanzas for [souce://] are breaking the events correctly
pipes ( OR ) in the monitoring stanza doesn't works. so i had used blacklist & whitelist concept to work it out.
[monitor:///mydata/my_folder/ToSplunk/*.]
index = xyz
_TCP_ROUTING = dev_indexers,qa_indexers
sourcetype = XYZ_SRCTYPE
crcSalt = <SOURCE>
blacklist = (xml.*.edi|COMPARE.xml.*.edi|RCNO*.P.OUT.*|SBS*.xml.edi)
With this is will blacklist the source and i wrote separate stanza for backlisted sources and it worked.
@Naa_Win , we do have an EDI solutions accelerator. Love to connect and give you some dump on the solution. Let me know if you are interested.
Sure ! Thanks. Let me know how we can do this
My email is youngc@splunk.com
Please send me an email, I will send a zoom meeting invite for us to connect.
pipes ( OR ) in the monitoring stanza doesn't works. so i had used blacklist & whitelist concept to work it out.
[monitor:///mydata/my_folder/ToSplunk/*.]
index = xyz
_TCP_ROUTING = dev_indexers,qa_indexers
sourcetype = XYZ_SRCTYPE
crcSalt = <SOURCE>
blacklist = (xml.*.edi|COMPARE.xml.*.edi|RCNO*.P.OUT.*|SBS*.xml.edi)
With this is will blacklist the source and i wrote separate stanza for backlisted sources and it worked.