I have a single very huge file with different formats. So I decided to create 3 different sourcetypes for this single file. I tried the below, but I did not succeed. Can any one point out where am I lagging?
disabled = false
followTail = 0
index = main
sourcetype = sourcetype1name
I had the problem of different time formats in the same file. I used a script to parse the data out into separate files and ingest those, because it wasn't easy to correct the situation at the source. I was doing this on a heavy forwarder.
here was my script: (with edits for understanding)
What does your infrastructure look like? Are you reading that file from a universal forwarder? Where did you place your props/transforms. They should be on some server that does parsing in your environment, typically an indexer or heavy forwarder.
And of course, are you sure the regex is correct?
If all of that looks good, what if you try placing the TRANSFORMS-myfileformats key into a [sourcetype1name] stanza since you are setting that explicitly on your inputs anyway.