we use csv to track app's performance. I added the csv to forwarder and keep monitoring it. The problem is that while app is running it keeps writing to the CSV, however, only the few minutes at the beginning will be indexed but not the rest. If I move the whole file, while no more update, to another location, I can get everything indexed. Any suggestions?
blacklist = -\d
index = test
sourcetype = csv
crcSalt = <SOURCE>
Thanks Cusello. I should have noticed the special char being removed when I posted it. It should be "crcSalt = <SOURCE>". I did try with/without that line. Doesn't help.
I looked into that little bit more, and found out that the data seemed to have been indexed, but, only the first few minutes are correct, and at certain point, the time stamp all messed up which being set months back.
I then created a custom csv sourcetype with specifying the timestamp format and field. That seems to have addressed my problem. I was not thinking of doing that because the timestamp being logged doesn't have millisecond, nor a date.