Search on non formatted log files

[sushmitha_mj]

I am working on getting some datapower logs into Splunk. The problem is the logs are not in CSV format, there are no delimiters. The data is usually follows certain field pattern, but sometimes it does not. I know if I insert this logs into splunk I can search for particular words like "error" etc in the log and display their count etc , but if I want to do more operations on this log, how do I do them?
I could not find splunk documentation relating to handling such data. If there are please share link.

[jkat54 · Accepted solution]

Splunk is excellent at handling unstructured data. Just add the data and manipulate it with the interactive field extractor (IFX), or rex & erex commands. Biggest thing is you want to get your event's broke into different events, one line per event (usually but not always the case), etc. You do that with LINE_BREAKER, MUST_BREAK_BEFORE, etc. See props.conf documentation (link is provided here: props.conf) and read the event breaking portions.

Once you have the data in splunk it should be easy to use the IFX to pull out fields you want.