Getting Data In
Highlighted

Multi line log issue

Communicator

I have a Windows system with 4.1.5 forwarding to my Splunk indexer, that puts out logs in this format:

begin error

lines of interesting log entries

end error

I've been noodling around with different options within the PROPS.CONF on the Forwarder system. So far no luck. My goal is to be have the forwarder sear the data correctly then transfer to the Indexing server.

Any tips or ideas I'm missing?

Tags (1)
0 Karma
Highlighted

Re: Multi line log issue

Motivator

So do you want multiline events or not? Can't quite tell from your question. Can you please clarify?

0 Karma
Highlighted

Re: Multi line log issue

Motivator

Are these Windows Event Log events or text-based? What do you want the indexed events to look like?

If all events follow the format you describe, then it should be enough to do:

#props.conf
[yoursourcetype]
LINE_BREAKER=([\r\n]+)(### begin error)

Not sure if the hash marks would need to be escaped.

View solution in original post

Highlighted

Re: Multi line log issue

Communicator

The logs I'm going after are not Event logs, they're output from a custom program. I tried as you noted and no luck. The first event shows the header line (===== begin error =====) and the next line from the file (ASP error on page: http://server/page.asp).
The next event begins with the 3rd line of text (At 10/25/2010 3:55:09pm) and shows the remaining lines of text up to the last line (===== end error =====).

By the way, I tested the = sign in REGEX Buddy and it recognized the = sign as a character.

Any other ideas?

0 Karma
Highlighted

Re: Multi line log issue

Communicator

A question - in my setup the forwarder system has the props.conf with the LINE_BREAKER entry in it. Should this props.conf be moved to the indexing server or left on the forwarding system?

0 Karma
Highlighted

Re: Multi line log issue

Motivator

Equals signs would not need to be escaped. In your original question, you had hashes, which might have been interpreted as the beginning of a comment in the config file. If you are using a lightweight forwarder, then do this at the indexer. For heavy-weight forwarders, do it at the forwarder.

0 Karma
Highlighted

Re: Multi line log issue

Communicator

Sorry for the posting glitch, my bad with the # versus = sign. In this case the forwarder being used is the light forwarder. Thanks for the tip, I'll make the entry in the indexer's props.conf and see how that works.

0 Karma
Highlighted

Re: Multi line log issue

Communicator

Hey SoutheringtonP,

I changed the client from running LightForwarder to Forwarder. Your change to the props.conf worked perfect! Thanks for the insight!!

0 Karma