file monitor input only ingest last 200 bytes of the file

[season88481]

Hi guys,

I need to on-board some log files, but only the last 200 bytes of each log files contain useful information.

Is there any settings from props.conf that will only ingest the last XX characters of each log file?

Each file only contains a single line of log.

Or is there any other workaround could do the trick?

Many thanks.
S

[PavelP · Accepted solution]

Hello @season88481 ,

there are several possible options, for example SEDCMD or LINE_BREAKER to name a few. For example if your log file looks like this:

STANDARD BLA 12 XXXXXXX Message: connection from A to B

you can use this line in props.conf, anything inside of the first capture group will be removed:

LINE_BREAKER = ([\r\n]+.* Message: )connection

To see what can be done you have to post a couple of examples.

[season88481]

Thanks Pavel,

Yes, i found the same. There is no such setting in props.conf that allows us to only ingest the first/last xx bytes of the log.

I end up using SHOULD_LINEMERGE=false and LINE_BREAKER in props.conf, and filter out the unwanted part by using props.conf and transforms.conf

[to4kawa]

PREAMBLE_REGEX = <regex>
* A regular expression that lets Splunk software ignore "preamble lines",
  or lines that occur before lines that represent structured data.
* When set, Splunk software ignores these preamble lines,
  based on the pattern you specify.
* Default: not set