How to edit props.conf to line break my multiline event based on timestamp?

[andakun_222]

Hi,

We are trying to break the following lines based on date/timestamp but multiline event is not working as we expected, can some one please us on this issue.

Sample log:

[02.13.2017 15:35:47.920]  INFO - [CargoHub.ShipmentPieceModule.ShipmentPieceServiceImpl.createShipmentPiece] TID[WMQJCAResourceAdapter : 4] SID[sabre:AWBReplication] RID[29004928] [<== cargo.PreLogHandler ==>  Pre Log :Shipmentpiece goods qty count 4]
[02.13.2017 15:35:47.923]  INFO - [CargoHub.AirWaybillSCPSModule] TID[WMQJCAResourceAdapter : 2] SID[sabre:AWBReplication] RID[29004929] [<== com.ibm.bpe.generated.Abstract_PT_ ==> Create Itinerary Process : End]
[02.13.2017 15:35:47.929]  INFO - [CargoHub.ShipmentPieceModule.ShipmentPieceServiceImpl.createShipmentPiece] TID[WMQJCAResourceAdapter : 4] SID[sabre:AWBReplication] RID[29004928] [<== com.ibm.bpe.generated.Abstract_PT_ ==> (ShipmentPiece Module -> ShipmentPieceAggregateServiceImpl -> createPieceAggregate) started]
[02.13.2017 15:35:47.939]  INFO - [CargoHub.ShipmentPieceModule.ShipmentPieceServiceImpl.createShipmentPiece] TID[WMQJCAResourceAdapter : 4] SID[sabre:AWBReplication] RID[29004928] [<== com.ibm.bpe.generated.Abstract_PT_ ==> (ShipmentPiece Module -> ShipmentPieceAggregateServiceImpl -> createPieceAggregate) finished]
[02.13.2017 15:35:47.940]  INFO - [CargoHub.ShipmentPieceModule.ShipmentPieceServiceImpl.createShipmentPiece] TID[WMQJCAResourceAdapter : 4] SID[sabre:AWBReplication] RID[29004928] [<== cargo.PostLogHandler ==>  Post Log : Exit from Process Goods qty count 4]

we want to break the line based on timestamp sample( [02.13.2017 15:35:47.920] )

Please find below the configuration we added at props.conf on server side.

BREAK_ONLY_BEFORE = ^\[\d+\.\d+\.\d+\s+\d+:\d+:\d+\.\d+\]
MAX_TIMESTAMP_LOOKAHEAD = 150
NO_BINARY_CHECK = 1
SHOULD_LINEMERGE = true
TIME_FORMAT = %m.%d.%Y %H:%M:%S.%3Q
pulldown_type = 1

[somesoni2 · Accepted solution]

Give this a try

props.conf (on Indexer/Heavy Forwarder)

[YourSourceType]
SHOULD_LINEMERGE=false
LINE_BREAKER = ([\r\n]+)(?=\[\d+\.\d+\.\d+\s+\d+:\d+:\d+\.\d+\])
TIME_PREFIX = ^\[
TIME_FORMAT = %m.%d.%Y %H:%M:%S.%3Q
MAX_TIMESTAMP_LOOKAHEAD = 23

[dtregonning_spl]

I had some success with. Let me know how you went.

[ __auto__learned__ ]
SHOULD_LINEMERGE=true
NO_BINARY_CHECK=true
LINE_BREAKER=\[\d+\.\d+\.\d+\s+\d+:\d+:\d+\.\d+\]
BREAK_ONLY_BEFORE=\[\d+\.\d+\.\d+\s+\d+:\d+:\d+\.\d+\]
 NO_BINARY_CHECK=1
pulldown_type=1
TIME_FORMAT=%m.%d.%Y %H:%M:%S.%3