Getting Data In

linebreaking issue

ebailey
Communicator

I need some help getting Splunk to line break properly. I have a poorly formatted log file that is pulled from a mainframe so getting the formatting of the message changed would be extremely tough. I was hoping to use Splunk's built-in tools to handle the issue, but I cannot get this to work right at all.

Here is a sample event

20131992359247000|2013|199|235924|7000|7000|xxxx|xxxx|xxxx|xxxxxxx|xxxx|3101600002xx 02153604 |1|06|xx|U |00000000|205|xxxx - xxxxxxxxxxxxxxxxxxxxxx| |

I want to have every line break before

20131992359247000 (this is not a static value - it changes for every events)

but my regex is not working. Any suggestions

Thanks

0 Karma
1 Solution

gkanapathy
Splunk Employee
Splunk Employee
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)20131992359247000

well, if it's not a static value, then you are going to have to say how it differs from every other line. i'm also assuming that you actually have multiple lines and that your problem is that you want to break at the beginning of a particular line that matches some pattern. So the question is, what is that pattern? And the answer to that is your regex. It would be helpful if you described it in words if you can't do so in regex. But let's say that it's just every line that starts with a 17-digit number, and then the pipe. Then it's just:

([\r\n]+)\d{17}\|

Or maybe it's more restrictive, and there are other lines with 17-digit numbers at the start that you don't want to break on. Or maybe the 17-digit number in the middle of the line. You need to say it, not make us guess.

View solution in original post

gkanapathy
Splunk Employee
Splunk Employee
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)20131992359247000

well, if it's not a static value, then you are going to have to say how it differs from every other line. i'm also assuming that you actually have multiple lines and that your problem is that you want to break at the beginning of a particular line that matches some pattern. So the question is, what is that pattern? And the answer to that is your regex. It would be helpful if you described it in words if you can't do so in regex. But let's say that it's just every line that starts with a 17-digit number, and then the pipe. Then it's just:

([\r\n]+)\d{17}\|

Or maybe it's more restrictive, and there are other lines with 17-digit numbers at the start that you don't want to break on. Or maybe the 17-digit number in the middle of the line. You need to say it, not make us guess.

ebailey
Communicator

that is it - i almost had it right - Thanks!

0 Karma

ddarmand
Communicator

use transform.conf to make a field ?

0 Karma

ebailey
Communicator

20131992359247000 is not a static value

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Manage Edge Processor pipelines end to end with APIs

This guide uses the Data Management APIs to create an Edge Processor, onboard one instance, deploy a pipeline ...

Splunk Brings Proof of Control to Digital Asset Transfers

Digital assets now sit inside the core of financial services. HSBC launched its Tokenized Deposit Service in ...

Manage Ingest Processor pipelines end to end with APIs

This guide uses the Data Management APIs to create an Ingest Processor pipeline, deploy it to the ...