Splunk Search

Modifying multi line event before indexing

mmaier_splunk
Splunk Employee
Splunk Employee

Hello,

i have an application that has an bug in the logging, but i need to workaround it.

log structure:

Dec  10 13:21:09 abc: request:
Session: ******
User-Agent: ********
Content-Length:     ****
Content-Type: *********

positionDec  10 13:22:09 abc: reply:
Session: ********
Date: 2014-12-09T14:33:09Z
Range: *****
Scale: ****
Content-Type: ****
Content-Length: ***

position: ******

this are two events. in the request event it writes at the and in the beginning of the replay message "position:"

i tried already with seed to remove the "position:" - but it is valid in the replay event and it would remove this one as well.

i guess i need to do it via transforms.conf as it needs to be done before we check for the timestamp, otherwise the full line will be used to the event to detect the timestamp.

i tried to add via transforms a line break, but did not work.

[position-fix]
REGEX = (?m)^(.*)position.*
FORMAT = $1\n position1$2
DEST_KEY = _raw

thanks a lot for any advice.

Tags (1)
0 Karma

richgalloway
SplunkTrust
SplunkTrust

The following should remove the stray "position"

REGEX = (.*)position(?=[^:])(.*)
FORMAT = $1\n$2
DEST_KEY = _raw
---
If this reply helps you, Karma would be appreciated.
0 Karma

lrudolph
Path Finder

Hi richgalloway,

your suggestion doesn't work 😞

props.conf:

[abc_xyzprovider]
BREAK_ONLY_BEFORE = abc:
MAX_TIMESTAMP_LOOKAHEAD = 15
NO_BINARY_CHECK = 1
TIME_FORMAT = %b %d %H:%M:%S
TRANSFORMS-positionfix = position-fix
TZ = UTC
pulldown_type = 1 

transforms.conf:

[position-fix]
REGEX = (.)position(?=[^:])(.)
FORMAT = $1\n$2
DEST_KEY = _raw

Still the "position" element (which belongs to the last event) is shown in the next event.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Your REGEX line is not the same as mine. The asterisks are critical. If you want "position" to remain with the first event, try this regex:

(.*position)(?=[^:])(.*)
---
If this reply helps you, Karma would be appreciated.
0 Karma

lrudolph
Path Finder

Sorry, copy & paste must have missed something. But also with correct regex the result is:

\nDec  9 14:33:09 abc: reply:

Sideeffect is also that everything that comes after this first line from the event is somehow deleted.

0 Karma
Get Updates on the Splunk Community!

Leveraging Automated Threat Analysis Across the Splunk Ecosystem

Enhance Security Operations with Automated Threat Analysis in the Splunk EcosystemAre you leveraging ...

Splunk Developers: Go Beyond the Dashboard with These .Conf25 Sessions

  Whether you’re building custom apps, diving into SPL2, or integrating AI and machine learning into your ...

Index This | How do you write 23 only using the number 2?

July 2025 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with this month’s ...