topic Re: Using sedcmd to truncate QPM= in Getting Data In

Using sedcmd to truncate QPM=

eoc — Mon, 09 Sep 2019 06:55:20 GMT

Hi Everyone,

Wondering if anyone has a solution to an issue I'm having truncating out some values we deem to be "junk".

We have Splunk indexing logs from AD security and I have the below sedcmd configured:

SEDCMD-shorternQPM = s/(.*QPM.*).*//g

Post restarting the indexer service I couldn't see any noticeable difference in the output.

Below is a snippet of the line where QPM exists:

QPM=

Re: Using sedcmd to truncate QPM=

richgalloway — Mon, 09 Sep 2019 12:10:35 GMT

You didn't give us much to work with.

The SEDCMD string is expecting a single character before "QPM", but the sample data has no such character. That's one possible explanation for the failure.

Re: Using sedcmd to truncate QPM=

somesoni2 — Mon, 09 Sep 2019 13:39:42 GMT

Are you looking to truncate part of your raw data or just drop the whole event itself (no indexing)? If it's the latter, I would suggest reading/implementing this:

https://docs.splunk.com/Documentation/Splunk/7.3.1/Forwarding/Routeandfilterdatad#Filter_event_data_and_send_to_queues

Your sample data and configuration entries will get truncated if you do not format them using "100010" button on top of the text editor in this page (or select and press Ctrl+K).

Re: Using sedcmd to truncate QPM=

eoc — Mon, 09 Sep 2019 23:38:18 GMT

Hi Everyone,

Apologies, appears I hit a character limit.

We need most of the log file and need to omit a small component of it.

An example of a line we are attempting to clean can be seen below:

QPM=<root> <QPM id="the-one"> <public> <Options> <param name="sprPrmLockCount" value="2" /> <param name="sprPrmLockTimestamp" value="11.06.2016 20:36:31" /> <param name="sprPrmLocked" value="no" /> <param name="Layout" value="M|M|M|M|M" /> <param name="sprAnswersHashed" value="no" /> <param name="sprForceEnrollStartDate" /> <param name="sprInvalidQAProfile" value="no" /> <param name="sprLID" value="en" /> <param name="sprShortestAnswerSize" value="5" /> </Options> </public>

Any help is much appreciated

Re: Using sedcmd to truncate QPM=

richgalloway — Tue, 10 Sep 2019 12:59:27 GMT

What part do you need to omit?

Re: Using sedcmd to truncate QPM=

eoc — Mon, 16 Sep 2019 04:31:02 GMT

All of it from QPM to the end of the line

Re: Using sedcmd to truncate QPM=

richgalloway — Tue, 17 Sep 2019 12:55:15 GMT

You don't say which "QPM", but (QPM.*) should match everything from the first one.