Getting Data In

Search-time Mask

sdwilkerson
Contributor

Some events flow into the Splunk instance via syslog sockets.

For a brief period of time, the sourcetypes that came in via syslog were prepended with the standard syslog code like <142> (so the line would read <142>May 6 ...). This was only for a few days.

Is there a way I can do a built-in search-time extraction to mask the <\d+> prepended to these sourcetypes so that 1. It is not seen and 2. All of the rest of the props/transforms extractions will work as if it wasn't there?

Note: I do not mean to use something like rex in the search-command, but rather, make the change in the splunk configs to mask these characters from any search results.

Sean

Tags (2)

gkanapathy
Splunk Employee
Splunk Employee

I recommend against it, but you could do:

[mysourcetype]
EXTRACT-out = (?s)^(?:\<\d+\>)?(?<altraw>.*)
FIELDALIAS-raw = altraw as _raw

Update: Oh, but FIELDALIAS runs after all other extractions, so no the other extractions won't work as if it wasn't there. And as you've no doubt found, you can't overwrite _raw with an EXTRACT or REPORT.

gkanapathy
Splunk Employee
Splunk Employee

You can use the | delete command to prevent specific events from being returned in search. (Note it does not reclaim disk space or remove the data from the disk, but simply makes it unretrieveable via Splunk.) Yes, then you can edit the raw data (or set up a TRANSFORM or SEDCMD to edit the data as is comes in and reindex.

0 Karma

sdwilkerson
Contributor

If sensitive info were accidentally indexed (e.g. PII like SSN#) as part of other logdata that you didn't want any Splunk users to see, would the only solution be to delete, edit(raw) and then reIndex?

Thanks,
Sean

0 Karma
Get Updates on the Splunk Community!

Get ready to show some Splunk Certification swagger at .conf24!

Dive into the deep end of data by earning a Splunk Certification at .conf24. We're enticing you again this ...

Built-in Service Level Objectives Management to Bridge the Gap Between Service & ...

Now On-Demand Join us to learn more about how you can leverage Service Level Objectives (SLOs) and the new ...

Database Performance Sidebar Panel Now on APM Database Query Performance & Service ...

We’ve streamlined the troubleshooting experience for database-related service issues by adding a database ...