Splunk Search

What is the correct REGEX for this?

echojacques
Builder

Hello,

What is the correct REGEX to match the following field and value in all events from any sourcetype:

dest_port=443

I will be adding this to a nullQueue in transforms.conf. I have tried REGEX=dest_port=443 but this does not work (I am very new with REGEX).

Thanks

Tags (1)
0 Karma
1 Solution

kristian_kolb
Ultra Champion

Does the text in the actual raw event contain the string?

dest_port=443

Because if it does not, you are probably referring to a field name that is extracted at search time. nullQueueing takes place during the parsing/indexing phase, and no fields are available then (apart from stuff like host, source etc). Also, make sure that you're editing the correct config files. nullQueueing takes place during the parsing phase, and depending on your setup, that might be on the indexer or on a Heavy Forwarder;

http://wiki.splunk.com/Where_do_I_configure_my_Splunk_settings
http://docs.splunk.com/Documentation/Splunk/6.0/Forwarding/Routeandfilterdatad#Discard_specific_even...

[default]
TRANSFORMS-nullqueue_443= remove443

transforms.conf:

[remove443]
REGEX = dest_port=443
DEST_KEY = queue
FORMAT = nullQueue

But as you may have noticed, it does not seem to work. Please provide a few sample events. Also, I'm not 100% sure that you can put the TRANSFORM in props.conf under the [default] stanza. It's a rather unusual request, and I have never tried it.

If that does not work, (under default) then you might have to put the TRANSFORMS line in each source/sourcetype stanza that may contain data that you want to filter out.

EDIT: Typo in the stanza header in transforms.conf... fixed it.

/K

View solution in original post

kristian_kolb
Ultra Champion

Does the text in the actual raw event contain the string?

dest_port=443

Because if it does not, you are probably referring to a field name that is extracted at search time. nullQueueing takes place during the parsing/indexing phase, and no fields are available then (apart from stuff like host, source etc). Also, make sure that you're editing the correct config files. nullQueueing takes place during the parsing phase, and depending on your setup, that might be on the indexer or on a Heavy Forwarder;

http://wiki.splunk.com/Where_do_I_configure_my_Splunk_settings
http://docs.splunk.com/Documentation/Splunk/6.0/Forwarding/Routeandfilterdatad#Discard_specific_even...

[default]
TRANSFORMS-nullqueue_443= remove443

transforms.conf:

[remove443]
REGEX = dest_port=443
DEST_KEY = queue
FORMAT = nullQueue

But as you may have noticed, it does not seem to work. Please provide a few sample events. Also, I'm not 100% sure that you can put the TRANSFORM in props.conf under the [default] stanza. It's a rather unusual request, and I have never tried it.

If that does not work, (under default) then you might have to put the TRANSFORMS line in each source/sourcetype stanza that may contain data that you want to filter out.

EDIT: Typo in the stanza header in transforms.conf... fixed it.

/K

kristian_kolb
Ultra Champion

Just be aware that this setup may cause you to lose events if 443 turns up anyplace in an event. (in a timestamp, as response time, error codes, byte counts etc).

/k

0 Karma

echojacques
Builder

Perfect, that worked! I found that this also works:

\b/443\b

but i like \D better...

Thanks!

0 Karma

lukejadamec
Super Champion

Try \D for not a digit at the end
REGEX=/443\D

0 Karma

echojacques
Builder

Hi, thanks. You were right- the raw event does not contain the "dest_port=443" string. So I tried with REGEX=/443 and it worked.

But now, how do I tell regex to stop after 443 and not match 4435, 4436, etc. as well so it only matches "443".

Thanks!

0 Karma
Get Updates on the Splunk Community!

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...