Splunk Enterprise

REGEX filter in transforms.conf file setting question

erictodor
New Member

We're forwarding events to a 3rd party. In our transforms.conf file, the filter looks like the following

REGEX = .

For some reason, this filter capture names without any hyphens. Here's what I'm talking about

Success - Computer
Failure - Co-m-puter

We have computer names with the '-' in them but they don't get captured. Is there a better wild card string that can be used to capture all computer names, regardless of what characters are in them.

Thanks!

Tags (1)
0 Karma

richgalloway
SplunkTrust
SplunkTrust

If you share some sample data and your transforms.conf settings we can help better.

---
If this reply helps you, Karma would be appreciated.
0 Karma

erictodor
New Member

Here's some additional info. Let me know if this is helpful.

Thank you

Props.conf

[host::*]
TRANSFORMS-main = test_eventlogs

Transforms.conf

[test_eventlogs]
REGEX = .
DEST_KEY = _SYSLOG_ROUTING
FORMAT = windowsworkstationsmain

Outputs.conf

[syslog:windowsworkstationsmain]
server = my.destination.com
sendCookedData = true

0 Karma

micahkemp
Champion

That configuration tells Splunk where to send the data (in this case by syslog to my.destination.com). That is uninvolved in the parsing of the data.

Can you include props.conf (from the search head, not the forwarder) for the sourcetype in question?

0 Karma

erictodor
New Member

This is from the search head.

0 Karma

erictodor
New Member

This is the complete props.conf

[set_sourcetype_to_stash]
REGEX = .
DEST_KEY = MetaData:Sourcetype
FORMAT = sourcetype::stash

[extract_spent]
REGEX = (?P\d+)ms$

[eliminate-eventcodes]
REGEX = EventCode=(0000)
DEST_KEY = queue
FORMAT = nullQueue

[wstn_eventlogs]
REGEX = .
DEST_KEY = _SYSLOG_ROUTING
FORMAT = windowsworkstationsmain

0 Karma

richgalloway
SplunkTrust
SplunkTrust

The REGEX string "." matches any single character, which is probably not what you want.
If you share some sample data and let us know what you want extracted from it we can probably help create a working regex.

---
If this reply helps you, Karma would be appreciated.
0 Karma

erictodor
New Member

Basically, we have computer names that are whole words without any non-characters in the name (i.e. Computer01) and we have names with non-characters in them (i.e. Computer-01). It looks like the filter only matches the 1st, names with only characters and numbers. I need to capture all names, including the ones with '-' in them.

Any help would be appreciated.

0 Karma

micahkemp
Champion

The issue is we still haven't seen the regex that is matching computer names. None of the transforms you posted above show that extraction.

Can you show the props.conf configuration for the sourcetype of this data that is being improperly extracted?

0 Karma

erictodor
New Member

This is what I have. I'm assuming the text below represents the computer name filter.

--Eric

[WinEventLog:Security]
TRANSFORMS-t1 = eliminate-eventcodes

[CSC_FileMon]
DATETIME_CONFIG = CURRENT
NO_BINARY_CHECK = 1
SHOULD_LINEMERGE = false
pulldown_type = 1

[host::*]
TRANSFORMS-main = wstn_eventlogs

0 Karma

micahkemp
Champion

When you search that data in Splunk, what does it show is the sourcetype for the events with the mis-parsed computer name? If it's CSC_FileMon then the above shows the props for it, but there is no extraction defined, which means Splunk may be grabbing the computer name via KV_MODE = auto (which looks for key=value in the events).

Please include some sample events that have computer name, both where it is properly extracted and where it is not.

0 Karma

erictodor
New Member

I'm sorry, the limits of my Splunk knowledge is being hit by your question. I don't know how to query Splunk to see the output of the data stream to the 3rd party. I'm looking at 3rd party tool itself and looking at the data coming in. By looking at that data, I can see the computer names that are being matched. The pattern I've discerned is what I've posted before about the computer names. Those with letters and numbers are fine. Anything with something else in it doesn't reach the 3rd party.

Is there a way I can write a query to see the output to compare and provide the data you're looking for ?

(and thank you for your patience with me on this question)

0 Karma

micahkemp
Champion

So the question here is why is a 3rd party tool not parsing the logs correctly?

Your config above looks correct to forward logs, and doesn't do any other parsing or formatting. If a 3rd party tool isn't parsing as you'd expect that sounds like an issue with that other tool.

0 Karma

erictodor
New Member

That's a possibility. When I'm looking at the 3rd party feed, I'm assuming I'm looking at the feed before it's parsed and when I look at the raw data, I don't see all computer names. Is there a way to mimic this props.conf filter in a Splunk query so I can see what is being captured?

I guess the question is, IS Splunk sending entries from all hosts or is it sending a sub-set of hosts? I can't verify this from the Splunk side (yet?). If I can verify that it is, I can then focus on the 3rd party for not parsing the entries correctly but right now I don't know if the data out is good.

I hope that makes sense.

0 Karma

micahkemp
Champion

Splunk's concept of host is the machine that created or sent the logs. Your configuration tells Splunk to forward all of those hosts' logs to this other tool.

The other concept of host you appear to be using is the computer name referenced in the log, and there is no pattern matching against that based on what you have posted from your configs. The REGEX = . line means "match everything, seriously."

0 Karma

erictodor
New Member

Thank you for the distinction in definitions. Helps with explaining the issue more. This configuration is what we want Splunk to do. To forward all logs to our 3rd party. In these logs are computer names (my definition of host I was using). While the REGEX = . means match everything, is there any way to double verify that everything, is indeed, being forwarded?

0 Karma

micahkemp
Champion

REGEX = . isn't capturing names at all. It's not capturing anything, and it matches anything. I think you're looking at the wrong transform.

0 Karma
Get Updates on the Splunk Community!

Take Your Breath Away with Splunk Risk-Based Alerting (RBA)

WATCH NOW!The Splunk Guide to Risk-Based Alerting is here to empower your SOC like never before. Join Haylee ...

SignalFlow: What? Why? How?

What is SignalFlow? Splunk Observability Cloud’s analytics engine, SignalFlow, opens up a world of in-depth ...

Federated Search for Amazon S3 | Key Use Cases to Streamline Compliance Workflows

Modern business operations are supported by data compliance. As regulations evolve, organizations must ...