How to use regex in eventtypes.conf

[lakromani]

I have data in following formats:

Nov 04 21:47:59 server1 gtu[22038]: 2833CA0D c   (master)   1A 0B 81 2D 5F 66 36 A7 DC F3 60 B0 
Nov 04 21:47:59 server1 gtu[22038]: 2833CA0D c   (master)   02 6D A0 3C B1 B3 59 CD EC BC CB 7B 55 65 85 CA 
Nov 04 21:47:59 server1 gtu[22038]: 2833CA0D c   (master)   82 70 29 01 02 06 02 BE 04 A5 FB 6C 1F 90 1D 40 
Nov 04 21:47:58 server1 gtu[22038]: 2833CA0D c   (master)   7E A0 51 E5 B2 CA 

I need to set this as one eventtype.
Number of data field can go from 2 to 16.
With normal search, I can use this format:

* | regex _raw="gtu.* \(master\)\s+\w\w\s+\w\w" 

But in eventypes.conf this does not work.

[gtu-master-data]
search = regex _raw="gtu.* \(master\)\s+\w\w\s+\w\w" 

Does regex not work in *eventypes.conf

[aljohnson_splun]

For any search, you can look at the job inspector (Job > Inspect Job) and find the row "canBeEventType" which will be set to either 0 (cannot) or 1 (can).

[woodcock]

You need to create a field-extraction that qualifies/classifies the events (like number_of_bytes) and then create an eventtype based on that (like number_of_bytes = 2).

[alacercogitatus]

Pipes, and other non-streaming commands are not allowed in event types. The search definition must contain only the basics of a simple search - no pipes, no transactions, not regex, nothing other than the "base search" that will match your event type.

So your eventtype in this case will be the following, since this will only include things that match the search definition.

eventtypes.conf

[gtu-master-data]
 search = gtu master master_hex=*

props.conf

 [your_sourcetype]
 EXTRACT-hex_values = gtu\[(?<process_pid>[^\]]+)\]:[^\(]+\(master\)\s+(?<master_hex>(?:[a-fA-F0-9\s]{2})+)

UPDATE: so this should work, but I haven't tested it. Basically, in the props.conf you specify the Extraction for "Master Hex" values, (call it what you want), that matches the formation of hex values. Then you use the event type to limit those events that have any value of master hex.

http://docs.splunk.com/Documentation/Splunk/6.0/Knowledge/Abouteventtypes
Here's the restriction documentation (from below comment) http://docs.splunk.com/Documentation/Splunk/6.3.1/Knowledge/defineeventtypes#Important_event_type_de...

[lakromani]

Problem is that doing this will hit lots of other stuff, that is already to tagged.
So I need to differentiate this data from other stuff.

[woodcock]

@alacercogitatus; I did read that link before I posted my comment and you will note that the documentation does not mention this limitation (which I was pretty sure existed, since that was always how I wrote them). The documentation definitely needs an update to call this out.

[alacercogitatus]

The documentation in the Knowledge Manager Manual does have the restrictions in place. I agree, the spec should be updated to include the specific line.

 You cannot base an event type on a search that includes a pipe operator or a subsearch .

  In addition, you cannot base an event type on a search that references a report. For example, if you have a report with the name failed_login_search, you can't create an event type that is defined by savedsearch=failed_login_search. In a case like this you should always give the event type the same search string as the report. 

http://docs.splunk.com/Documentation/Splunk/6.3.1/Knowledge/defineeventtypes#Important_event_type_de...

[clorne]

Hello,
There is some examples in this post that may help you:
https://answers.splunk.com/answers/293531/how-to-write-the-regex-for-transformsconf-to-extra.html

Regards