what is the minimum level of logging required for erex to output its regex in the search information icon in splunk?

[mjones414]

I'm trying to use splunk on a search head I don't manage but I noticed that whenever I try to use erex on the search head, the regex never comes back to me. I see logs at the end of my search that erex executed fine:

05-04-2020 21:41:04.191 INFO  DispatchExecutor - END OPEN: Processor=erex
05-04-2020 21:41:04.366 INFO  script - Invoked script erex with 6203295 input bytes (20351 events).  Returned 41 output bytes in 120 ms

but I can't find anywhere in the search.log or in the GUI where the learned regex is being displayed. My guess is that the level of logging required to print this output may be too high. What is the required logging level for erex to function?

[mjones414]

Thanks for this. So it turned out my problem with it showing in the jobs context was actually a limit of what can show up in the job insepector context. The environment has over 50 indexers and if each one throws up 1 thing informational, that is enough to fill up the maximum amount of messages that can show in the jobs menu. If I whiddle down the search in another splunk window, it'll give me enough of a regex to start with. Is there anything else in _internal which may catch the fully generated regex I executed?

[richgalloway]

See my updated answer.

[codebuilder]

Tabling a field that you name explicilty will display the results of what was found, not the actual regex that was used. To find the actual regex that was generated (which is the purpose of erex) you need to look at the job inspector once your search returns.

Example:

[tauliang]

What is your search?

I did a dummy search like this,

index=_internal sourcetype=splunkd_ui_access
| erex METHOD examples="GET , POST"
| table METHOD
| dedup METHOD

and I was able to see logs like this

2020-05-04 17:14:29,457 -0700 INFO FieldLearning:62 - GENERATING RULES FROM 100 EVENTS
2020-05-04 17:16:15,310 -0700 INFO erex:25 - erex run with examples: GET , POST
2020-05-04 17:16:15,411 -0700 INFO FieldLearning:62 - GENERATING RULES FROM 100 EVENTS
2020-05-04 17:16:55,592 -0700 INFO erex:25 - erex run with examples: GET , POST
2020-05-04 17:16:55,744 -0700 INFO FieldLearning:62 - GENERATING RULES FROM 100 EVENTS
2020-05-04 17:17:18,915 -0700 INFO erex:25 - erex run with examples: GET , POST
2020-05-04 17:17:20,037 -0700 INFO FieldLearning:62 - GENERATING RULES FROM 100 EVENTS
2020-05-04 17:17:31,770 -0700 INFO erex:25 - erex run with examples: GET , POST
2020-05-04 17:17:41,178 -0700 INFO FieldLearning:62 - GENERATING RULES FROM 100 EVENTS
2020-05-04 17:17:53,240 -0700 INFO erex:25 - erex run with examples: GET , POST
2020-05-04 17:18:00,550 -0700 INFO erex:25 - erex run with examples: GET , POST
2020-05-04 17:18:03,215 -0700 INFO FieldLearning:62 - GENERATING RULES FROM 100 EVENTS
2020-05-04 17:18:04,787 -0700 INFO erex:25 - erex run with examples: GET , POST
2020-05-04 17:18:07,979 -0700 INFO erex:25 - erex run with examples: GET , POST
2020-05-04 17:18:08,187 -0700 INFO FieldLearning:62 - GENERATING RULES FROM 100 EVENTS
2020-05-04 17:18:10,647 -0700 INFO FieldLearning:62 - GENERATING RULES FROM 100 EVENTS
2020-05-04 17:18:12,615 -0700 INFO FieldLearning:62 - GENERATING RULES FROM 100 EVENTS
2020-05-04 17:18:14,448 -0700 INFO erex:25 - erex run with examples: GET , POST
2020-05-04 17:18:16,148 -0700 INFO FieldLearning:62 - GENERATING RULES FROM 100 EVENTS
2020-05-04 17:21:48,911 -0700 INFO erex:25 - erex run with examples: GET , POST
2020-05-04 17:21:50,206 -0700 INFO FieldLearning:62 - GENERATING RULES FROM 100 EVENTS

The log level is INFO.

[richgalloway]

The output from the erex command should be available in the Jobs menu. Or you can open the job's search.log (Inspect Job then click on one of the search.log links) and search for "Successfully learned regex.".