topic Need help with Regex in Knowledge Management

Need help with Regex

AK_Splunk — Thu, 05 Jan 2023 05:34:25 GMT

Need help with Regex

field ------------------------feildvalue

servername ---------- xtestf100s

log_level--------------INFO OR error or warning

message ------------ anything from gofer till end

Jan 3 03:50:38 xtestf100s goferd: [INFO][worker-0] gofer.messaging.adapter.connect:28 - connecting: proton+amqps://xtest123s.pharma.aventis.com:5647
Jan 3 03:50:38 xtestf100s goferd: [INFO][worker-0] gofer.messaging.adapter.proton.connection:87 - open: URL: amqps://xtest123s.pharma.aventis.com:5647|SSL: ca: /etc/rhsm/ca/katello-default-ca.pem|key: None|certificate: /etc/pki/consumer/bundle.pem|host-validation: None
Jan 3 03:50:38 xtestf100s goferd: [ERROR][worker-0] gofer.messaging.adapter.connect:33 - connect: proton+amqps://xtest123s.pharma.aventis.com:5647, failed: Connection amqps://xtest123s.pharma.aventis.com:5647 disconnected: Condition('proton.pythonio', 'Connection refused to all addresses')
Jan 3 03:50:38 xtestf100s goferd: [INFO][worker-0] gofer.messaging.adapter.connect:35 - retry in 106 seconds
Jan 3 03:50:54 xtestf100s kernel: type=1400 audit(1672714254.276:566412): avc: denied { read } for pid=75981 comm="ip" name="libCvDllFilter.so" dev="dm-13" ino=393745 scontext=system_u:system_r:ifconfig_t:s0 tcontext=system_u:object_r:unlabeled_t:s0 tclass=file permissive=1
Jan 3 03:50:54 xtestf100s kernel: type=1400 audit(1672714254.276:566413): avc: denied { open } for pid=75981 comm="ip" path="/opt/commvault/Base64/libCvDllFilter.so" dev="dm-13" ino=393745 scontext=system_u:system_r:ifconfig_t:s0 tcontext=system_u:object_r:unlabeled_t:s0 tclass=file permissive=1
Jan 3 03:50:54 xtestf100s kernel: type=1400 audit(1672714254.276:566414): avc: denied { getattr } for pid=75981 comm="ip" path="/opt/commvault/Base64/libCvDllFilter.so" dev="dm-13" ino=393745 scontext=system_u:system_r:ifconfig_t:s0 tcontext=system_u:object_r:unlabeled_t:s0 tclass=file permissive=1
Jan 3 03:50:54 xtestf100s kernel: type=1400 audit(1672714254.276:566415): avc: denied { execute } for pid=75981 comm="ip" path="/opt/commvault/Base64/libCvDllFilter.so" dev="dm-13" ino=393745 scontext=system_u:system_r:ifconfig_t:s0 tcontext=system_u:object_r:unlabeled_t:s0 tclass=file permissive=1
Jan 3 03:51:43 xtestf100s kernel: type=1400 audit(1672714303.392:566416): avc: denied { read } for pid=77988 comm="ip" name="Base" dev="dm-13" ino=116 scontext=system_u:system_r:ifconfig_t:s0 tcontext=unconfined_u:object_r:unlabeled_t:s0 tclass=lnk_file permissive=1
Jan 3 03:51:43 xtestf100s kernel: type=1400 audit(1672714303.392:566417): avc: denied { read } for pid=77988 comm="ip" name="libCvDllFilter.so" dev="dm-13" ino=393745 scontext=system_u:system_r:ifconfig_t:s0 tcontext=system_u:object_r:unlabeled_t:s0 tclass=file permissive=1

Re: Need help with Regex

PickleRick — Thu, 05 Jan 2023 08:15:46 GMT

Again - use preformatted style or code block to include blocks of text which should not be "massacred" by the browser. (like event samples).

The hostname extraction should work pretty well with standard transform "syslog-host". And it should be done in index-time since you probably want to have this as host field. Otherwise, if you really only need it as search-time extraction, just copy the regex from the syslog-host transform.

For log level... the most obvious one that comes to mind is

\[(?<log_level>INFO|ERROR|WARNING)\]

But this might not be the best idea depending on how you want it anchored within the event.

For message - with your definition - it would be

(?<message>goferd.*)

But that's probably _not_ what you want. And while Splunk can sometimes optimize some things relatively well, you might want to rewrite multiple separate extractions into a single regex with multiple capturing groups.

Re: Need help with Regex

gcusello — Thu, 05 Jan 2023 08:23:05 GMT

Hi @AK_Splunk,

in your regex there are two different logs, if you want to take as message from "gofer" to the end of row you can use the following regex:

<your_search> |c rex "^(?<timestamp>\w+ \d+ \d+:\d+:\d*)\s+(?<servername>\w+)[^\[]+\[(?<log_level>[^\]]+)\][^\]]+\]\s+(?<message>.*)"

that you can test at https://regex101.com/r/tQJVAm/1

about the second part of logs, what do you want to extract?

Ciao.

Giuseppe

Re: Need help with Regex

AK_Splunk — Fri, 06 Jan 2023 09:30:23 GMT

Hi ,

Thanks for your response but can you guide how to send sample data by using preformatted style or code block .

Re: Need help with Regex

gcusello — Fri, 06 Jan 2023 10:46:41 GMT

Hi @AK_Splunk,

to send code or sample data use the "Insert/Edit code sample" ("</>") button.

Ciao.

Giuseppe