Extracting from multiple fields and group by Domain name

thiagarajan — Tue, 24 Sep 2013 03:48:55 GMT

My logs looks like this

Tue Aug 27 2013 00:34:47 [DEV][MyTest][error] mpgw(IntegrationGateway): tid(372165969)[error][10.11.12.123]: Either service is down or transaction timed out for Service:WorkspaceData
UUID:4c4b1672-9af1-4f95-a28b-d78611bd6a6
Backend:lprva1234.test.com:6090
Domain:SpaceK

Tue Aug 27 2013 00:35:28 [DEV][MyTest][error] mpgw(IntegrationGateway): tid(379832419)[error][10.14.24.263]: Either service is down or transaction timed out for Service:MyList
UUID:8f3dc371-845c-4768-928b-35938dacffb6
Backend:lprva4567.test.com:6087
Domain:SpaceH

Tue Aug 27 2013 00:54:39 [DEV][MyTest][error] mpgw(IntegrationGateway): tid(327317173)[error][10.11.12.123]: Either service is down or transaction timed out for Service:WorkspaceData
UUID:99dafd8f-9639-4d8e-ac5d-5d0d5a35ae77
Backend:lprva7891.test.com:6090
Domain:SpaceK

Sun Sep 01 2013 00:23:27 [DEV][MyTest][error] mpgw(IntegrationGateway): tid(112725141)[error][10.11.12.123]: Either service is down or transaction timed out for Service:MyConnnect
UUID:2e57e791-e6fe-4b0e-b401-77de0a2ba511
Backend:lprva8225.test.com:6091
Domain:SpaceL

Sun Sep 01 2013 00:23:37 [DEV][MyTest][error] mpgw(IntegrationGateway): tid(112727877)[error][10.11.12.123]: Either service is down or transaction timed out for Service:MyConnnect
UUID:523b378f-14d3-41c2-8357-e8642a595c5d
Backend:lprva8228.test.com:6091
Domain:SpaceL

The regex for timedoutservice is -> (?i)^(?:[^:]*:){5}(?P<timedoutservice>[^\s]+)
Search query is -> sourcetype="MyLog" ("transaction timed out for Service:" MyTest) |stats count as errorcount by timedoutservice

I am getting result something similar to this

timedoutservice                         errorcount
WorkspaceData                               2
MyList                                      1
MyConnnect                                  2

Expected result

timedoutservice                        errorcount        Domain
WorkspaceData                               2            SpaceK
MyList                                      1            SpaceH
MyConnnect                                  2            SpaceL

UUID and Backend will be changing but the domain name remains same for all the services.
The regex for domainName is -> (?i)\tDomain:(?P<Domain>.+)

I just tried to combine both the results

((?i)^(?:[^:]*:){5}(?P<timedoutservice>[^\s]+)(?i)\tDomain:(?P<Domain>.+))

I am getting the exception "Invalid regex: no named extraction at position 0 (i.e., "((?i)^(?:[..."). Expected "(?Ppattern)"
Do i need to use group by. How to extract from multiple fields. Any help is appreciated.

Re: Extracting from multiple fields and group by Domain name

lukejadamec — Tue, 24 Sep 2013 04:03:19 GMT

The field

domain:value

Should be extracted automatically.

If it is not, have you tried the automated field extraction wizard? (down arrow key next to the event in a regular search)

If it is, then something like this should work:

sourcetype="MyLog" ("transaction timed out for Service:" MyTest) |stats count as errorcount by timedoutservice,domain

Re: Extracting from multiple fields and group by Domain name

thiagarajan — Tue, 24 Sep 2013 04:58:41 GMT

This is what I expected. Thank you very much for the quick reply. Can you explain why I am able get the host name without group by.

Re: Extracting from multiple fields and group by Domain name

thiagarajan — Tue, 24 Sep 2013 05:16:55 GMT

This is what I expected. Thank you for the quick reply. But can u tell how I am getting the domain name without any group by. Is it because domain name is unique.

topic Re: Extracting from multiple fields and group by Domain name in Splunk Search

Extracting from multiple fields and group by Domain name

Expected result

Re: Extracting from multiple fields and group by Domain name

Re: Extracting from multiple fields and group by Domain name

Re: Extracting from multiple fields and group by Domain name