Matching only part of a string to return a result when different

[ssaenger]

Hi,

I am streaming results from a Kubernetes cluster and i am monitoring for pod restarts by looking at the name of each pod and reporting when it changes.

I am able to return the pod name, however i am unable to make my match statement work to only return the different pod names.

the pods are named;

prod-K8-1-b5c85b547-26wqn 
prod-K8-2-7c56dc8559-kzpwm 
prod-K8-3-7c7bccf947-4skx2 
prod-K8-4-769bb9d4f5-tmwbz 
...

i have code that returns the pod names over a time frame;

   index=K8 "kubernetes.container_name"=tfs kubernetes.pod_name=prod* earliest=-5m@m latest=@m| stats first(kubernetes.pod_name) as old_pod last(kubernetes.pod_name) as new_pod by kubernetes.pod_name | where old_pod=new_pod

kubernetes.pod_name                     old_pod                                                  new_pod

prod-K8-1-b5c85b547-26wqn	prod-K8-1-b5c85b547-26wqn	prod-K8-1-b5c85b547-26wqn
prod-K8-1-b5c85b547-tdgwg	prod-K8-1-b5c85b547-tdgwg	prod-K8-1-b5c85b547-tdgwg

I think i need to use some regex in my where statement, but i end up getting no results, the above where clause was to show an output.

I would like to get my output table to list the current pod name then have the two different pod names.

any help would be much appreciated.

[gcusello · Accepted solution]

Hi @ssaenger,

the result of your search will be always the same because you have the same field in old_pod, now_pod and in the by cluase.

Is there a key to correlate old_pod and new pod?

something like this:

 index=K8 "kubernetes.container_name"=tfs kubernetes.pod_name=prod* earliest=-5m@m latest=@m
| stats first(kubernetes.pod_name) as old_pod last(kubernetes.pod_name) as new_pod by key 
| where old_pod=new_pod

Ciao.

Giuseppe

[ssaenger]

Hi Giuseppe,

yes thanks for that.

i created a new field;

| rex field=kubernetes.pod_name "^(?P<pod>.{14})"

then added pod to the stats by clause, this was the bit I could not see.

Thanks for the pointer.

[gcusello]

Hi @ssaenger,

good for you, see next time!

you could also try something like this:

 index=K8 "kubernetes.container_name"=tfs kubernetes.pod_name=prod* earliest=-5m@m latest=@m
| stats dc(kubernetes.pod_name) as pods by pod 
| where pods>1

Ciao and happy splunking.

Giuseppe

P.S.: Karma Points are appreciated 😉