topic Re: How: transform csv field value, split, join and create dynamic field value pair in Splunk Search

How: transform csv field value, split, join and create dynamic field value pair

cmckenna — Wed, 05 Jan 2022 00:13:40 GMT

I use a lookup to define alert/SLO specifications. I use the lookups as input filters to my alert searches where I can.

The lookup column name is sli_dimensions_alert: (there are other columns in the lookup):

sli_dimensions_alert="env,service_name,type,class"

The sli_dimensions_alert field specification can have multiple comma separated values.

For example:

sli_dimensions_alert="env,service_name,type,class"

My goal is to create an alert_name based on that CSV value list.

Example raw data:
env="PRD"
service_name="EXGMGR"
type="ERROR"
class="TIMEOUT"

I want to create a macro, calculated field or automatic lookup to transform sli_dimensions_alert="env,service_name,type,class" into alert_name="PRD-EXGMGR-ERROR-TIMEOUT".

I've tried a variety of combinations with split, mvjoin, mvmap, but haven't found a way to make it work.

Re: How: transform csv field value, split, join and create dynamic field value pair

richgalloway — Wed, 05 Jan 2022 01:01:07 GMT

It would help to see what you've already tried and they didn't meet expectations, but perhaps this will help.

... | eval parts=split(sli_dimensions_alert, ",") | eval sli_dimensions_alert = mvindex(parts,0)."-".mvindex(parts,1)."-".mvindex(parts,2)."-".mvindex(parts,3)

Re: How: transform csv field value, split, join and create dynamic field value pair

cmckenna — Wed, 05 Jan 2022 02:00:03 GMT

The SLO lookup file that I use to filter and specify SLO configurations contains these columns (more info than needed really):

lookup file: hi2_slo_config

COLUMNS:
slo_spec_version, slo_name, slo_description, slo_service, slo_env, slo_domain, slo_type, slo_class, slo_lob, slo_severity, slo_category, slo_product, slo_eventtype, slo_time_isrolling, slo_time_count, slo_time_unit, slo_timeslices, slo_budget_method, slo_op, slo_threshold, slo_docref, sli_name, sli_type, sli_source, sli_sourcetype, sli_metric_name, sli_dimensions_metric, sli_dimensions_alert, slo_flag_mock, slo_flag_snow, slo_msg_id_snow, slo_flag_mm, slo_msg_id_mm, slo_webhook_id_mm, slo_flag_email, slo_msg_id_email, slo_msg_dest_email

I use the sli_dimensions_alert field to define the alert naming conventions.

Here are two examples (I want to be able to customize sli_dimensions_alert as desired):

* sli_dimensions_alert="env,service_name,type,class,product,resource,lob"

* sli_dimensions_alert="env,service_name,type,class"

I've split and joined the sli_dimensions_alert field using the following:

| eval alert_name=mvjoin(split(replace(sli_dimensions_alert," ",""), ","), "-")

That eval creates the following:

alert_name="env-service_name-type-class-product-resource-lob"

alert_name="env-service_name-type-class"

I want to substitute/replace/map the fields env, service_name, type, class, product, resource, lob, etc with the value for those fields that come from my source. Example values in previous post.

I haven't nailed it with mvmap and am trying a for command to loop concatenation. Having a hard time with it.

I was thinking I could create a macro that passes in the CSV field (e.g. sli_dimensions_alert="env,service_name,type,class,product,resource,lob") and loop process it with a for command.

Struggling with that. I'm hoping there's a simpler solution that hasn't come to mind.

Re: How: transform csv field value, split, join and create dynamic field value pair

johnhuang — Wed, 05 Jan 2022 04:25:40 GMT

Could you post what your current search looks like?

Re: How: transform csv field value, split, join and create dynamic field value pair

cmckenna — Wed, 05 Jan 2022 17:47:35 GMT

Mock up snippet:

| makeresults
| eval COMMENT="The following fields are search time results"

| eval env="PRD", service_name="EXGMGR", type="LOBREQUEST", class="TIMEOUTERROR", lob="GOV"

| eval COMMENT="The sli_dimensions_alert field is retrieved from a lookup and has a variable number of comma delimited values. Two examples below"

| eval sli_dimensions_alert="env,service_name,type,class"

| eval sli_dimensions_alert="env,service_name,type,class,product,resource,lob"

| eval alert_name=mvjoin(split(sli_dimensions_alert,","),"-")

| eval COMMENT="I want alert_name to contain the actual field values"

| eval COMMENT="alert_name = PRD-EXGMGR-LOBREQUEST-TIMEOUTERROR-GOV"

Re: How: transform csv field value, split, join and create dynamic field value pair

johnhuang — Wed, 05 Jan 2022 21:35:01 GMT

This should work. We are doing a foreach on the field_name to match against the fields listed in "sli_dimensions_alert", and a true match will result in the field_value being appended to new variable "alert_values".

| makeresults | eval env="PRD", service_name="EXGMGR", type="LOBREQUEST", class="TIMEOUTERROR", lob="GOV", product="APPLE", resource="SERVER" | eval sli_dimensions_alert="env,service_name,type,class" | eval sli_dimensions_alert="env,service_name,type,class,product,resource,lob" | eval alert_fields=SPLIT(sli_dimensions_alert, ",") | foreach env service_name type class product resource lob [| eval field_name="<<FIELD>>" | eval alert_values=IF(field_name=alert_fields, MVAPPEND(alert_values, <<FIELD>>), alert_values)] | eval alert_name=MVJOIN(alert_values, "-")