All Apps and Add-ons

Label Encoding in Machine Learning Toolkit

wbw4am
New Member

I believe by default the Machine Learning Toolkit utilizes one hot encoding when converting categorical variables to numerical. Is there an easy way to utilize label encoding? For example - I want to assign a risk score based on country. So China may map to a 5 and US may map to a 1, where 5 is riskier than 1.

I imagine I could do this with a bunch of eval commands in the query or alternatively an additional field extract, but is there a "prettier" way to do this?

0 Karma

aoliner_splunk
Splunk Employee
Splunk Employee

One option, if the scores are relatively static, is to use a lookup. Another option, if you've calculated all the 'risk_score's and want to keep them "up to date" as conditions change, is to use a regression model:

... | fit LinearRegression risk_score from factor_A factor_B ... into my_risk_model

You could use whatever regression algorithm you want and whatever factors you want. Then, when it's time to score:

... | apply my_risk_model
0 Karma

aljohnson_splun
Splunk Employee
Splunk Employee

Have you considered using a lookup?

0 Karma
Get Updates on the Splunk Community!

Get Your Exclusive Splunk Certified Cybersecurity Defense Engineer at Splunk .conf24 ...

We’re excited to announce a new Splunk certification exam being released at .conf24! If you’re headed to Vegas ...

Share Your Ideas & Meet the Lantern team at .Conf! Plus All of This Month’s New ...

Splunk Lantern is Splunk’s customer success center that provides advice from Splunk experts on valuable data ...

Combine Multiline Logs into a Single Event with SOCK: a Step-by-Step Guide for ...

Combine multiline logs into a single event with SOCK - a step-by-step guide for newbies Olga Malita The ...