All Apps and Add-ons

How to find most common words used by cluster command

edoardo_vicendo
Contributor

Hi All,

We are trying to cluster a Description field with cluster command in this way:

| cluster t=0.5 labelonly=t showcount=t field=Description match=termset
| table cluster_label cluster_count Description

Do you know if is it feasible to know which are the most common words or string, on each cluster, the algoithm has used on the Dataset to generate the clusterization?
I would like to add one column on the right to my output table containing the patterns that have generate this cluster (that are linked to Description field).

cluster_count | cluster_label | Description             | words
         120  |            1  | Bla bla bla ciao ciao   | bla, ciao
          80  |            2  | Day after day is better | Day

Thanks a lot,
Edoardo

0 Karma
1 Solution

grana_splunk
Splunk Employee
Splunk Employee

Hi,

Did you tried TFIDF algorithm?
https://docs.splunk.com/Documentation/MLApp/4.1.0/User/Algorithms#TFIDF

Additionally, if you are interested in Text Analytics usecase, I would recommend to look into NLP Text Analytics app which is using MLTK Algorithms: https://splunkbase.splunk.com/app/4066/#/details

Let me know if it helps.

Thanks,

View solution in original post

0 Karma

mschaaf
Path Finder

After your cluster command, pipe to the findkeywords command. It will add fields "includeKeywords" and "excludeKeywords," among others, which indicate the terms used for each cluster_label.

0 Karma

sijingwu
New Member

hi, could you please explain how the findkeywords command works? I couldn't find it anywhere on the Splunk documents.

0 Karma

grana_splunk
Splunk Employee
Splunk Employee

Hi,

Did you tried TFIDF algorithm?
https://docs.splunk.com/Documentation/MLApp/4.1.0/User/Algorithms#TFIDF

Additionally, if you are interested in Text Analytics usecase, I would recommend to look into NLP Text Analytics app which is using MLTK Algorithms: https://splunkbase.splunk.com/app/4066/#/details

Let me know if it helps.

Thanks,

0 Karma

skoelpin
SplunkTrust
SplunkTrust

Yep! x2 for the NLP app. It gives you the cleantext command

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...