All Apps and Add-ons
Highlighted

How to find most common words used by cluster command

Communicator

Hi All,

We are trying to cluster a Description field with cluster command in this way:

| cluster t=0.5 labelonly=t showcount=t field=Description match=termset
| table cluster_label cluster_count Description

Do you know if is it feasible to know which are the most common words or string, on each cluster, the algoithm has used on the Dataset to generate the clusterization?
I would like to add one column on the right to my output table containing the patterns that have generate this cluster (that are linked to Description field).

cluster_count | cluster_label | Description             | words
         120  |            1  | Bla bla bla ciao ciao   | bla, ciao
          80  |            2  | Day after day is better | Day

Thanks a lot,
Edoardo

0 Karma
Highlighted

Re: How to find most common words used by cluster command

Splunk Employee
Splunk Employee

Hi,

Did you tried TFIDF algorithm?
https://docs.splunk.com/Documentation/MLApp/4.1.0/User/Algorithms#TFIDF

Additionally, if you are interested in Text Analytics usecase, I would recommend to look into NLP Text Analytics app which is using MLTK Algorithms: https://splunkbase.splunk.com/app/4066/#/details

Let me know if it helps.

Thanks,

View solution in original post

0 Karma
Highlighted

Re: How to find most common words used by cluster command

SplunkTrust
SplunkTrust

Yep! x2 for the NLP app. It gives you the cleantext command

0 Karma
Highlighted

Re: How to find most common words used by cluster command

Explorer

After your cluster command, pipe to the findkeywords command. It will add fields "includeKeywords" and "excludeKeywords," among others, which indicate the terms used for each cluster_label.

0 Karma
Highlighted

Re: How to find most common words used by cluster command

New Member

hi, could you please explain how the findkeywords command works? I couldn't find it anywhere on the Splunk documents.

0 Karma
Highlighted

Re: How to find most common words used by cluster command

Explorer
0 Karma