How to make a loop including a dataframe for each jobname

[nsantiago17]

I have a dataset like this:

jobname     date                   avg      start             end
A              30/03/2019            84900  03:13:25           20:59:47
B              02/04/2019            60798  16:53:05           16:00:05
C              02/04/2019            60798  16:53:05           16:00:04
D              02/04/2019            79200  22:00:01           15:00:39

The length is 100k, and I need to do a loop including a dataframe for each jobname, for example:

df1 = A, 30/03/2019, 84900, 03:13:25,20:59:47
df2 = B, 02/04/2019,60798,16:53:05,16:00:05

So I can use the kmeans algorithm to cluster all of the data.

How can I do this?

[evania]

Hi @nsantiago17 ,

Did you have a chance to check out an answer? If it worked, please resolve this post by approving it! If your problem is still not solved, keep us updated so that someone else can help you.

Thanks for posting!

[nsantiago17]

Hi, I replied Martyn but he didn't answer anymore, so I'm still without a solution or something that can help me to solve the question.

[martynoconnor]

Hi There,

What's the end use case you're trying to achieve here? I have a similar sort of search that I use in a dashboard of my own. The data you have looks fairly simple in structure and so you could index the data, ensuring the fields are correctly extracted and then use a search something like the following:

index=<your_index>  
| kmeans k=<how_many_clusters> dt=<your_distype> <numeric_field_1> <numeric_field_2> cfield="Cluster number"
| stats mode(<field1>) mode(<field2>) by "Cluster number"
| rename mode(<field1>) AS "<Better Name>" mode(<field2>) AS "<Better Name>"

The search above is one I use to cluster patterns in airline traffic, grouping them by heading and altitude.

Hopefully that's enough to get going with - let me know if you have any more questions.

Regards,

Martyn

[nsantiago17]

Hi Martyn,

I need to cluster my dataset in 4 groups: those who have outliers early in the month, 15th, 21th and last day of the month by the jobname, and I have to use the avg, date and one other numerical column.
So I can analyze each jobname already knowing their pattern. I'm trying to separate each jobname with their information and then start to figure how to cluster them. If you have some idea, please share with me.

Regards,

Nick