Deployment Architecture

Using |bucket for events instead of _time?

SilviaGebel
Path Finder

Hi, I am currently trying to find a connection between events, that are from different sources and have different timestamps, so they have no relation at all.

My thought is to use the bucket command, to bucket for example, 5 events and see if during that timespan of theses 5 events, an event of a different source is occuring.

Has anyone tried this?

Tags (3)
0 Karma
1 Solution

SilviaGebel
Path Finder

I solved the problem by using | filldown . This will give me the last measured value for all time points until the next values was measured. The temperature was only recorded when a change occured, so filldown is perfect!

View solution in original post

0 Karma

SilviaGebel
Path Finder

I solved the problem by using | filldown . This will give me the last measured value for all time points until the next values was measured. The temperature was only recorded when a change occured, so filldown is perfect!

0 Karma

Runals
Motivator

I'd suggest looking into the transaction command. In that you can combine multiple events into based on a number of different criteria: common fields, max or min timeframe between individual events, define what is the start or stop of an event, etc

0 Karma

SilviaGebel
Path Finder

Transaction could help, yes. I would need to have a command like:
combine all events that occur in a certain timespan, but only choose the events closest to fieldX. Every event has one specific field.

0 Karma

Runals
Motivator

Sorry - am taking a closer look at your data now. Assuming you have data in fields named something like Raumtemperatur and Luftfeuchtigkeit I'd start with something like

index=foo | bucket span=5m _time | stats avg(Raumtemperatur) as Raumtemperatur avg(Luftfeuchtigkeit) as Luftfeuchtigkeit by _time

I did 5 minutes and avg because there didn't seem to be data for each minute from your sample and am not sure how granular you want to get. Alternatively you could use max instead - what it comes to is figuring out how you want to handle buckets where there happens to be more than one event. If the primary way to consume the data is visually you can show a timechart where the data points are connected. If the primary way is in a table and you want very granular timespans you will probably need to check out streamstats to fill in the "blanks."

Update: if those fields aren't created throw this in there (assuming I'm looking at this right)

... | rex "Raumtemperatur;.+?,(?<Raumtemperatur>\d+)" | ... rex "Luftfeuchtigkeit;.+?,(?<Luftfeuchtigkeit>\d+)" | bucket ...
0 Karma

SilviaGebel
Path Finder

Thank you for your time and help! The fields Raumtemperatur (values: ~23°C in the raw text right after the timestamp) and Luftfeuchtigkeit (~16%) are already extracted.
I want to see what humiditiy (Luftfeuchtigkeit) I have at a certain roomtemperature (Raumtemperatur). So the goal is to have a table which I can then visualize. I want to create a chart, that shows me, what humidity I have at a certain roomtemperatue. (y-axis: humidity; x-axis: roomtemperatur) Or even analyze weather I can predict the humidity by the roomtemperature or vice verca. Kind of like a regression analysis. For that, I have to find some kind of connection between these two attributes.

The problem really is that I cannot connect the attributes by the timestamp because the attributes are measured at random times, independent of each other.

0 Karma

neelamssantosh
Contributor

With bucketing i doubt you can achieve the relation instead of that
Good method is to search by Looking at specific time period for events that happened around the same time can help correlate results and find the root cause(in Date and Time Range with with precise time) while debugging
There must be some id/pid/xx.. for every process figure it out by studying those logs hope that can help you to figure out the relation.

0 Karma

SilviaGebel
Path Finder

380462;._PLC.CVI.Z010 [\xE5\xA1C]: aktuelle Raumtemperatur;06.03.2015 15:35;24;\xE5\xA1C;268298
380462;._PLC.CVI.Z010 [\xE5\xA1C]: aktuelle Raumtemperatur;06.03.2015 15:20;23,9;\xE5\xA1C;268298
380462;._PLC.CVI.Z010 [\xE5\xA1C]: aktuelle Raumtemperatur;06.03.2015 15:08;23,8;\xE5\xA1C;268298
380462;._PLC.CVI.Z010 [\xE5\xA1C]: aktuelle Raumtemperatur;06.03.2015 15:01;23,7;\xE5\xA1C;268298
380462;._PLC.CVI.Z010 [\xE5\xA1C]: aktuelle Raumtemperatur;06.03.2015 14:50;23,8;\xE5\xA1C;268298
380462;._PLC.CVI.Z010 [\xE5\xA1C]: aktuelle Raumtemperatur;06.03.2015 14:45;23,7;\xE5\xA1C;268298
380462;._PLC.CVI.Z010 [\xE5\xA1C]: aktuelle Raumtemperatur;06.03.2015 14:43;23,8;\xE5\xA1C;268298
380462;._PLC.CVI.Z010 [\xE5\xA1C]: aktuelle Raumtemperatur;06.03.2015 14:40;23,7;\xE5\xA1C;268298

380461;._PLC.CVI.Z009 [%]: aktuelle Luftfeuchtigkeit;06.03.2015 15:28;16,7;%;4796
380461;._PLC.CVI.Z009 [%]: aktuelle Luftfeuchtigkeit;06.03.2015 15:27;16,6;%;4796
380461;._PLC.CVI.Z009 [%]: aktuelle Luftfeuchtigkeit;06.03.2015 15:26;16,4;%;4796
380461;._PLC.CVI.Z009 [%]: aktuelle Luftfeuchtigkeit;06.03.2015 15:25;16,9;%;4796
380461;._PLC.CVI.Z009 [%]: aktuelle Luftfeuchtigkeit;06.03.2015 15:23;16,7;%;4796
380461;._PLC.CVI.Z009 [%]: aktuelle Luftfeuchtigkeit;06.03.2015 15:22;17,1;%;4796
380461;._PLC.CVI.Z009 [%]: aktuelle Luftfeuchtigkeit;06.03.2015 15:21;16,6;%;4796
380461;._PLC.CVI.Z009 [%]: aktuelle Luftfeuchtigkeit;06.03.2015 15:20;16,8;%;4796

0 Karma

neelamssantosh
Contributor

hope,
Raumtemperatur and Luftfeuchtigkeit keywords can help you to seperate the events.
if you are not sure how many such strings are there use regex to fetch the values..

0 Karma

SilviaGebel
Path Finder

Yes, I can use them to seperate the events, but I need to find a way to combine them. I was thinking about commands like |eval coalesce(Raumtemperatur,Luftfeuchtigkeit) in order for the values to be in combined in one event

0 Karma

SilviaGebel
Path Finder

unfortunately, I am not looking at logs, but my data are measured data like temperature and humidity, so there are absolutely no common fields in the saved rawdata

0 Karma

neelamssantosh
Contributor

Kindly share couple of your sample log..

0 Karma
Get Updates on the Splunk Community!

Index This | Divide 100 by half. What do you get?

November 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with this ...

Stay Connected: Your Guide to December Tech Talks, Office Hours, and Webinars!

❄️ Celebrate the season with our December lineup of Community Office Hours, Tech Talks, and Webinars! ...

Splunk and Fraud

Watch Now!Watch an insightful webinar where we delve into the innovative approaches to solving fraud using the ...