Splunk Search

Document Flow tracking over multiple sources. Linking unique field

MorphiusX
Engager

Beginner user here.

PART 1
Wanting to track documents over multiple sources to ensure they reach their destination

Source 1 - Source 2 or 3 - Source 4
Start Point (Sent) - Middle Points (Accepted or Rejected) - End Point (Received) 

Each document has the following
ID = Unique to each document
DATE \ TIME STAMP = Says what time the document arrived to that point
DESCRIPTION = like a subject what the document contains

All documents have a unique ID that is tracked on each source. 
I want to track this ID and ensure that it has gone from source 1 ,2 or 3 and arrived at 4. If for some reason its in 2 and not in 4 display that Doc ID in a table.

PART 2 - I can probably work this one out myself after I know how to link everything.
After they are linked I would like to compare the time between when it was at source 1 to when it arrived at source 3. 

Labels (4)
0 Karma
1 Solution

MorphiusX
Engager

Hi,
Thanks for the help.

With some troubleshooting, I was able to get it working. The one thing that messed me up was putting quotes around the source in the if statements.

My final code is


ID=* index=ind ((source=start) OR (source=accept) OR (source=reject) OR (source=received))
| eval time1=if(source="start", _time, null())
| eval time2=if(source="accept", _time, null())
| eval time3=if(source="reject", _time, null())
| eval time4=if(source="received", _time, null())
| stats values(time1) AS time1 values(time2) AS time2 values(time3) AS time3 values(time4) AS time4 by ID

View solution in original post

0 Karma

PickleRick
Ultra Champion

You need something like this (a general idea - you have to tweak it to your situation)

ID=* ((index=src1 sourcetype=src1) OR (index=src2 sourcetype=src2) OR (index=src3 sourcetype=src3) OR (index=src4 sourcetype=src4))
| eval starttime=if(index=src1,_time,null())
| eval middletime=if(index=src2 OR index=src3,_time,null())
| eval endtime=if(index=src3,_time,null())
| stats values(starttime) AS starttime values(middletime) AS middletime values(endtime) AS endtime by ID

Now you should have a table listing timestamps when the doc with given ID was registered at each of waypoints - start, middle and end. You can easily filter out some of the results and get, for example, only those which have middle time but don't have endtime

| search middletime=* NOT endtime=*

 And of course when you have those timestamps you can calculate the delay between various steps. For example

| eval start_to_end=endtime - starttime

MorphiusX
Engager

Hi,
Thanks for the help.

With some troubleshooting, I was able to get it working. The one thing that messed me up was putting quotes around the source in the if statements.

My final code is


ID=* index=ind ((source=start) OR (source=accept) OR (source=reject) OR (source=received))
| eval time1=if(source="start", _time, null())
| eval time2=if(source="accept", _time, null())
| eval time3=if(source="reject", _time, null())
| eval time4=if(source="received", _time, null())
| stats values(time1) AS time1 values(time2) AS time2 values(time3) AS time3 values(time4) AS time4 by ID
0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

Something like this: obviously you will have to put in the details of your indexes, sourcetypes, etc.

<your index(es) and sourcetype(s)>
| eval time1=if(source is 1, _time, null())
| eval time2=if(source is 2, _time, null())
| eval time3=if(source is 3, _time, null())
| eval time4=if(source is 4, _time, null())
| stats values(time1) as time1 values(time2) as time2 values(time3) as time3 values(time4) as time4 values(description) as description by docID
Get Updates on the Splunk Community!

.conf23 Registration is Now Open!

Time to toss the .conf-etti &#x1f389; —  .conf23 registration is open!   Join us in Las Vegas July 17-20 for ...

Don't wait! Accept the Mission Possible: Splunk Adoption Challenge Now and Win ...

Attention everyone! We have exciting news to share! We are recruiting new members for the Mission Possible: ...

Unify Your SecOps with Splunk Mission Control

In today’s post, I'm excited to share some recent Splunk Mission Control innovations. With Splunk Mission ...