Splunk Search

Document Flow tracking over multiple sources. Linking unique field

MorphiusX
Engager

Beginner user here.

PART 1
Wanting to track documents over multiple sources to ensure they reach their destination

Source 1 - Source 2 or 3 - Source 4
Start Point (Sent) - Middle Points (Accepted or Rejected) - End Point (Received) 

Each document has the following
ID = Unique to each document
DATE \ TIME STAMP = Says what time the document arrived to that point
DESCRIPTION = like a subject what the document contains

All documents have a unique ID that is tracked on each source. 
I want to track this ID and ensure that it has gone from source 1 ,2 or 3 and arrived at 4. If for some reason its in 2 and not in 4 display that Doc ID in a table.

PART 2 - I can probably work this one out myself after I know how to link everything.
After they are linked I would like to compare the time between when it was at source 1 to when it arrived at source 3. 

Labels (4)
0 Karma
1 Solution

MorphiusX
Engager

Hi,
Thanks for the help.

With some troubleshooting, I was able to get it working. The one thing that messed me up was putting quotes around the source in the if statements.

My final code is


ID=* index=ind ((source=start) OR (source=accept) OR (source=reject) OR (source=received))
| eval time1=if(source="start", _time, null())
| eval time2=if(source="accept", _time, null())
| eval time3=if(source="reject", _time, null())
| eval time4=if(source="received", _time, null())
| stats values(time1) AS time1 values(time2) AS time2 values(time3) AS time3 values(time4) AS time4 by ID

View solution in original post

0 Karma

PickleRick
SplunkTrust
SplunkTrust

You need something like this (a general idea - you have to tweak it to your situation)

ID=* ((index=src1 sourcetype=src1) OR (index=src2 sourcetype=src2) OR (index=src3 sourcetype=src3) OR (index=src4 sourcetype=src4))
| eval starttime=if(index=src1,_time,null())
| eval middletime=if(index=src2 OR index=src3,_time,null())
| eval endtime=if(index=src3,_time,null())
| stats values(starttime) AS starttime values(middletime) AS middletime values(endtime) AS endtime by ID

Now you should have a table listing timestamps when the doc with given ID was registered at each of waypoints - start, middle and end. You can easily filter out some of the results and get, for example, only those which have middle time but don't have endtime

| search middletime=* NOT endtime=*

 And of course when you have those timestamps you can calculate the delay between various steps. For example

| eval start_to_end=endtime - starttime

MorphiusX
Engager

Hi,
Thanks for the help.

With some troubleshooting, I was able to get it working. The one thing that messed me up was putting quotes around the source in the if statements.

My final code is


ID=* index=ind ((source=start) OR (source=accept) OR (source=reject) OR (source=received))
| eval time1=if(source="start", _time, null())
| eval time2=if(source="accept", _time, null())
| eval time3=if(source="reject", _time, null())
| eval time4=if(source="received", _time, null())
| stats values(time1) AS time1 values(time2) AS time2 values(time3) AS time3 values(time4) AS time4 by ID
0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

Something like this: obviously you will have to put in the details of your indexes, sourcetypes, etc.

<your index(es) and sourcetype(s)>
| eval time1=if(source is 1, _time, null())
| eval time2=if(source is 2, _time, null())
| eval time3=if(source is 3, _time, null())
| eval time4=if(source is 4, _time, null())
| stats values(time1) as time1 values(time2) as time2 values(time3) as time3 values(time4) as time4 values(description) as description by docID
Get Updates on the Splunk Community!

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...

Introducing the 2024 Splunk MVPs!

We are excited to announce the 2024 cohort of the Splunk MVP program. Splunk MVPs are passionate members of ...

Splunk Custom Visualizations App End of Life

The Splunk Custom Visualizations apps End of Life for SimpleXML will reach end of support on Dec 21, 2024, ...