Splunk Search

Document Flow tracking over multiple sources. Linking unique field

MorphiusX
Engager

Beginner user here.

PART 1
Wanting to track documents over multiple sources to ensure they reach their destination

Source 1 - Source 2 or 3 - Source 4
Start Point (Sent) - Middle Points (Accepted or Rejected) - End Point (Received) 

Each document has the following
ID = Unique to each document
DATE \ TIME STAMP = Says what time the document arrived to that point
DESCRIPTION = like a subject what the document contains

All documents have a unique ID that is tracked on each source. 
I want to track this ID and ensure that it has gone from source 1 ,2 or 3 and arrived at 4. If for some reason its in 2 and not in 4 display that Doc ID in a table.

PART 2 - I can probably work this one out myself after I know how to link everything.
After they are linked I would like to compare the time between when it was at source 1 to when it arrived at source 3. 

Labels (3)
0 Karma
1 Solution

MorphiusX
Engager

Hi,
Thanks for the help.

With some troubleshooting, I was able to get it working. The one thing that messed me up was putting quotes around the source in the if statements.

My final code is


ID=* index=ind ((source=start) OR (source=accept) OR (source=reject) OR (source=received))
| eval time1=if(source="start", _time, null())
| eval time2=if(source="accept", _time, null())
| eval time3=if(source="reject", _time, null())
| eval time4=if(source="received", _time, null())
| stats values(time1) AS time1 values(time2) AS time2 values(time3) AS time3 values(time4) AS time4 by ID

View solution in original post

0 Karma

PickleRick
SplunkTrust
SplunkTrust

You need something like this (a general idea - you have to tweak it to your situation)

ID=* ((index=src1 sourcetype=src1) OR (index=src2 sourcetype=src2) OR (index=src3 sourcetype=src3) OR (index=src4 sourcetype=src4))
| eval starttime=if(index=src1,_time,null())
| eval middletime=if(index=src2 OR index=src3,_time,null())
| eval endtime=if(index=src3,_time,null())
| stats values(starttime) AS starttime values(middletime) AS middletime values(endtime) AS endtime by ID

Now you should have a table listing timestamps when the doc with given ID was registered at each of waypoints - start, middle and end. You can easily filter out some of the results and get, for example, only those which have middle time but don't have endtime

| search middletime=* NOT endtime=*

 And of course when you have those timestamps you can calculate the delay between various steps. For example

| eval start_to_end=endtime - starttime

MorphiusX
Engager

Hi,
Thanks for the help.

With some troubleshooting, I was able to get it working. The one thing that messed me up was putting quotes around the source in the if statements.

My final code is


ID=* index=ind ((source=start) OR (source=accept) OR (source=reject) OR (source=received))
| eval time1=if(source="start", _time, null())
| eval time2=if(source="accept", _time, null())
| eval time3=if(source="reject", _time, null())
| eval time4=if(source="received", _time, null())
| stats values(time1) AS time1 values(time2) AS time2 values(time3) AS time3 values(time4) AS time4 by ID
0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

Something like this: obviously you will have to put in the details of your indexes, sourcetypes, etc.

<your index(es) and sourcetype(s)>
| eval time1=if(source is 1, _time, null())
| eval time2=if(source is 2, _time, null())
| eval time3=if(source is 3, _time, null())
| eval time4=if(source is 4, _time, null())
| stats values(time1) as time1 values(time2) as time2 values(time3) as time3 values(time4) as time4 values(description) as description by docID
Get Updates on the Splunk Community!

Index This | What goes up and never comes down?

January 2026 Edition  Hayyy Splunk Education Enthusiasts and the Eternally Curious!   We’re back with this ...

Splunkers, Pack Your Bags: Why Cisco Live EMEA is Your Next Big Destination

The Power of Two: Splunk &#43; Cisco at "Ludicrous Scale"   You know Splunk. You know Cisco. But have you seen ...

Data Management Digest – January 2026

Welcome to the January 2026 edition of Data Management Digest! Welcome to the January 2026 edition of Data ...