Splunk Search

Document Flow tracking over multiple sources. Linking unique field

MorphiusX
Engager

Beginner user here.

PART 1
Wanting to track documents over multiple sources to ensure they reach their destination

Source 1 - Source 2 or 3 - Source 4
Start Point (Sent) - Middle Points (Accepted or Rejected) - End Point (Received) 

Each document has the following
ID = Unique to each document
DATE \ TIME STAMP = Says what time the document arrived to that point
DESCRIPTION = like a subject what the document contains

All documents have a unique ID that is tracked on each source. 
I want to track this ID and ensure that it has gone from source 1 ,2 or 3 and arrived at 4. If for some reason its in 2 and not in 4 display that Doc ID in a table.

PART 2 - I can probably work this one out myself after I know how to link everything.
After they are linked I would like to compare the time between when it was at source 1 to when it arrived at source 3. 

Labels (4)
0 Karma
1 Solution

MorphiusX
Engager

Hi,
Thanks for the help.

With some troubleshooting, I was able to get it working. The one thing that messed me up was putting quotes around the source in the if statements.

My final code is


ID=* index=ind ((source=start) OR (source=accept) OR (source=reject) OR (source=received))
| eval time1=if(source="start", _time, null())
| eval time2=if(source="accept", _time, null())
| eval time3=if(source="reject", _time, null())
| eval time4=if(source="received", _time, null())
| stats values(time1) AS time1 values(time2) AS time2 values(time3) AS time3 values(time4) AS time4 by ID

View solution in original post

0 Karma

PickleRick
SplunkTrust
SplunkTrust

You need something like this (a general idea - you have to tweak it to your situation)

ID=* ((index=src1 sourcetype=src1) OR (index=src2 sourcetype=src2) OR (index=src3 sourcetype=src3) OR (index=src4 sourcetype=src4))
| eval starttime=if(index=src1,_time,null())
| eval middletime=if(index=src2 OR index=src3,_time,null())
| eval endtime=if(index=src3,_time,null())
| stats values(starttime) AS starttime values(middletime) AS middletime values(endtime) AS endtime by ID

Now you should have a table listing timestamps when the doc with given ID was registered at each of waypoints - start, middle and end. You can easily filter out some of the results and get, for example, only those which have middle time but don't have endtime

| search middletime=* NOT endtime=*

 And of course when you have those timestamps you can calculate the delay between various steps. For example

| eval start_to_end=endtime - starttime

MorphiusX
Engager

Hi,
Thanks for the help.

With some troubleshooting, I was able to get it working. The one thing that messed me up was putting quotes around the source in the if statements.

My final code is


ID=* index=ind ((source=start) OR (source=accept) OR (source=reject) OR (source=received))
| eval time1=if(source="start", _time, null())
| eval time2=if(source="accept", _time, null())
| eval time3=if(source="reject", _time, null())
| eval time4=if(source="received", _time, null())
| stats values(time1) AS time1 values(time2) AS time2 values(time3) AS time3 values(time4) AS time4 by ID
0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

Something like this: obviously you will have to put in the details of your indexes, sourcetypes, etc.

<your index(es) and sourcetype(s)>
| eval time1=if(source is 1, _time, null())
| eval time2=if(source is 2, _time, null())
| eval time3=if(source is 3, _time, null())
| eval time4=if(source is 4, _time, null())
| stats values(time1) as time1 values(time2) as time2 values(time3) as time3 values(time4) as time4 values(description) as description by docID
Get Updates on the Splunk Community!

Cloud Platform & Enterprise: Classic Dashboard Export Feature Deprecation

As of Splunk Cloud Platform 9.3.2408 and Splunk Enterprise 9.4, classic dashboard export features are now ...

Explore the Latest Educational Offerings from Splunk (November Releases)

At Splunk Education, we are committed to providing a robust learning experience for all users, regardless of ...

New This Month in Splunk Observability Cloud - Metrics Usage Analytics, Enhanced K8s ...

The latest enhancements across the Splunk Observability portfolio deliver greater flexibility, better data and ...