Each API call has a unique transactionID. The api call goes thru various indexes. How do i track the progress of the call (ie when the call enters and leaves each index)?
The problem is made harder by the fact that sometimes a call goes from index1 to index2, returns to index1, then goes to index 3.....and so on
First question, why are you having the data go to different indexes? I'm assuming your segregating the data based off the application its coming from?
Now to solve your problem, you can simply include all 3 indexes in your initial search like this
index=index1 OR index=index2 OR index=index3
This will search all 3 indexes and bring back data which you can than massage into a search to track the transaction
Distributed apps - hence the different indexes
That solution does not help out if we have back and forth calls between indexes
I'm doing the following
|stats earliest(_time) as starttime, latest(_time) as stoptime by index |sort starttime
This gives me
index
index1 1545851255 1545851259
index2 1545851256 1545851257
index3 1545851257 1545851258
This does not show each entry and exit into indexes...
I would like the output to be something like
index1 1545851255 1545851256
index2 1545851256 1545851257
index3 1545851257 1545851258
index1 1545851258 1545851259
any ideas??
Does each app require a different retention period for logging OR require different user access controls? If not then this should be in a single index, regardless if they are from different apps.
The solution DOES help if they are going from one index to another. The best way to think about this is, if you use the solution I suggested, your returning results from 3 different indexes into a single view. Index is irrelevant then at this point. Fields will be relative to sourcetype, so you will have identical fields across multiple indexes assuming your using the same sourcetype. You then join on that field which allows you to track the session.
If there's a particular order they come in, you can write the logic to account for that order. If not, then do it based off _time. You can structure each transaction relative to _time than coalesce
them together to group the transactions
How do i group them using coalesce
say my data is
12:00:00 app=A,
12:00:01 app=A
12:00:02 app=B
12:00:03 app=B
12:00:04 app=A
12:00:05 app=A,
12:00:06 app=C
12:00:07 app=C
12:00:08 app=A
12:00:09 app=A
Coalesce
will merge multiple fields values into a single field. You have 1 field called app
with multiple values.. You should provide actual sample data if you want an accurate solution as you're not showing the whole picture here..
Another solution would be to add a unique guid to track the transactions.
here is the real data (i've removed the actual app and guid names and moved all the calls into 1 index. The question now becomes how I show when the call enters and leaves each app
01/08/2019 16:49:11.045 app=A, guid=12345678, event=startAPI
01/08/2019 16:49:11.046 app=A, guid=12345678, event=endAPI
01/08/2019 16:49:11.047 app=B, guid=12345678, event=startAPI
01/08/2019 16:49:11.055 app=B, guid=12345678, event=endAPI
01/08/2019 16:49:11.056 app=A, guid=12345678, event=startAPI
01/08/2019 16:49:11.062 app=A, guid=12345678, event=endAPI
01/08/2019 16:49:11.063 app=C, guid=12345678, event=startAPI
01/08/2019 16:49:11.073 app=C, guid=12345678, event=endAPI
01/08/2019 16:49:11.074 app=A, guid=12345678, event=startAPI
01/08/2019 16:49:11.090 app=A, guid=12345678, event=endAPI
Sorry.....prior data was wrong
01/08/2019 16:49:11.045 app=A, guid=12345678, event=startAPI
01/08/2019 16:49:11.046 app=A, guid=12345678, event=endAPI
01/08/2019 16:49:11.047 app=B, guid=12345678, event=startAPI
01/08/2019 16:49:11.055 app=B, guid=12345678, event=endAPI
01/08/2019 16:49:11.056 app=A, guid=12345678, event=startAPI
01/08/2019 16:49:11.062 app=A, guid=12345678, event=endAPI
01/08/2019 16:49:11.063 app=C, guid=12345678, event=startAPI
01/08/2019 16:49:11.073 app=C, guid=12345678, event=endAPI
01/08/2019 16:49:11.074 app=A, guid=12345678, event=startAPI
01/08/2019 16:49:11.090 app=A, guid=12345678, event=endAPI
and yes, retention period is the same