Consolidating data from different indexes into one table

[jonthree]

I have 4 applications integrated with each other -  their names let's say A, B, C, D respectively. All these applications have different index, sourcetype and source. Now, when i call my API from application A, it traverse to B, C and D to perform some operations. I want to trace the error logs of all these applications at one location in splunk. Also, if an error occurs, there is a common reference id which is logged in all the application's logs respectively.

How to achieve this ? Is there a straightforward concept for this type of scenario?

What i am imagining is like this :

Ref ID	Application A Logs	Application B  Logs	Application C  Logs	Application D  Logs
1234	<logs of app A for id 1234>	<logs of app B for id 1234>	<logs of app C for id 1234>	<logs of app D for id 1234>
4567	<logs of app A for id 4567>	<logs of app B for id 4567>	<logs of app C for id 4567>	<logs of app D for id 4567>

Or is there a better way of doing this? I am trying to create a dashboard panel with this result.

[jonthree]

Thanks @scelikok .

Actually I want to see the error logs. And it is a little bit complex. There are different format of logs in different application. What i was trying to do was this:

Example:  error log for a particular API call is :

In Application A:

10.0.9.456 - - 23/Mar/2021:17:29:52 +0000 "POST /abc/test/performuse HTTP/1.1"  dest tenant1.myapp.com status 400 refid 1615 msg Some bad request error occured in application A

In Application B:

10.0.9.456 - - 23/Mar/2021:17:29:52 +0000 "POST Error occured in Application C status 400 bad request refid 1615 msg Some bad request error occured in application B.

Application C:

{"app":"myapp","timestamp":"2021-03-23T18:44:21.695Z","eventName":"My-event","component":"My-component","response":{"status":"FAILED","statusCode":"400", "refId":"1615","msg": "something went wrong"}

I am using rex to fetch ref id from application A & B. So i am tracing the propagating logs here. So is there any way i can add the logic to fetch refids in the query mentioned.

Apart from consolidated panel, I have tried to use subsearch to make different panels also, like :  index=*applicationB* [search index=*applicationA* rex <some expression> | fields refid] . It gives me the logs of Application B propagated from A.

Now how do i subsearch this refid in application C(as it is in json format) from the data i received from application B?  

This does not worked for me:

index=*applicationC*[search index=*applicationB* [search index=*applicationA* rex <some expression> | fields refid] 
| rex <some expression> | fields rexid statusCode]  

[scelikok]

Hi @jonthree,

If you want to see the _raw logs you can use below,

 index=A OR index=B OR index=D OR index=D
| chart values(_raw) over ref_id by index