Logs with disorganized data- How to view logs for complete processes and not for fractions of these?

[Kleydert]

Hii, good day everyone

I need your help please. I need to join a log that gives me the events by date, but I require it from the beginning of the event, to the end of it (the event begins when a card is entered and ends when the mentioned card is extracted)

I have tried to do it by means of Regex from the sourcetype, but not success.

[Kleydert]

I need to put all the paragraphs together, to be able to see the entire process registered in each use case. I can only see all the events that are associated with a CardId (paragraphs in green), but I need to see the entire use case filtering by the CardId and not by the host (as I'm doing at the moment), since it brings me all the information of all use cases and I only need them by CardId.

The ideal is to be able to join all the paragraphs (the amount varies, depending on whether returns or other types of events have to be made), for each use case in particular, to which a single CardId corresponds. As I mentioned, each method starts with reading the card, brings up various events, and ends when a new card is registered.
Please help me, I've been trying to solve this for about 5 weeks without success

[richgalloway]

What common field/value connects related events ("paragraphs")?  Splunk needs to see something in events that tells it they belong together.

[Kleydert]

Not all fields have a value in common. In the following image you can see a paragraph with the IdCard of the use case, but in the next paragraph, said IdCard is not shown, despite the fact that both paragraphs belong to the same use case. In short, I need to be able to observe all the paragraphs belonging to a single use case, regardless of whether or not they have a common field. All use cases start by reading a particular IdCard and end when a new IdCard is entered, but not all paragraphs have the IdCard field.

[richgalloway]

Perhaps this will help.  I found a way to distribute the CardId value to events that don't have one.  It assumes, however, that transactions are sequential and not interleaved.

| makeresults | eval data="2022-10-26 05:01:22.861 -05:00 [INF] RESPONSE:RechargeCardBalanceController/ SetStartTransaction - SetStartTransaction => StatusTransaction: WAITING_CASH, UseEscrow: False
2022-10-26 05:01:22.025 -05:00 [INF] REQUEST:RechargeCardBalanceController/ SetStartTransaction - SetStartTransaction {\"IdCard\":\"119\",\"CurrentBalance\":0,\"TransactionValue\":50,\"UseEscrow\":false,\"SessionNumber\":\"134\",\"IncludeBuyingSmartCard\":false,\"SmartCardCost\":0,\"AutoChange\":false}
2022-10-26 05:01:20.953 -05:00 [INF] --- 1.Notice: --- MachineType: 0
2022-10-26 05:01:19.449 -05:00 [INF]Reading CardId: 119
2022-10-26 05:01:18.861 -05:00 [INF] RESPONSE:RechargeCardBalanceController/ SetStartTransaction - SetStartTransaction => StatusTransaction: WAITING_CASH, UseEscrow: False
2022-10-26 05:01:18.025 -05:00 [INF] REQUEST:RechargeCardBalanceController/ SetStartTransaction - SetStartTransaction {\"IdCard\":\"118\",\"CurrentBalance\":0,\"TransactionValue\":50,\"UseEscrow\":false,\"SessionNumber\":\"133\",\"IncludeBuyingSmartCard\":false,\"SmartCardCost\":0,\"AutoChange\":false}
2022-10-26 05:01:15.953 -05:00 [INF] --- 1.Notice: --- MachineType: 0
2022-10-26 05:01:14.449 -05:00 [INF]Reading CardId: 118"| eval data=split(data,"
") | mvexpand data | eval _raw=data | fields - data 
```Above creates demo data.  Delete IRL```
```Put events in chronological order```
| reverse
```Extract CardId field```
| rex "CardId: (?<CardId>\d+)"
```Add CardId field to events that don't have one```
| filldown CardId

Not the use of the reverse command to put events in ascending order by _time (Splunk returns them in descending order by default).

[Kleydert]

OMG!! Apparently this could be the solution. The previous paragraphs belong to a log called "MTLogData:4G"; I consult it in splunk in this way: sourcetype="MT4G:MTLogData:4G", but I have not been able to do the query correctly, I need it to take the log and not the text that it is searched for. Please, can you help me to make the search directly from the mentioned log and not on a text fragment? I've been trying for several hours, but I don't know what I'm doing wrong.

Thx

[Kleydert]

Can you teach me how to do it? please, I've been trying all day, in various ways, but I can't get it right, I'm still a newbie using Splunk, I've only been using it for a few months.

Thank you for your patience

[richgalloway]

I guess my instructions weren't clear enough.  In the SPL I provided, REMOVE EVERYTHING UP TO AND INCLUDING 

```Above creates demo data.  Delete IRL```

Then insert your code for reading MTLogData:4G log.

[richgalloway]

Replace the parts of my query from | makereults through and including 'IRL```' with the SPL that reads the MTLogData:4G log.

[Kleydert]

I have done it this way, among others, but I still have no success, the first "eval data=" is the one that is stopping me, I don't know how to do the query properly in this part

[Kleydert]

This is a typical use case thrown by the mentioned Log:

2022-10-26 05:01:14.449 -05:00 [INF]Reading CardId: 118

2022-10-26 05:01:15.953 -05:00 [INF] --- 1.Notice: --- MachineType: 0

2022-10-26 05:01:18.025 -05:00 [INF] REQUEST:RechargeCardBalanceController/ SetStartTransaction - SetStartTransaction
{"IdCard":"118","CurrentBalance":0,"TransactionValue":50,"UseEscrow":false,"SessionNumber":"133","IncludeBuyingSmartCard":false,"SmartCardCost":0,"AutoChange":false}

2022-10-26 05:01:18.861 -05:00 [INF] RESPONSE:RechargeCardBalanceController/ SetStartTransaction - SetStartTransaction => StatusTransaction: WAITING_CASH, UseEscrow: False

[richgalloway]

Can you tell us more about this use case?  What you describe sounds like a transaction, but the transaction command is costly so it would be better to use alternatives.

FTR, using the transaction command might look like this

index=foo
| transaction startswith=<<card is entered>> endswith=<<card is extracted>>