Dashboards & Visualizations

How to fetch the values from raw logs

aditsss
Motivator

Hi All,

Below is my raw log and I want to fetch the highlighted value from it:

2023-08-08 10:25:48.407 [INFO ] [Thread-3] CollateralProcessor - compareCollateralStatsData : statisticData: StatisticData [selectedDataSet=0, rejectedDataSet=0, totalOutputRecords=0, totalInputRecords=0, fileSequenceNum=0, fileHeaderBusDt=null, busDt=08/06/2023, fileName=SETTLEMENT_TRANSFORM_COLLATERAL_LENDING, totalAchCurrOutstBalAmt=2.722379487286E10, totalAchBalLastStmtAmt=2.722379487286E10, totalClosingBal=2.722379487286E10, sourceName=null, version=0, associationStats={}] with collateralSum 2.722379487286E10 openingBal 2.73215647389E10 ageBalTot 2.722379487286E10 busDt 08/07/2023 with prevStatisticData null

Below is my query but I am not able to fetch that:

index="abc*" sourcetype =600000304_gg_abs_ipc2 source="/amex/app/gfp-settlement-transform/logs/gfp-settlement-transform.log" "CollateralProcessor - compareCollateralStatsData"|rex "CollateralProcessor - compareCollateralStatsData busDt=(?<busDt>),fileName=(?<fileName>),collateralSum =(?<collateralSum>)"|table busDt fileName collateralSum | sort busDt

Can someone guide me how I can fetch highlight

Labels (3)
0 Karma

bowesmana
SplunkTrust
SplunkTrust

@aditsss when posting code snippets, please use the code tag </> to make your code easier to read. Also it helps to split the command pipes on to separate lines for ease of reading - you can do this with the Ctrl-\ character when in the Splunk UI SPL editor

0 Karma

isoutamo
SplunkTrust
SplunkTrust

Hi

for me it seems to work as @smurf suggested with your test data. See below

 

| makeresults
| eval _raw = "2023-08-08 10:25:48.407 [INFO ] [Thread-3] CollateralProcessor - compareCollateralStatsData : statisticData: StatisticData [selectedDataSet=0, rejectedDataSet=0, totalOutputRecords=0, totalInputRecords=0, fileSequenceNum=0, fileHeaderBusDt=null, busDt=08/06/2023, fileName=SETTLEMENT_TRANSFORM_COLLATERAL_LENDING, totalAchCurrOutstBalAmt=2.722379487286E10, totalAchBalLastStmtAmt=2.722379487286E10, totalClosingBal=2.722379487286E10, sourceName=null, version=0, associationStats={}] with collateralSum 2.722379487286E10 openingBal 2.73215647389E10 ageBalTot 2.722379487286E10 busDt 08/07/2023 with prevStatisticData null 
2023-08-09 10:31:57.834 [INFO ] [Thread-3] CollateralProcessor - compareCollateralStatsData : statisticData: StatisticData [selectedDataSet=0, rejectedDataSet=0, totalOutputRecords=0, totalInputRecords=0, fileSequenceNum=0, fileHeaderBusDt=null, busDt=08/07/2023, fileName=SETTLEMENT_TRANSFORM_COLLATERAL_LENDING, totalAchCurrOutstBalAmt=2.71092692285E10, totalAchBalLastStmtAmt=2.71092692285E10, totalClosingBal=2.71092692285E10, sourceName=null, version=0, associationStats={}] with collateralSum 2.71092692285E10 openingBal 2.722379487286E10 ageBalTot 2.71092692285E10 busDt 08/08/2023 with prevStatisticData null"
| multikv noheader=t
```previous make test events```
| rex "busDt=(?<busDt>.*?),\sfileName=(?<fileName>.+?),.*collateralSum\s(?<collateralSum>[\d|\.|E]+)\sopeningBal\s(?<openingBal>[\d|\.|E]+)\sageBalTot\s(?<ageBalTot>[\d|\.|E]+)"
| table busDt fileName collateralSum openingBal ageBalTot

 

This shows those two events with required fields.

Works even you change that regex to 

| rex "busDt=(?<busDt>.*?),\sfileName=(?<fileName>.+?),.*collateralSum\s(?<collateralSum>[\d\.E]+)\sopeningBal\s(?<openingBal>[\d\.E]+)\sageBalTot\s(?<ageBalTot>[\d\.E]+)"

You shouldn't use "|" as a separator inside "[]".

r. Ismo 

0 Karma

smurf
Communicator

Hi,

your regex seems wrong, so it does not extract anything. I recommend using something like regex101 to test your regex.

 

Try replacing it with this (this should work with your sample event):

| rex "busDt=(?<busDt>.*?),\sfileName=(?<fileName>.+?),.*collateralSum\s(?<collateralSum>[\d|\.|E]+)\sopeningBal\s(?<openingBal>[\d|\.|E]+)\sageBalTot\s(?<ageBalTot>[\d|\.|E]+)"

 

Hope this helps.

smurf

aditsss
Motivator

@smurf 

I used this:

index="abc*" sourcetype =600000304_gg_abs_ipc2 source="/amex/app/gfp-settlement-transform/logs/gfp-settlement-transform.log" "CollateralProcessor - compareCollateralStatsData"| rex "busDt=(?<busDt>.*?),\sfileName=(?<fileName>.+?),.*collateralSum\s(?<collateralSum>[\d|\.|E]+)\sopeningBal\s(?<openingBal>[\d|\.|E]+)\sageBalTot\s(?<ageBalTot>[\d|\.|E]+)"|table busDt fileName collateralSum openingBal

But not able to see openingBal and collateralSum

Below screenshot:

newscreenshot.PNG

raw logs:

2023-08-09 10:31:57.834 [INFO ] [Thread-3] CollateralProcessor - compareCollateralStatsData : statisticData: StatisticData [selectedDataSet=0, rejectedDataSet=0, totalOutputRecords=0, totalInputRecords=0, fileSequenceNum=0, fileHeaderBusDt=null, busDt=08/07/2023, fileName=SETTLEMENT_TRANSFORM_COLLATERAL_LENDING, totalAchCurrOutstBalAmt=2.71092692285E10, totalAchBalLastStmtAmt=2.71092692285E10, totalClosingBal=2.71092692285E10, sourceName=null, version=0, associationStats={}]

with collateralSum 2.71092692285E10 openingBal 2.722379487286E10 ageBalTot 2.71092692285E10 busDt 08/08/2023 with prevStatisticData null

raw.PNG

 

 

0 Karma
Get Updates on the Splunk Community!

Dashboards: Hiding charts while search is being executed and other uses for tokens

There are a couple of features of SimpleXML / Classic dashboards that can be used to enhance the user ...

Splunk Observability Cloud's AI Assistant in Action Series: Explaining Metrics and ...

This is the fourth post in the Splunk Observability Cloud’s AI Assistant in Action series that digs into how ...

Brains, Bytes, and Boston: Learn from the Best at .conf25

When you think of Boston, you might picture colonial charm, world-class universities, or even the crack of a ...