Dashboards & Visualizations

How to fetch the values from raw logs

aditsss
Motivator

Hi All,

Below is my raw log and I want to fetch the highlighted value from it:

2023-08-08 10:25:48.407 [INFO ] [Thread-3] CollateralProcessor - compareCollateralStatsData : statisticData: StatisticData [selectedDataSet=0, rejectedDataSet=0, totalOutputRecords=0, totalInputRecords=0, fileSequenceNum=0, fileHeaderBusDt=null, busDt=08/06/2023, fileName=SETTLEMENT_TRANSFORM_COLLATERAL_LENDING, totalAchCurrOutstBalAmt=2.722379487286E10, totalAchBalLastStmtAmt=2.722379487286E10, totalClosingBal=2.722379487286E10, sourceName=null, version=0, associationStats={}] with collateralSum 2.722379487286E10 openingBal 2.73215647389E10 ageBalTot 2.722379487286E10 busDt 08/07/2023 with prevStatisticData null

Below is my query but I am not able to fetch that:

index="abc*" sourcetype =600000304_gg_abs_ipc2 source="/amex/app/gfp-settlement-transform/logs/gfp-settlement-transform.log" "CollateralProcessor - compareCollateralStatsData"|rex "CollateralProcessor - compareCollateralStatsData busDt=(?<busDt>),fileName=(?<fileName>),collateralSum =(?<collateralSum>)"|table busDt fileName collateralSum | sort busDt

Can someone guide me how I can fetch highlight

Labels (3)
0 Karma

bowesmana
SplunkTrust
SplunkTrust

@aditsss when posting code snippets, please use the code tag </> to make your code easier to read. Also it helps to split the command pipes on to separate lines for ease of reading - you can do this with the Ctrl-\ character when in the Splunk UI SPL editor

0 Karma

isoutamo
SplunkTrust
SplunkTrust

Hi

for me it seems to work as @smurf suggested with your test data. See below

 

| makeresults
| eval _raw = "2023-08-08 10:25:48.407 [INFO ] [Thread-3] CollateralProcessor - compareCollateralStatsData : statisticData: StatisticData [selectedDataSet=0, rejectedDataSet=0, totalOutputRecords=0, totalInputRecords=0, fileSequenceNum=0, fileHeaderBusDt=null, busDt=08/06/2023, fileName=SETTLEMENT_TRANSFORM_COLLATERAL_LENDING, totalAchCurrOutstBalAmt=2.722379487286E10, totalAchBalLastStmtAmt=2.722379487286E10, totalClosingBal=2.722379487286E10, sourceName=null, version=0, associationStats={}] with collateralSum 2.722379487286E10 openingBal 2.73215647389E10 ageBalTot 2.722379487286E10 busDt 08/07/2023 with prevStatisticData null 
2023-08-09 10:31:57.834 [INFO ] [Thread-3] CollateralProcessor - compareCollateralStatsData : statisticData: StatisticData [selectedDataSet=0, rejectedDataSet=0, totalOutputRecords=0, totalInputRecords=0, fileSequenceNum=0, fileHeaderBusDt=null, busDt=08/07/2023, fileName=SETTLEMENT_TRANSFORM_COLLATERAL_LENDING, totalAchCurrOutstBalAmt=2.71092692285E10, totalAchBalLastStmtAmt=2.71092692285E10, totalClosingBal=2.71092692285E10, sourceName=null, version=0, associationStats={}] with collateralSum 2.71092692285E10 openingBal 2.722379487286E10 ageBalTot 2.71092692285E10 busDt 08/08/2023 with prevStatisticData null"
| multikv noheader=t
```previous make test events```
| rex "busDt=(?<busDt>.*?),\sfileName=(?<fileName>.+?),.*collateralSum\s(?<collateralSum>[\d|\.|E]+)\sopeningBal\s(?<openingBal>[\d|\.|E]+)\sageBalTot\s(?<ageBalTot>[\d|\.|E]+)"
| table busDt fileName collateralSum openingBal ageBalTot

 

This shows those two events with required fields.

Works even you change that regex to 

| rex "busDt=(?<busDt>.*?),\sfileName=(?<fileName>.+?),.*collateralSum\s(?<collateralSum>[\d\.E]+)\sopeningBal\s(?<openingBal>[\d\.E]+)\sageBalTot\s(?<ageBalTot>[\d\.E]+)"

You shouldn't use "|" as a separator inside "[]".

r. Ismo 

0 Karma

smurf
Communicator

Hi,

your regex seems wrong, so it does not extract anything. I recommend using something like regex101 to test your regex.

 

Try replacing it with this (this should work with your sample event):

| rex "busDt=(?<busDt>.*?),\sfileName=(?<fileName>.+?),.*collateralSum\s(?<collateralSum>[\d|\.|E]+)\sopeningBal\s(?<openingBal>[\d|\.|E]+)\sageBalTot\s(?<ageBalTot>[\d|\.|E]+)"

 

Hope this helps.

smurf

aditsss
Motivator

@smurf 

I used this:

index="abc*" sourcetype =600000304_gg_abs_ipc2 source="/amex/app/gfp-settlement-transform/logs/gfp-settlement-transform.log" "CollateralProcessor - compareCollateralStatsData"| rex "busDt=(?<busDt>.*?),\sfileName=(?<fileName>.+?),.*collateralSum\s(?<collateralSum>[\d|\.|E]+)\sopeningBal\s(?<openingBal>[\d|\.|E]+)\sageBalTot\s(?<ageBalTot>[\d|\.|E]+)"|table busDt fileName collateralSum openingBal

But not able to see openingBal and collateralSum

Below screenshot:

newscreenshot.PNG

raw logs:

2023-08-09 10:31:57.834 [INFO ] [Thread-3] CollateralProcessor - compareCollateralStatsData : statisticData: StatisticData [selectedDataSet=0, rejectedDataSet=0, totalOutputRecords=0, totalInputRecords=0, fileSequenceNum=0, fileHeaderBusDt=null, busDt=08/07/2023, fileName=SETTLEMENT_TRANSFORM_COLLATERAL_LENDING, totalAchCurrOutstBalAmt=2.71092692285E10, totalAchBalLastStmtAmt=2.71092692285E10, totalClosingBal=2.71092692285E10, sourceName=null, version=0, associationStats={}]

with collateralSum 2.71092692285E10 openingBal 2.722379487286E10 ageBalTot 2.71092692285E10 busDt 08/08/2023 with prevStatisticData null

raw.PNG

 

 

0 Karma
Get Updates on the Splunk Community!

Splunk Observability Cloud’s AI Assistant in Action Series: Analyzing and ...

This is the second post in our Splunk Observability Cloud’s AI Assistant in Action series, in which we look at ...

Elevate Your Organization with Splunk’s Next Platform Evolution

 Thursday, July 10, 2025  |  11AM PDT / 2PM EDT Whether you're managing complex deployments or looking to ...

Splunk Answers Content Calendar, June Edition

Get ready for this week’s post dedicated to Splunk Dashboards! We're celebrating the power of community by ...