How to extract particular string in the data?

[pench2k19]

Hi Team,

I m planning to collect the highlited text from the raw data as below

info : Detailed logging to /apps/dat/aasconap/prod/admin/cnapp/cnapp_src/cnapp_src_us/log/./cnapp_generic_reformat_control_file_2019-04-10-06-35-06_**10471**.log
info : Detailed logging to /apps/dat/aasconap/prod/admin/cnapp/cnapp_src/cnapp_src_us/log/./eapp_generic_publish_status_2019-04-10-06-35-11_11311.log
info : Detailed logging to /apps/dat/aasconap/prod/admin/cnapp/cnapp_src/cnapp_src_us/log/./aiblk_linear_framework_us__msa104_gl_txn__feed_2019-04-10-06-35-58_18695.log
info : Detailed logging to /apps/dat/aasconap/prod/admin/cnapp/cnapp_src/cnapp_src_us/log/./cnapp_process_acqit_log_files_2019-04-10-06-43-49_4398.log
info : Detailed logging to /apps/dat/aasconap/prod/admin/cnapp/cnapp_src/cnapp_src_us/log/./eapp_generic_publish_status_2019-04-10-06-44-21_8468.log

Can you please help me wit regex expression for the same . thank you

@vnravikumar @jkat54

[woodcock]

Try this:

|makeresults | eval _raw="apps/dat/aasconap/prod/admin/cnapp/cnapp_src/cnapp_src_us/log/./cnapp_generic_reformat_control_file_2019-04-10-06-35-06_10471.log
info : Detailed logging to /apps/dat/aasconap/prod/admin/cnapp/cnapp_src/cnapp_src_us/log/./eapp_generic_publish_status_2019-04-10-06-35-11_11311.log
info : Detailed logging to /apps/dat/aasconap/prod/admin/cnapp/cnapp_src/cnapp_src_us/log/./aiblk_linear_framework_us_msa104_gl_txn_feed_2019-04-10-06-35-58_18695.log
info : Detailed logging to /apps/dat/aasconap/prod/admin/cnapp/cnapp_src/cnapp_src_us/log/./cnapp_process_acqit_log_files_2019-04-10-06-43-49_4398.log
info : Detailed logging to /apps/dat/aasconap/prod/admin/cnapp/cnapp_src/cnapp_src_us/log/./eapp_generic_publish_status_2019-04-10-06-44-21_8468.log"
| rex max_match=1 "_\d{4}(?:-\d{2}){5}_(?<log_number>\d+)\.log"

[pench2k19]

thanks @woodcock ...but i cant use makeresults command in my query...do you have any alternative way to get this

[woodcock]

The makeresults was to generate fake events to test your solution, which is only the last line.

[harsmarvania57]

Hi,

Please try below regex, it will extract highlighted value in new field called ext_value

<yourBaseSearch> | rex field=_raw "_\d{4}-\d{2}-\d{2}-\d{2}-\d{2}-\d{2}_(?<ext_value>\d+)\.log"

[pench2k19]

thanks for the resply @harsmarvania57 ....its matching with all the rows , but i need to extract the value only from first row.

[harsmarvania57]

Sample data which you have provided is single event only or those are different events ?

[pench2k19]

its from single event.

[harsmarvania57]

Try this | rex field=_raw max_match=1 "^(?s)(?:[^\/]*[\/]){11}([^\d]*)\d{4}-\d{2}-\d{2}-\d{2}-\d{2}-\d{2}_(?<ext_value>\d+)\.log"

[pench2k19]

its not working @harsmarvania57 marvania...actually my raw data is like this and its coming as single event... i need to extract the hightlighted value

[2019-04-15 06:12:26] Plan File: /apps/src/aasconap/prod/abinitio/cnapp/cnapp_src/cnapp_src_msp/pset/planpset/processing_plan.msp_master_708_936.pset
[2019-04-15 06:12:26] Recovery File: /apps_run_aasconap/prod/processing_plan.mspmaster_708_936.rec
[2019-04-15 06:12:26] Beginning plan '/'
[2019-04-15 06:12:28] Method '/Get RUN_ID/perform' changed parameter 'RUN_ID' from '' to '28090'
[2019-04-15 06:12:43] Standard Output for '/Standardize control file/perform':
info : ++++ STARTED ++++ Job cnapp_generic_reformat_control_file_2019-04-15-06-12-43_7803
info : Central logging to /apps/dat/aasconap/prod/admin/log/environment_operations_2019_04.log
info : Raw tracking to /apps/dat/aasconap/prod/admin/cnapp/cnapp_src/cnapp_src_msp/tracking/./cnapp_generic_reformat_control_file_2019-04-15-06-12-43_7803.tracking
info : Input pset archived to /apps/dat/aasconap/prod/admin/cnapp/cnapp_src/cnapp_src_msp/parameter/./cnapp_generic_reformat_control_file_2019-04-15-06-12-43_7803.pset
info : Summary is not being collected
info : Error logging to /apps/dat/aasconap/prod/admin/cnapp/cnapp_src/cnapp_src_msp/error/./cnapp_generic_reformat_control_file_2019-04-15-06-12-43_7803.err
info : Duplicating stderr
info : Detailed logging to /apps/dat/aasconap/prod/admin/cnapp/cnapp_src/cnapp_src_msp/log/./cnapp_generic_reformat_control_file_2019-04-15-06-12-437803.log
[2019-04-15 06:12:46] Standard Output for '/Standardize control file/perform':
info : ++++ COMPLETED ++++ Job cnapp_generic_reformat_control_file_2019-04-15-06-12-43_7803
[2019-04-15 06:12:47] Method '/Set dynamic plan variables from control file/perform' changed parameter 'EFF_DATE' from '2019-04-14' to '2019-04-14'
[2019-04-15 06:12:48] Method '/Set dynamic plan variables from control file/perform' changed parameter 'DATA_READ_LOCATION' from '' to 'hdfs:/datalake/consumer/msp/raw/tmp/MSP_DELTA_PR708_936_MASTER_190414'
[2019-04-15 06:12:48] Method '/Set dynamic plan variables from control file/perform' changed parameter 'REC_CNT' from '' to '580157'
[2019-04-15 06:12:48] Method '/Set dynamic plan variables from control file/perform' changed parameter 'CNAPP_PUB_KEY_REG_PG' from 'PG777' to '708_936'
[2019-04-15 06:12:48] Standard Output for '/Set dynamic plan variables from control file/perform':
Successfully validated effective date format from control file (value = 2019-04-14)
[2019-04-15 06:12:51] Standard Output for '/Publish module start metadata/perform':
info : ++++ STARTED ++++ Job eapp_generic_publish_status_2019-04-15-06-12-50_8647
info : Central logging to /apps/dat/aasconap/prod/admin/log/environment_operations_2019_04.log
info : Raw tracking to /apps/dat/aasconap/prod/admin/cnapp/cnapp_src/cnapp_src_msp/tracking/./eapp_generic_publish_status_2019-04-15-06-12-50_8647.tracking
info : Input pset archived to /apps/dat/aasconap/prod/admin/cnapp/cnapp_src/cnapp_src_msp/parameter/./eapp_generic_publish_status_2019-04-15-06-12-50_8647.pset
info : Summary is not being collected
info : Error logging to /apps/dat/aasconap/prod/admin/cnapp/cnapp_src/cnapp_src_msp/error/./eapp_generic_publish_status_2019-04-15-06-12-50_8647.err
info : Duplicating stderr
info : Detailed logging to /apps/dat/aasconap/prod/admin/cnapp/cnapp_src/cnapp_src_msp/log/./eapp_generic_publish_status_2019-04-15-06-12-50_8647.log

[pench2k19]

@FrankVl can you help me here.