Help with REX command

mdyunusraza — Wed, 31 Aug 2022 05:39:09 GMT

Hi,

I want to create a table from the sample log file entry by computing the field names based on the entries defined in the JSON structure. There will be multiple filed names and not just one.

e.g. in, the JSON structure, it has entries like "something":"value"

"something" will be the field name, and "value" will be the value that will form the table entries.

By working in https://regex101.com I have got the regex query that is doing the job. However, when I try to put that in the Splunk search query, it does not like the "]" in the regex query I have generated.

This is the regex query: "((?:[^"\\\/\b\f\n\r\t]|\\u\d{4})*)"

Query in Splunk | rex "((?:[^"\\\/\b\f\n\r\t]|\\u\d{4})*)"

Error in Splunk : Error in 'SearchParser': Mismatched ']'.

This is the sample log:

-------------------

2022/08/31 04:33:10.897 | server| service| INFO | 1-223 |x.x.x.x.x.Payload | xxx-1111-1111111-11-111111111 | AAt: Update Headers: {AAgid=ID:jaaana-11111-1111111111111-3:487:1:1:50, cccc_ff_ssss=ABC_XYZ, ssssdel=false, cdmode=1, DelMode=2, abc_corel_id=xyx-11111-11111-11-111111, aa_rrr_cccc_cccc=AAAA, cust_svc_id=AAAA-DDD, crumberid=xyx-11111-11111-11-111111, svc_tran_origin=SSS, SSScoreed=Camel-SSS-1111-1111111-111, cccc_ff_ssss_aaaaa=AAAA, AAAType=null, cccc_ff_ssss_tata=AAA, AAAexxxx=0, avronnnn=url.add.add.com, AAAssssssss=1661920390882,tang_dik_jagah=ABC_XYZ, ver=0.1.2, AAAprrrrrr=4, AAArptooo=null, source_DOT_adaptr=mom, AAAjaaana=tAAic://toic,tang_dik_jagah_tata=AAA, targCTService=progr, SSScoreedAsBytes=[a@123, CamelAAARequestTimeout=600000, sedaTimeout=600000} {[{"type":"AAtiongo","pAAo":"AAAA","ssssssss":"2022-08-31 00:00:00","data":[{"chabbbi":"ca_1111_11111_AAtiongo_AAAA","tatajahajqaki":"AA 111","jahajqaki":{"numeo":"111","jahaaj":{"cde":"ARL_AA","couAAa":"AA","aaoo":"AAR"},"AAsuf":null},"sgnnnn":"AAR111","stppp":"J","muddStatuscde":"AA","kissak":"III","AAType3lc":"111","AAType5lc":"B111","rggggggg":"AAAAA","carrrrr":{"cde":"ARL_AA","couAAa":"AA","aaoo":"AAR"},"ddddddcde":"pubbb","pubbbjahajqaki":"AA 111","jahajqakipubbb":{"numeo":"111","jahaaj":{"cde":"AA","couAAa":null,"aaoo":null}},"sssss":1098,"kkkkkss":834,"kitnaba":{"AAAAAA":"2022-08-2100:00:00","WWWW":"2022-08-2100:00:00","eeeeee":"2022-08-2100:00:00","sssssss":"2022-08-2100:00:00","ddddddd":"2022-08-2100:00:00","eeeeeeee":"2022-08-2100:00:00","ddddddddd":"2022-08-2100:00:00","ttttttt":"2022-08-2100:00:00","ttttttt":"2022-08-2100:00:00","Edddddd":"2022-08-2100:00:00","ffffff":"2022-08-2100:00:00","ddddddL":"2022-08-2100:00:00","dddddd":"2022-08-2100:00:00","Adddddd":"2022-08-2100:00:00","ssssT":"2022-08-2100:00:00","ddddd":"2022-08-2100:00:00","ggggg":"2022-08-2100:00:00","ffffff":"2022-08-2100:00:00","Eddddd":"2022-08-2100:00:00","ssssss":"2022-08-2100:00:00","Eddddd":"2022-08-2100:00:00"},"durdddd":{"Exxxxx":"Pdddd.oo","ScfffTTTT":"xxx1H0M0.000S","xxxxIDL":"-Pxxxx6M0.000S","ESTTTT":"PxxxxH26M0.000S"},"gallle":[{"aaaaaaa":"aaa000033","gffffnnnn":"111"}],"stsssss":[{"hhhhhh":"AA1111111","standnnnn":"S20"}],"blttttt":[{"hhhhhh":"ABB000003","beltnnnn":"aa11","beltAAenpttttt":"2022-08-2100:00:00","kkkkkkkpttttt":"2022-08-2100:00:00"}],"redddddd":{"SSSSS":[{"aalllll":"ALLUU99999","resssssss":"AA1111111","resssssssnnnn":"S20","pprrrrrsssss":"AAA11111"}],"bgggg_blt":[{"aalllll":"aaaaaa1111111","resssssss":"ABB000003","resssssssnnnn":"IB02","kitnaba":{"AAAAAA":"2022-08-31006:14:00a","AAAAAA":"2022-08-31006:14:00a"}}],"aaaaaaaaaaa_sss":[{"aalllll":"aaaaaa8888888","resssssss":"false"}],"aaaaaaaaaa_ssss":[{"aalllll":"aaaaaa8888888","resssssss":"GAT000033","resssssssnnnn":"120","pprrrrrsssss":"GAT000019"}],"qqqqqqqqqqqq":[{"aalllll":"qqqqqqqqqqqq","resssssss":"false"}]},"kkkkkk":[{"cde":"aaa_sss","tatAAde":"CAI","aaaaAAde":"PPPP","legnumeo":1},{"cde":"ABC_XYZ","tatAAde":"AAA","aaaaAAde":"AAAA","legnumeo":2}],"cdeshareList":[{"numeo":"1111","jahaaj":{"cde":"ARL_AA","couAAa":"AA","aaoo":"AAA"},"AAsuf":null,"pubbbjahajqaki":"AA 1111","jahajqakipubbb":{"numeo":"1111","jahaaj":{"cde":"AA","couAAa":null,"aaoo":null}}},{"numeo":"1111","jahaaj":{"cde":"ARL_CT","couAAa":"CT","aaoo":"CTH"},"AAsuf":null,"pubbbjahajqaki":"CT 1111","jahajqakipubbb":{"numeo":"1111","jahaaj":{"cde":"CT","couAAa":null,"aaoo":null}}}],"saaaaaa":{"ffff":"RRR","mapr":"Finalised","SSSGeneral":"AAened","AAceptance":"Finalised","loddacctrr":"SheCT_Finalised","brrrrrrdd":"AAened","IIIernal":"110"}}]}]}
host = mucAAuplfrAA02

-----------------------

Re: Help with REX command

chaker — Wed, 31 Aug 2022 05:59:44 GMT

Hi @mdyunusraza ,

Try escaping those bracket symbols you are matching with a backslash.

| rex "((?:\[^"\\\/\b\f\n\r\t\]|\\u\d{4})*)"

Re: Help with REX command

gcusello — Wed, 31 Aug 2022 06:04:42 GMT

Hi @mdyunusraza,

this is a json log so did you tried the spath command (https://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Spath) to extract fields?

Ciao.

Giuseppe

Re: Help with REX command

chaker — Wed, 31 Aug 2022 06:06:50 GMT

Also review the docs for the rex command. It uses named capture groups for the extracted field names.

https://docs.splunk.com/Documentation/Splunk/9.0.1/SearchReference/Rex

Re: Help with REX command

mdyunusraza — Wed, 31 Aug 2022 06:11:21 GMT

@chaker ,

Yes, I tried that, but it gives me another error, like so,

Error in 'SearchParser': Missing a search command before '\'. Error at position '100' of search query 'search index=indname "service" "AAt: Upd...{snipped} {errorcontext = f\n\r\t\]|\\u\d{4})*)}'.

Re: Help with REX command

mdyunusraza — Wed, 31 Aug 2022 06:13:23 GMT

@gcusello

No, I have not tried that. We have used the REX command to extract fields using regex, but those were not JSON logs. We have this new log we need to dissect and form a table. Hence the requirement.

I will read about SPATH and see how it goes.

topic Re: Help with REX command in Splunk Search

Help with REX command

Re: Help with REX command

Re: Help with REX command

Re: Help with REX command

Re: Help with REX command

Re: Help with REX command