Splunk Search

Help with REX command

mdyunusraza
Observer

Hi,

I want to create a table from the sample log file entry by computing the field names based on the entries defined in the JSON structure. There will be multiple filed names and not just one. 

e.g. in, the JSON structure, it has entries like "something":"value"

"something" will be the field name, and "value" will be the value that will form the table entries.

By working in https://regex101.com I have got the regex query that is doing the job. However, when I try to put that in the Splunk search query, it does not like the "]" in the regex query I have generated.

 

This is the regex query: "((?:[^"\\\/\b\f\n\r\t]|\\u\d{4})*)"

Query in Splunk  | rex "((?:[^"\\\/\b\f\n\r\t]|\\u\d{4})*)"

Error in Splunk : Error in 'SearchParser': Mismatched ']'.

This is the sample log:

-------------------


2022/08/31 04:33:10.897 | server| service| INFO | 1-223 |x.x.x.x.x.Payload | xxx-1111-1111111-11-111111111 | AAt: Update Headers: {AAgid=ID:jaaana-11111-1111111111111-3:487:1:1:50, cccc_ff_ssss=ABC_XYZ, ssssdel=false, cdmode=1, DelMode=2, abc_corel_id=xyx-11111-11111-11-111111, aa_rrr_cccc_cccc=AAAA, cust_svc_id=AAAA-DDD, crumberid=xyx-11111-11111-11-111111, svc_tran_origin=SSS, SSScoreed=Camel-SSS-1111-1111111-111, cccc_ff_ssss_aaaaa=AAAA, AAAType=null, cccc_ff_ssss_tata=AAA, AAAexxxx=0, avronnnn=url.add.add.com, AAAssssssss=1661920390882,tang_dik_jagah=ABC_XYZ, ver=0.1.2, AAAprrrrrr=4, AAArptooo=null, source_DOT_adaptr=mom, AAAjaaana=tAAic://toic,tang_dik_jagah_tata=AAA, targCTService=progr, SSScoreedAsBytes=[a@123, CamelAAARequestTimeout=600000, sedaTimeout=600000} {[{"type":"AAtiongo","pAAo":"AAAA","ssssssss":"2022-08-31 00:00:00","data":[{"chabbbi":"ca_1111_11111_AAtiongo_AAAA","tatajahajqaki":"AA 111","jahajqaki":{"numeo":"111","jahaaj":{"cde":"ARL_AA","couAAa":"AA","aaoo":"AAR"},"AAsuf":null},"sgnnnn":"AAR111","stppp":"J","muddStatuscde":"AA","kissak":"III","AAType3lc":"111","AAType5lc":"B111","rggggggg":"AAAAA","carrrrr":{"cde":"ARL_AA","couAAa":"AA","aaoo":"AAR"},"ddddddcde":"pubbb","pubbbjahajqaki":"AA 111","jahajqakipubbb":{"numeo":"111","jahaaj":{"cde":"AA","couAAa":null,"aaoo":null}},"sssss":1098,"kkkkkss":834,"kitnaba":{"AAAAAA":"2022-08-2100:00:00","WWWW":"2022-08-2100:00:00","eeeeee":"2022-08-2100:00:00","sssssss":"2022-08-2100:00:00","ddddddd":"2022-08-2100:00:00","eeeeeeee":"2022-08-2100:00:00","ddddddddd":"2022-08-2100:00:00","ttttttt":"2022-08-2100:00:00","ttttttt":"2022-08-2100:00:00","Edddddd":"2022-08-2100:00:00","ffffff":"2022-08-2100:00:00","ddddddL":"2022-08-2100:00:00","dddddd":"2022-08-2100:00:00","Adddddd":"2022-08-2100:00:00","ssssT":"2022-08-2100:00:00","ddddd":"2022-08-2100:00:00","ggggg":"2022-08-2100:00:00","ffffff":"2022-08-2100:00:00","Eddddd":"2022-08-2100:00:00","ssssss":"2022-08-2100:00:00","Eddddd":"2022-08-2100:00:00"},"durdddd":{"Exxxxx":"Pdddd.oo","ScfffTTTT":"xxx1H0M0.000S","xxxxIDL":"-Pxxxx6M0.000S","ESTTTT":"PxxxxH26M0.000S"},"gallle":[{"aaaaaaa":"aaa000033","gffffnnnn":"111"}],"stsssss":[{"hhhhhh":"AA1111111","standnnnn":"S20"}],"blttttt":[{"hhhhhh":"ABB000003","beltnnnn":"aa11","beltAAenpttttt":"2022-08-2100:00:00","kkkkkkkpttttt":"2022-08-2100:00:00"}],"redddddd":{"SSSSS":[{"aalllll":"ALLUU99999","resssssss":"AA1111111","resssssssnnnn":"S20","pprrrrrsssss":"AAA11111"}],"bgggg_blt":[{"aalllll":"aaaaaa1111111","resssssss":"ABB000003","resssssssnnnn":"IB02","kitnaba":{"AAAAAA":"2022-08-31006:14:00a","AAAAAA":"2022-08-31006:14:00a"}}],"aaaaaaaaaaa_sss":[{"aalllll":"aaaaaa8888888","resssssss":"false"}],"aaaaaaaaaa_ssss":[{"aalllll":"aaaaaa8888888","resssssss":"GAT000033","resssssssnnnn":"120","pprrrrrsssss":"GAT000019"}],"qqqqqqqqqqqq":[{"aalllll":"qqqqqqqqqqqq","resssssss":"false"}]},"kkkkkk":[{"cde":"aaa_sss","tatAAde":"CAI","aaaaAAde":"PPPP","legnumeo":1},{"cde":"ABC_XYZ","tatAAde":"AAA","aaaaAAde":"AAAA","legnumeo":2}],"cdeshareList":[{"numeo":"1111","jahaaj":{"cde":"ARL_AA","couAAa":"AA","aaoo":"AAA"},"AAsuf":null,"pubbbjahajqaki":"AA 1111","jahajqakipubbb":{"numeo":"1111","jahaaj":{"cde":"AA","couAAa":null,"aaoo":null}}},{"numeo":"1111","jahaaj":{"cde":"ARL_CT","couAAa":"CT","aaoo":"CTH"},"AAsuf":null,"pubbbjahajqaki":"CT 1111","jahajqakipubbb":{"numeo":"1111","jahaaj":{"cde":"CT","couAAa":null,"aaoo":null}}}],"saaaaaa":{"ffff":"RRR","mapr":"Finalised","SSSGeneral":"AAened","AAceptance":"Finalised","loddacctrr":"SheCT_Finalised","brrrrrrdd":"AAened","IIIernal":"110"}}]}]}
host = mucAAuplfrAA02

-----------------------

 

Labels (3)
0 Karma

chaker
Contributor

Also review the docs for the rex command. It uses named capture groups for the extracted field names.

https://docs.splunk.com/Documentation/Splunk/9.0.1/SearchReference/Rex

 

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @mdyunusraza,

this is a json log so did you tried the spath command (https://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Spath) to extract fields?

Ciao.

Giuseppe

 

0 Karma

mdyunusraza
Observer

@gcusello 

No, I have not tried that. We have used the REX command to extract fields using regex, but those were not JSON logs. We have this new log we need to dissect and form a table. Hence the requirement.

I will read about SPATH and see how it goes.

0 Karma

chaker
Contributor

Hi @mdyunusraza ,

Try escaping those bracket symbols you are matching with a backslash.

 | rex "((?:\[^"\\\/\b\f\n\r\t\]|\\u\d{4})*)"

0 Karma

mdyunusraza
Observer

@chaker ,

Yes, I tried that, but it gives me another error, like so,

Error in 'SearchParser': Missing a search command before '\'. Error at position '100' of search query 'search index=indname "service" "AAt: Upd...{snipped} {errorcontext = f\n\r\t\]|\\u\d{4})*)}'.

0 Karma
Get Updates on the Splunk Community!

Streamline Data Ingestion With Deployment Server Essentials

REGISTER NOW!Every day the list of sources Admins are responsible for gets bigger and bigger, often making the ...

Remediate Threats Faster and Simplify Investigations With Splunk Enterprise Security ...

REGISTER NOW!Join us for a Tech Talk around our latest release of Splunk Enterprise Security 7.2! We’ll walk ...

Introduction to Splunk AI

WATCH NOWHow are you using AI in Splunk? Whether you see AI as a threat or opportunity, AI is here to stay. ...