Hello,
{ [-]
guessedService: ejj
logGroup: /aws/ejj/cluster
logStream: kube-apt-15444d2f8c4b216a9cb69ac
message:{"kind":"Event","stage":"ResponseComplete","requestURI":"/api/v1/namespaces/jej/endpoints/eji.com-aws-eji","verb":"update","user":{"username":"system:serviceaccount:efs:efs-provisioner","uid":"ab5d27b4c-71a4f77323b0","groups":["system:serviceaccounts","system:serviceaccounts:eji","system:authenticated"]},"sourceIPs":["10.0.0.0"],"userAgent":"eji-provisioner/v0.0.0 (linux/amd64) kubernetes/$Format","objectRef":{"resource":"endpoints","namespace":"edd","name":"dds.com-aws-edds","uid":"44ad8-899f-fbc1f4befb2f","apiVersion":"v1","resourceVersion":"8852157"},"responseStatus":{"metadata":{},"code":200}}
i already a below props and transforms to extract all the fields from message.
Props.conf
[json_no_new]
REPORT-json = report-json,report-json-new
KV_MODE = none
INDEXED_EXTRACTIONS = json
LINE_BREAKER = ^{
NO_BINARY_CHECK = true
disabled = false
pulldown_type = true
Transforms.conf
[report-json]
SOURCE_KEY = message
REGEX = (?P<json2>{.+)
DEST_KEY = _raw
[report-json-new]
REGEX = \\*"([^"]+)\":[\s]*"*(\[.*?\]|\{.*?\}"*\}*|[^"]+|\d+),*
FORMAT = $1::$2
SOURCE_KEY = json2
Now from the result i have below field with json value
user = {"username":"system:serviceaccount:efs:efs-provisioner","uid":"ab5d27b4c-71a4f77323b0","groups":["system:serviceaccounts","system:serviceaccounts:eji","system:authenticated"]}
again with props and transform i want to extract values from user field.
Please some one let me know if thats possible
Thanks
You should be getting all the fields being extracted just with INDEXED_EXTRACTION. As your data in proper JSON format, you don't even need those transforms.
You should see fields like: logGroup, message.kind, message.user.username, message.user.uid, etc.
Though alternatively, you can use search time extraction, which is what I would do: Using KV_MODE=json instead of INDEXED_EXTRACTION=json.
Try this below configuration if you can on test system:
[json_no_new]
KV_MODE = json
LINE_BREAKER = }([\r\n]+)
SHOULD_LINEMERGE = false
NO_BINARY_CHECK = true
disabled = false
pulldown_type = true
You may need to change the SHOULD_LINEMERGE along with other configurations to make sure the data being extracted in the right events according to your _raw data.
You should see fields like with search-time extraction as well: logGroup, message.kind, message.user.username, message.user.uid, etc.
Fields are hierarchical with the use of .(dot).
As far as I remember, the automatic json extraction (contrary to the spath command) does not care about attributes hierarchy.