Truncate Log

[riposans]

I'm confused how to truncate from this log. how do I do it from props.conf or from the SPL command? Can anyone provide a solution to this?

<11>1 2021-03-18T15:05:30.501Z abcdefghi-jajaj-b1bc07001-xb0k7.abcdefghi-user - - - [Originator@7776 kubernetes__container_name="abcdefghi-jajaj" docker__container_id="a1bbddc80312d8501f1b1ac015d525722f105a71d6521be0728e8b057066eda1" kubernetes__pod_name="abcdefghi-jajaj-b1bc07001-xb0k7" bosh_index="0" stream="stbcd" kubernetes__namespace_name="abcdefghi-develop" bosh_id="e0700d15-ca5a-1f35-8e01-bd83d3eb705a" bosh_deployment="service-instance_f08cb851-fa53-1206-0a6b-705f3fa0f301" docker_id="a1bbddc80312d" tag="kubernetes.var.log.containers.abcdefghi-user-b1bc07001-xb0k7_abcdefghi-develop_abcdefghi-user-a1bbddc80312d8501f1b1ac015d525722f105a71d6521be0728e8b057066eda1.log" instance_type="werkir"] 2021-03-18 22:05:00.210 INFO [abcdefghi-jajaj,3010acf256f7c7e0,717ea36c0d67f3da,true] 6 --- [nio-0020-exec-1] c.id.bankabcde.common.util.SplunkUtil : [LOGIN_abc]|mobilePhone=ABC123|mobilePhone=ABC123|mobilePhone=ABC123|mobilePhone=ABC123|mobilePhone=ABC123|mobilePhone=ABC123|mobilePhone=ABC123|uobilePhone=ABC123|mobilePhone=ABC123|mobilePhone=ABC123|mobilePhone=ABC123|sessionID=ABC123|mobilePhone=ABC123|mobilePhone=ABC123|appVersion=ABC123|mobilePhone=ABC123|custGroup=ABC123

i want to cut it to something like this:

[LOGIN_abc]|mobilePhone=ABC123|mobilePhone=ABC123|mobilePhone=ABC123|mobilePhone=ABC123|mobilePhone=ABC123|mobilePhone=ABC123|mobilePhone=ABC123|uobilePhone=ABC123|mobilePhone=ABC123|mobilePhone=ABC123|mobilePhone=ABC123|sessionID=ABC123|mobilePhone=ABC123|mobilePhone=ABC123|appVersion=ABC123|mobilePhone=ABC123|custGroup=ABC123

THANKYOU

[yuanliu · Accepted solution]

This problem is not well defined.  But before that, I would caution any data truncation in props.conf.

Anyway, you need to prescribe a formula/criterion you want this done.  Are you using the front part (that you discard) or the end part (that you want to preserve) to make the determination?  If front, how are you going to determine that is the front part?  If end, how are you going to determine that it is the end part?  Without giving us this information, there can be a million ways to make this specific log entry trimmed, but most of these methods will fail you in general.

One example to do this in the front could be:

| eval _raw = replace(_raw, ".*util.SplunkUtil : ", "")

Here, I'm assuming that util.SplunkUtil and the spacing are fixed values.

One example of using the end to do the same could be

| eval _raw = replace(_raw, ".*: *(\[LOGIN_\w+\]|.+)", "\1")

Here, I assume that the colon and LOGIN_* in square brackets are the fixture.

As you can see, the combinations are infinite.  What exactly is your design?

[riposans]

thankyou so much its works for me