topic Re: How to use STOP_PROCESSING_IF in transforms.conf in Getting Data In

How to use STOP_PROCESSING_IF in transforms.conf

vincentwhn — Mon, 15 Sep 2025 16:52:26 GMT

Can anyone give me some examples of using STOP_PROCESSING_IF in transforms.conf? Seems there is no examples exists within transforms.conf.example

Can I use it to stop processing events once match a regex?

Re: How to use STOP_PROCESSING_IF in transforms.conf

richgalloway — Mon, 15 Sep 2025 17:06:34 GMT

I've seen that setting used like this

transforms.conf [mytransform] INGEST_EVAL = queue=if(condition, "nullQueue", queue) STOP_PROCESSING_IF = queue=="nullQueue"

Re: How to use STOP_PROCESSING_IF in transforms.conf

vincentwhn — Mon, 15 Sep 2025 17:07:51 GMT

Thanks for your reply.

Can you also help to review below configs as well? Many Thanks.

[testlog]
RULESET-ruleset_testlog = rule1, rule2, rule3, rule4
RULESET_DESC-ruleset_testlog =

transforms.conf

[rule2]
INGEST_EVAL = _raw:=replace(_raw, "(\"(?!(?:EntityId|EntityName|Message|QueryResults|CorrelationId|InstanceUrl|Fields|Name|Value|UserId|Id|CreationTime|Workload|ResultStatus))[^\"]+\"\\s*:\\s*(?:\\{[^{}]*\\}|\\[[^\\[\\]]*\\]|\"(?:[^\"\\\\]|\\\\.)*\"|[^,}\\r\\n]+))", "")

[rule3]
INGEST_EVAL = _raw:=replace(_raw, ",{2,}", ",")

[rule4]
INGEST_EVAL = _raw:=replace(_raw, ",}\\s*$", "}")

I would like to use rule1 to limit where those INGEST_EVAL applies. Am I doing anything wrong here?
Applied above config but found no changes.

Re: How to use STOP_PROCESSING_IF in transforms.conf

richgalloway — Mon, 15 Sep 2025 20:48:42 GMT

It's hard to evaluate the expressions without sample data, but I have doubts about the number of escape characters (particularly "\\s*"). Have you used regex101.com to verify the expressions?

Re: How to use STOP_PROCESSING_IF in transforms.conf

vincentwhn — Tue, 16 Sep 2025 08:08:00 GMT

Thanks for your reply. The problem is solved.😁

Re: How to use STOP_PROCESSING_IF in transforms.conf

richgalloway — Tue, 16 Sep 2025 12:01:49 GMT

Please share the solution.

Re: How to use STOP_PROCESSING_IF in transforms.conf

vincentwhn — Wed, 17 Sep 2025 02:59:40 GMT

This is my sample log:

{"AppAccessContext": {"AADSessionID": "f3a2b6d4-1234-4c89-9a67-8b2e5d7c9f01", "ClientAppId": "00000003-0000-0ff1-ce00-000000000000", "ClientAppName": "SharePoint Online", "CorrelationId": "e5d6a7b8-2345-4f90-8c12-9d8e7f6a5b4c", "TokenIssuedAtTime": "2025-09-16T05:12:33Z", "UniqueTokenId": "abc123xyz789", "UserObjectid": "d2f3a4b5-6789-4cde-9012-3456789abcde"}, "CreationTime": "2025-09-16T05:12:35Z", "Id": "9876543210", "Operation": "FileAccessed", "OrganizationId": "contoso.onmicrosoft.com", "RecordType": 6, "UserKey": "i:0#.f|membership|vincent.wong@contoso.com", "UserType": 0, "Version": 1, "Workload": "SharePoint", "ClientIP": "203.0.113.45", "UserId": "vincent.wong@contoso.com", "ApplicationiD": "spfx-app-12345", "AuthenticationType": "Federated", "BrowserName": "Edge", "BrowserVersion": "116.0.1938.81", "EventSource": "SharePoint", "GeoLocation": "HK", "IsManagedDevice": true, "ItemType": "File", "ListItemUniqueId": "b3c4d5e6-f789-4abc-9012-3456789def01", "Pleatform": "Windows", "Site": "https://contoso.sharepoint.com/sites/ProjectX", "UserAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64)", "WebId": "a1b2c3d4-e5f6-7890-abcd-ef1234567890", "DeviceDisplayName": "Vincent-Laptop", "HighPriorityMediaProcessing": false, "ListBassType": 101, "ListServerTemplate": 101, "DestinationFileExtension": ".docx", "SourceFileExtension": ".docx", "SourceRelativeUrl": "/sites/ProjectX/Shared Documents/Report.docx", "SourceFileName": "Report.docx", "DestinationRelativeUrl": "/sites/ProjectX/Shared Documents/Report.docx", "DestinationFileName": "Report.docx", "ApplicationDisplayName": "Microsoft Office Word", "SiteUrl": "https://contoso.sharepoint.com/sites/ProjectX-prd-test", "ObjectId": "https://contoso.sharepoint.com/sites/ProjectX/Shared Documents/Report.docx"}

I use the config below to control if the log masking is enabled or not:

props.conf

[testlog]
RULESET-SharePoint_log_filter = SharePoint_log_filter_control, SharePoint_log_filter_rule1, SharePoint_log_filter_rule2, SharePoint_log_filter_rule3, SharePoint_log_filter_rule4

transforms.conf

[SharePoint_log_filter_control]
INGEST_EVAL = SharePoint_log_filter_control=if(match(_raw, "\"Workload\": \"SharePoint\".*\"SiteUrl\": \".*-prd.*\""), "match", "not_match")
STOP_PROCESSING_IF = SharePoint_log_filter_control=="not_match"

[SharePoint_log_filter_rule1]
INGEST_EVAL=_raw:=replace(_raw, "(\"(?!(?:Operation|SiteUrl|UserId|Workload))[^\"]+\"\\s*:\\s*(?:\\{[^{}]*\\}|\\[[^\\[\\]]*\\]|\"(?:[^\"\\\\]|\\\\.)*\"|[^,}\\r\\n]+))", "")

[SharePoint_log_filter_rule2]
INGEST_EVAL=_raw:=replace(_raw, "(,\\s+)+", ", ")

[SharePoint_log_filter_rule3]
INGEST_EVAL=_raw:=replace(_raw, ", }", "}")

[SharePoint_log_filter_rule4]
INGEST_EVAL=_raw:=replace(_raw, "{, ", "{")