All Apps and Add-ons

GCP Billing CSV Not Parsing Correctly

jamin358
Explorer

This is more of a bug report than a question.
I am working with the Billing data coming from our new GCP environment and have found that CSV data where a string contains a comma is not being parsed properly.
I have discovered this from data where the Credit1 field says "External IPs will not be charged until April 1, 2020." It splits the row into two columns, pushing all data out by one and therefore losing the Description Field value from that event.

The data looks like this is Splunk
*{ [-]
Account ID: 111111-222222-333333
Cost: GBP
Credit1: "External IPs will not be charged until April 1
Credit1 Amount: 2020."
Credit1 Currency: -0.156831
Currency: 0.156831
Description:
End Time: 2020-02-20T00:00:00-08:00
Line Item: com.google.cloud/services/compute-engine/ExternalIp
Measurement1: com.google.cloud/services/compute-engine/ExternalIp
Measurement1 Total Consumption: 183661
Measurement1 Units: seconds
Project: 12344566778
Project ID: 12344566778
Project Labels: project
Project Name: project
Project Number: GBP
Start Time: 2020-02-19T00:00:00-08:00
} *

This data parsing isnt done in the props of the sourcetype but in the python code. Line 260 of Splunk_TA_google-cloudplatform/bin/splunk_ta_gcp/modinputs/billing.py
I'm not an expert in GCP data, so I dont know what the best course of action would be to fix this bug without breaking other things.

My assumption is that no data values will have "word,word" but "word, word" with a space, so would the fix here to change the split from being just a comma to a comma and no space afterwards. ie in python line = line.split(',[^\s]')

0 Karma
Get Updates on the Splunk Community!

Splunk Observability Cloud | Unified Identity - Now Available for Existing Splunk ...

Raise your hand if you’ve already forgotten your username or password when logging into an account. (We can’t ...

Index This | How many sides does a circle have?

February 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

Registration for Splunk University is Now Open!

Are you ready for an adventure in learning?   Brace yourselves because Splunk University is back, and it's ...