Getting Data In

how to drop events containing null values coming from a csv using transforms

learnsplungeek
Loves-to-Learn Everything

Hi 
I am getting some events from a csv which contains the below format and would like to drop such events using transforms. 

null,null,0,null,null,null,null,null,null,  ---- to be dropped
null,null,0,null,null,null,null,null,null,  ---- to be dropped
null,null,0,null,null,null,null,null,null,  ---- to be dropped
null,null,0,null,null,null,null,null,null,  ---- to be dropped
null,null,0,null,null,null,null,null,null,  ---- to be dropped
52376,null,0,test,87387,2984,22,abc,99  ----- to be kept

Below is what i have done so far and is not working

Props.conf
[Reports5min]
TRANSFORMS-null = setnull


transforms.conf
[setnull]
REGEX = ^null,null\,0,null,null,null,null,null,null,$
DEST_KEY = queue
FORMAT = nullQueue

Labels (1)
0 Karma

PickleRick
SplunkTrust
SplunkTrust

1. Where are you putting those configs?

2. Do you use indexed extractions?

0 Karma

learnsplungeek
Loves-to-Learn Everything

Hi , I have placed both the transforms and props at indexer layer. We are getting the CSV data through UF's

0 Karma

glc_slash_it
Path Finder

Assuming the stanza [Reports5min] points to the right sourcetype, try placing a capture group in the regex property, like this:

REGEX = ^(null,null,0,null,null,null,null,null,null,)$

Also are you sure there is ni blacnk spaces at the end of each line?

The rest of the settinggs seems fine.

 

0 Karma

learnsplungeek
Loves-to-Learn Everything

I tried the regex and it did not work 😞

0 Karma

learnsplungeek
Loves-to-Learn Everything

Hi , Sure let me try this regex and see. Yes there are no blank spaces after each line containing null values

0 Karma

learnsplungeek
Loves-to-Learn Everything

Hi @woodcock @niketn 

Please help me here 

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Announcing Modern Navigation: A New Era of Splunk User Experience

We are excited to introduce the Modern Navigation feature in the Splunk Platform, available to both cloud and ...

Modernize your Splunk Apps – Introducing Python 3.13 in Splunk

We are excited to announce that the upcoming releases of Splunk Enterprise 10.2.x and Splunk Cloud Platform ...

Step into “Hunt the Insider: An Splunk ES Premier Mystery” to catch a cybercriminal ...

After a whole week of being on call, you fell asleep on your keyboard, and you hit a sequence of buttons that ...