Getting Data In

how to drop events containing null values coming from a csv using transforms

learnsplungeek
Loves-to-Learn Everything

Hi 
I am getting some events from a csv which contains the below format and would like to drop such events using transforms. 

null,null,0,null,null,null,null,null,null,  ---- to be dropped
null,null,0,null,null,null,null,null,null,  ---- to be dropped
null,null,0,null,null,null,null,null,null,  ---- to be dropped
null,null,0,null,null,null,null,null,null,  ---- to be dropped
null,null,0,null,null,null,null,null,null,  ---- to be dropped
52376,null,0,test,87387,2984,22,abc,99  ----- to be kept

Below is what i have done so far and is not working

Props.conf
[Reports5min]
TRANSFORMS-null = setnull


transforms.conf
[setnull]
REGEX = ^null,null\,0,null,null,null,null,null,null,$
DEST_KEY = queue
FORMAT = nullQueue

Labels (1)
0 Karma

PickleRick
SplunkTrust
SplunkTrust

1. Where are you putting those configs?

2. Do you use indexed extractions?

0 Karma

learnsplungeek
Loves-to-Learn Everything

Hi , I have placed both the transforms and props at indexer layer. We are getting the CSV data through UF's

0 Karma

glc_slash_it
Path Finder

Assuming the stanza [Reports5min] points to the right sourcetype, try placing a capture group in the regex property, like this:

REGEX = ^(null,null,0,null,null,null,null,null,null,)$

Also are you sure there is ni blacnk spaces at the end of each line?

The rest of the settinggs seems fine.

 

0 Karma

learnsplungeek
Loves-to-Learn Everything

I tried the regex and it did not work 😞

0 Karma

learnsplungeek
Loves-to-Learn Everything

Hi , Sure let me try this regex and see. Yes there are no blank spaces after each line containing null values

0 Karma

learnsplungeek
Loves-to-Learn Everything

Hi @woodcock @niketn 

Please help me here 

0 Karma
Get Updates on the Splunk Community!

Splunk Mobile: Your Brand-New Home Screen

Meet Your New Mobile Hub  Hello Splunk Community!  Staying connected to your data—no matter where you are—is ...

Introducing Value Insights (Beta): Understand the Business Impact your organization ...

Real progress on your strategic priorities starts with knowing the business outcomes your teams are delivering ...

Enterprise Security (ES) Essentials 8.3 is Now GA — Smarter Detections, Faster ...

As of today, Enterprise Security (ES) Essentials 8.3 is now generally available, helping SOC teams simplify ...