Getting Data In

how to drop events containing null values coming from a csv using transforms

learnsplungeek
Loves-to-Learn Everything

Hi 
I am getting some events from a csv which contains the below format and would like to drop such events using transforms. 

null,null,0,null,null,null,null,null,null,  ---- to be dropped
null,null,0,null,null,null,null,null,null,  ---- to be dropped
null,null,0,null,null,null,null,null,null,  ---- to be dropped
null,null,0,null,null,null,null,null,null,  ---- to be dropped
null,null,0,null,null,null,null,null,null,  ---- to be dropped
52376,null,0,test,87387,2984,22,abc,99  ----- to be kept

Below is what i have done so far and is not working

Props.conf
[Reports5min]
TRANSFORMS-null = setnull


transforms.conf
[setnull]
REGEX = ^null,null\,0,null,null,null,null,null,null,$
DEST_KEY = queue
FORMAT = nullQueue

Labels (1)
0 Karma

PickleRick
SplunkTrust
SplunkTrust

1. Where are you putting those configs?

2. Do you use indexed extractions?

0 Karma

learnsplungeek
Loves-to-Learn Everything

Hi , I have placed both the transforms and props at indexer layer. We are getting the CSV data through UF's

0 Karma

glc_slash_it
Path Finder

Assuming the stanza [Reports5min] points to the right sourcetype, try placing a capture group in the regex property, like this:

REGEX = ^(null,null,0,null,null,null,null,null,null,)$

Also are you sure there is ni blacnk spaces at the end of each line?

The rest of the settinggs seems fine.

 

0 Karma

learnsplungeek
Loves-to-Learn Everything

I tried the regex and it did not work 😞

0 Karma

learnsplungeek
Loves-to-Learn Everything

Hi , Sure let me try this regex and see. Yes there are no blank spaces after each line containing null values

0 Karma

learnsplungeek
Loves-to-Learn Everything

Hi @woodcock @niketn 

Please help me here 

0 Karma
Get Updates on the Splunk Community!

New Year, New Changes for Splunk Certifications

As we embrace a new year, we’re making a small but important update to the Splunk Certification ...

Stay Connected: Your Guide to January Tech Talks, Office Hours, and Webinars!

What are Community Office Hours? Community Office Hours is an interactive 60-minute Zoom series where ...

[Puzzles] Solve, Learn, Repeat: Reprocessing XML into Fixed-Length Events

This challenge was first posted on Slack #puzzles channelFor a previous puzzle, I needed a set of fixed-length ...