Splunk Search

How to ignore concrete rows from csv file before indexing?

spisiakmi
Contributor

Hi,

I spent really a lot of time, but found no solution. Here is my problem. There is CSV file, which should be indexed:

Lötprotokoll Version 1.0
Laufende Nummer;Version Lötprotokoll;Lötprogramm Version
1;1;1;
2;3;4;
937;381.000000;110.000000;
938;382.000000;113.000000;
565136;1;1;9;

I want to sent it to the Splunk using forwarder. I want, that the rows 1, 3, 4, 5, 6 will be ignored and not indexed. Can anybody help me, please, how to do it?

0 Karma
1 Solution

spisiakmi
Contributor

Finally I used the Powershell script, which picks only (indexed from 0) rows 1, 6:

Get-ChildItem *\*.protocol | ForEach { Get-Content $_.Fullname | Select-Object -Index 1,6 | Out-File "$($_.Directory)\$($_.BaseName).csv" -Encoding utf8}

View solution in original post

0 Karma

spisiakmi
Contributor

Finally I used the Powershell script, which picks only (indexed from 0) rows 1, 6:

Get-ChildItem *\*.protocol | ForEach { Get-Content $_.Fullname | Select-Object -Index 1,6 | Out-File "$($_.Directory)\$($_.BaseName).csv" -Encoding utf8}
0 Karma

spisiakmi
Contributor

I can create 2 very similar regex.
1. regex with 2 captured groups, which should be indexed
.+\n(.+\n).+\n.+\n.+\n.+\n(.+\n)
2. regex with 2 captured groups, which should be ignored
(.+\n).+\n(.+\n.+\n.+\n.+\n).+\n

I have probably 2 possible choises.

  1. preamble_regex: the question is, if it is possible to put here regex #2, and if the captured groups will be accepted
  2. transforms.conf and parsing [stanza]: [setparsing] REGEX = regex #1 DEST_KEY = queue FORMAT = indexQueue the same question, if I use regex #1, if the captured groups will be accepted
0 Karma

spisiakmi
Contributor

I tried to import the file manualy. I created my own sourcetype and added the parameter PREAMBLE_REGEX. It has no influence on the previewed content of indexed csv file. Even I tried simple regular expressions. The question is, if the parameter PREAMBLE_REGEX is either working properly.

0 Karma
Get Updates on the Splunk Community!

Demo Day: Strengthen Your SOC with Splunk Enterprise Security 8.1

Today’s threat landscape is more complex than ever. Security operation centers (SOCs) are overwhelmed with ...

Dashboards: Hiding charts while search is being executed and other uses for tokens

There are a couple of features of SimpleXML / Classic dashboards that can be used to enhance the user ...

Splunk Observability Cloud's AI Assistant in Action Series: Explaining Metrics and ...

This is the fourth post in the Splunk Observability Cloud’s AI Assistant in Action series that digs into how ...