Splunk Search

How to ignore concrete rows from csv file before indexing?

spisiakmi
Communicator

Hi,

I spent really a lot of time, but found no solution. Here is my problem. There is CSV file, which should be indexed:

Lötprotokoll Version 1.0
Laufende Nummer;Version Lötprotokoll;Lötprogramm Version
1;1;1;
2;3;4;
937;381.000000;110.000000;
938;382.000000;113.000000;
565136;1;1;9;

I want to sent it to the Splunk using forwarder. I want, that the rows 1, 3, 4, 5, 6 will be ignored and not indexed. Can anybody help me, please, how to do it?

0 Karma
1 Solution

spisiakmi
Communicator

Finally I used the Powershell script, which picks only (indexed from 0) rows 1, 6:

Get-ChildItem *\*.protocol | ForEach { Get-Content $_.Fullname | Select-Object -Index 1,6 | Out-File "$($_.Directory)\$($_.BaseName).csv" -Encoding utf8}

View solution in original post

0 Karma

spisiakmi
Communicator

Finally I used the Powershell script, which picks only (indexed from 0) rows 1, 6:

Get-ChildItem *\*.protocol | ForEach { Get-Content $_.Fullname | Select-Object -Index 1,6 | Out-File "$($_.Directory)\$($_.BaseName).csv" -Encoding utf8}
0 Karma

spisiakmi
Communicator

I can create 2 very similar regex.
1. regex with 2 captured groups, which should be indexed
.+\n(.+\n).+\n.+\n.+\n.+\n(.+\n)
2. regex with 2 captured groups, which should be ignored
(.+\n).+\n(.+\n.+\n.+\n.+\n).+\n

I have probably 2 possible choises.

  1. preamble_regex: the question is, if it is possible to put here regex #2, and if the captured groups will be accepted
  2. transforms.conf and parsing [stanza]: [setparsing] REGEX = regex #1 DEST_KEY = queue FORMAT = indexQueue the same question, if I use regex #1, if the captured groups will be accepted
0 Karma

spisiakmi
Communicator

I tried to import the file manualy. I created my own sourcetype and added the parameter PREAMBLE_REGEX. It has no influence on the previewed content of indexed csv file. Even I tried simple regular expressions. The question is, if the parameter PREAMBLE_REGEX is either working properly.

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...

New in Observability Cloud - Explicit Bucket Histograms

Splunk introduces native support for histograms as a metric data type within Observability Cloud with Explicit ...