Security

How to extract all sub strings ends with .csv in a string?

Explorer

Sorry I'm new to regex. I'm trying to get some meaning full data from the log files.

I want all the sub-strings ending with .csv in my log file at any given point of time. Below is the the log file preview. Any leads would be highly appreciated.

alt text

Tags (2)
0 Karma
1 Solution

Explorer

it fixed the problem.......

rex field=_raw ".*\s(?P.*\.csv)$" |search CSVFiles=*

View solution in original post

0 Karma

Explorer

it fixed the problem.......

rex field=_raw ".*\s(?P.*\.csv)$" |search CSVFiles=*

View solution in original post

0 Karma

Builder

This rex does not work with the above data.
It can not be complete here?

PS you do not need to specify field=_raw, if omitted, _raw is used by default.

0 Karma

Explorer

I have tried both the rex and both seems to be working fine.

0 Karma

Builder

(?P..csv) this does not extract anything. If it was more like (?<CSVFiles>\w+\.csv) it will work.

0 Karma

Builder

It's better if you past the text in stead of a picture of the text.

0 Karma

Explorer

Below are the logs. Need to list down all files names ending with .csv. e.g.:
1. adnattributeset.csv
2. adnnavigationattributes.csv

host=mdc1vr1002 sourcetype=MCOMETLOUT

2016-11-06 19:42:35,800 | DEBUG | main:ConcatNCopy | Appending smaller file: adnattributeset.csv
2016-11-06 19:42:35,801 | DEBUG | main:ConcatNCopy | Copy: adnattributeset.csv to /opt/pim/ETL/MCOM/etlc/output/site/adnattributeset.csv, size: 0, elapsed ms: 1
2016-11-06 19:42:35,801 | DEBUG | main:ConcatNCopy | Appending smaller file: adnnavigationattributes.csv
2016-11-06 19:42:35,801 | DEBUG | main:ConcatNCopy | Copy: adnnavigationattributes.csv to /opt/pim/ETL/MCOM/etlc/output/site/adnnavigationattributes.csv, size: 0, elapsed ms: 0
2016-11-06 19:42:35,809 | DEBUG | main:ConcatNCopy | Appending smaller file: archivedproducts.csv
2016-11-06 19:42:35,830 | DEBUG | main:ConcatNCopy | Copy: archived
products.csv to /opt/pim/ETL/MCOM/etlc/output/site/archivedproducts.csv, size: 2768026, elapsed ms: 21
2016-11-06 19:42:35,853 | DEBUG | main:ConcatNCopy | Appending smaller file: attr
cat.csv
2016-11-06 19:42:36,043 | DEBUG | main:ConcatNCopy | Copy: attrcat.csv to /opt/pim/ETL/MCOM/etlc/output/site/attrcat.csv, size: 201223799, elapsed ms: 190
2016-11-06 19:42:36,043 | DEBUG | main:ConcatNCopy | Appending smaller file: attrcategoryexclusion.csv
2016-11-06 19:42:36,044 | DEBUG | main:ConcatNCopy | Copy: attrcategoryexclusion.csv to /opt/pim/ETL/MCOM/etlc/output/site/attrcategoryexclusion.csv, size: 16705, elapsed ms: 1
2016-11-06 19:42:36,045 | DEBUG | main:ConcatNCopy | Appending smaller file: attrpagemedia.csv
2016-11-06 19:42:36,079 | DEBUG | main:ConcatNCopy | Copy: attrpagemedia.csv to /opt/pim/ETL/MCOM/etlc/output/site/attrpagemedia.csv, size: 38563205, elapsed ms: 34
2016-11-06 19:42:36,125 | DEBUG | main:ConcatNCopy | Appending smaller file: attrprodbrandship.csv
2016-11-06 19:42:36,169 | DEBUG | main:ConcatNCopy | Appending smaller file: attr
prodcopyreg.csv
2016-11-06 19:42:36,359 | DEBUG | main:ConcatNCopy | Appending smaller file: attrproddatasource.csv
2016-11-06 19:42:36,366 | DEBUG | main:ConcatNCopy | Appending smaller file: attr
prodforcednew.csv
2016-11-06 19:42:36,422 | DEBUG | main:ConcatNCopy | Appending smaller file: attrprodsitesearch.csv
2016-11-06 19:42:36,773 | DEBUG | main:ConcatNCopy | Appending smaller file: attr
prodtriggerdata.csv
2016-11-06 19:42:36,773 | DEBUG | main:ConcatNCopy | Appending smaller file: attrprodtupledata.csv
2016-11-06 19:42:36,977 | DEBUG | main:ConcatNCopy | Copy: attr
prodbrandship.csv attrprodcopyreg.csv attrproddatasource.csv attrprodforcednew.csv attrprodsitesearch.csv attrprodtriggerdata.csv attrprodtupledata.csv to /opt/pim/ETL/MCOM/etlc/output/site/attrprod.csv, size: 876915462, elapsed ms: 852
2016-11-06 19:42:36,977 | DEBUG | main:ConcatNCopy | Appending smaller file: attr
prod2.csv
2016-11-06 19:42:36,977 | DEBUG | main:ConcatNCopy | Copy: attrprod2.csv to /opt/pim/ETL/MCOM/etlc/output/site/attrprod2.csv, size: 0, elapsed ms: 0
2016-11-06 19:42:36,977 | DEBUG | main:ConcatNCopy | Appending smaller file: attrprod3.csv
2016-11-06 19:42:36,977 | DEBUG | main:ConcatNCopy | Copy: attr
prod3.csv to /opt/pim/ETL/MCOM/etlc/output/site/attrprod3.csv, size: 0, elapsed ms: 0
2016-11-06 19:42:36,977 | DEBUG | main:ConcatNCopy | Appending smaller file: attr
prod4.csv
2016-11-06 19:42:36,978 | DEBUG | main:ConcatNCopy | Copy: attrprod4.csv to /opt/pim/ETL/MCOM/etlc/output/site/attrprod4.csv, size: 0, elapsed ms: 1
2016-11-06 19:42:36,978 | DEBUG | main:ConcatNCopy | Appending smaller file: attrprod5.csv
2016-11-06 19:42:36,984 | DEBUG | main:ConcatNCopy | Copy: attr
prod5.csv to /opt/pim/ETL/MCOM/etlc/output/site/attrprod5.csv, size: 6903099, elapsed ms: 6
2016-11-06 19:42:36,985 | DEBUG | main:ConcatNCopy | Appending smaller file: attr
prod7.csv
2016-11-06 19:42:36,985 | DEBUG | main:ConcatNCopy | Copy: attrprod7.csv to /opt/pim/ETL/MCOM/etlc/output/site/attrprod7.csv, size: 0, elapsed ms: 1
2016-11-06 19:42:36,985 | DEBUG | main:ConcatNCopy | Appending smaller file: attrprod8.csv
2016-11-06 19:42:36,985 | DEBUG | main:ConcatNCopy | Copy: attr
prod8.csv to /opt/pim/ETL/MCOM/etlc/output/site/attrprod8.csv, size: 0, elapsed ms: 0
2016-11-06 19:42:36,985 | DEBUG | main:ConcatNCopy | Appending smaller file: attr
prodcolorway.csv
2016-11-06 19:42:36,985 | DEBUG | main:ConcatNCopy | Copy: attr
prodcolorway.csv to /opt/pim/ETL/MCOM/etlc/output/site/attrprodcolorway.csv, size: 11624, elapsed ms: 0
2016-11-06 19:42:36,985 | DEBUG | main:ConcatNCopy | Appending smaller file: attr
productexclusion.csv
2016-11-06 19:42:36,991 | DEBUG | main:ConcatNCopy | Copy: attr
productexclusion.csv to /opt/pim/ETL/MCOM/etlc/output/site/attrproductexclusion.csv, size: 5340406, elapsed ms: 6
2016-11-06 19:42:36,991 | DEBUG | main:ConcatNCopy | Appending smaller file: attr
promo.csv
2016-11-06 19:42:36,992 | DEBUG | main:ConcatNCopy | Copy: attrpromo.csv to /opt/pim/ETL/MCOM/etlc/output/site/attrpromo.csv, size: 577403, elapsed ms: 1
2016-11-06 19:42:36,992 | DEBUG | main:ConcatNCopy | Appending smaller file: attrreq.csv
2016-11-06 19:42:36,992 | DEBUG | main:ConcatNCopy | Copy: attr
req.csv to /opt/pim/ETL/MCOM/etlc/output/site/attrreq.csv, size: 52738, elapsed ms: 0
2016-11-06 19:42:37,010 | DEBUG | main:ConcatNCopy | Appending smaller file: attr
upc.csv
2016-11-06 19:42:37,495 | DEBUG | main:ConcatNCopy | Copy: attrupc.csv to /opt/pim/ETL/MCOM/etlc/output/site/attrupc.csv, size: 478738319, elapsed ms: 485
2016-11-06 19:42:37,495 | DEBUG | main:ConcatNCopy | Appending smaller file: attrvl.csv
2016-11-06 19:42:37,495 | DEBUG | main:ConcatNCopy | Copy: attr
vl.csv to /opt/pim/ETL/MCOM/etlc/output/site/attrvl.csv, size: 21284, elapsed ms: 0
2016-11-06 19:42:37,496 | DEBUG | main:ConcatNCopy | Appending smaller file: attr
vlitems.csv
2016-11-06 19:42:37,497 | DEBUG | main:ConcatNCopy | Copy: attrvlitems.csv to /opt/pim/ETL/MCOM/etlc/output/site/attrvlitems.csv, size: 1181621, elapsed ms: 1
2016-11-06 19:42:37,497 | DEBUG | main:ConcatNCopy | Appending smaller file: attribute.csv
2016-11-06 19:42:37,497 | DEBUG | main:ConcatNCopy | Copy: attribute.csv to /opt/pim/ETL/MCOM/etlc/output/site/attribute.csv, size: 173351, elapsed ms: 0
2016-11-06 19:42:37,497 | DEBUG | main:ConcatNCopy | Appending smaller file: brand.csv
2016-11-06 19:42:37,498 | DEBUG | main:ConcatNCopy | Copy: brand.csv to /opt/pim/ETL/MCOM/etlc/output/site/brand.csv, size: 117929, elapsed ms: 1
2016-11-06 19:42:37,498 | DEBUG | main:ConcatNCopy | Appending smaller file: brandconstraint.csv
2016-11-06 19:42:37,498 | DEBUG | main:ConcatNCopy | Copy: brand
constraint.csv to /opt/pim/ETL/MCOM/etlc/output/site/brandconstraint.csv, size: 0, elapsed ms: 0
2016-11-06 19:42:37,498 | DEBUG | main:ConcatNCopy | Appending smaller file: brand
constraintval.csv
2016-11-06 19:42:37,498 | DEBUG | main:ConcatNCopy | Copy: brand
constraintval.csv to /opt/pim/ETL/MCOM/etlc/output/site/brandconstraintval.csv, size: 0, elapsed ms: 0
2016-11-06 19:42:37,498 | DEBUG | main:ConcatNCopy | Appending smaller file: cat
pools.csv
2016-11-06 19:42:37,502 | DEBUG | main:ConcatNCopy | Copy: catpools.csv to /opt/pim/ETL/MCOM/etlc/output/site/catpools.csv, size: 4362512, elapsed ms: 4
2016-11-06 19:42:37,502 | DEBUG | main:ConcatNCopy | Appending smaller file: catprod.csv
2016-11-06 19:42:37,503 | DEBUG | main:ConcatNCopy | Copy: cat
prod.csv to /opt/pim/ETL/MCOM/etlc/output/site/catprod.csv, size: 54444, elapsed ms: 1
2016-11-06 19:42:37,503 | DEBUG | main:ConcatNCopy | Appending smaller file: catalog.csv
2016-11-06 19:42:37,503 | DEBUG | main:ConcatNCopy | Copy: catalog.csv to /opt/pim/ETL/MCOM/etlc/output/site/catalog.csv, size: 9310, elapsed ms: 0
2016-11-06 19:42:37,503 | DEBUG | main:ConcatNCopy | Appending smaller file: catalog
context.csv
2016-11-06 19:42:37,503 | DEBUG | main:ConcatNCopy | Copy: catalogcontext.csv to /opt/pim/ETL/MCOM/etlc/output/site/catalogcontext.csv, size: 31, elapsed ms: 0
2016-11-06 19:42:37,504 | DEBUG | main:ConcatNCopy | Appending smaller file: category.csv
2016-11-06 19:42:37,512 | DEBUG | main:ConcatNCopy | Copy: category.csv to /opt/pim/ETL/MCOM/etlc/output/site/category.csv, size: 9478833, elapsed ms: 8
2016-11-06 19:42:37,513 | DEBUG | main:ConcatNCopy | Appending smaller file: categoryfacet.csv
2016-11-06 19:42:37,543 | DEBUG | main:ConcatNCopy | Copy: category
facet.csv to /opt/pim/ETL/MCOM/etlc/output/site/categoryfacet.csv, size: 36649061, elapsed ms: 30
2016-11-06 19:42:37,544 | DEBUG | main:ConcatNCopy | Appending smaller file: contextual
category.csv
2016-11-06 19:42:37,547 | DEBUG | main:ConcatNCopy | Copy: contextualcategory.csv to /opt/pim/ETL/MCOM/etlc/output/site/contextualcategory.csv, size: 2614776, elapsed ms: 3

0 Karma

Builder

Then this should do:
your search | rex "(?<file>\w+\.csv)"

Sames as rich7177 posted. So if this works, accept his answer.

PS some lines have more than one file name, this rex gets them all.

2016-11-06 19:42:36,977 | DEBUG | main:ConcatNCopy | Copy: attr_prod_brand_ship.csv attr_prod_copy_reg.csv attr_prod_data_source.csv attr_prod_forced_new.csv attr_prod_site_search.csv attr_prod_trigger_data.csv attr_prod_tuple_data.csv to
0 Karma

SplunkTrust
SplunkTrust

Try..

 ... | rex "(?<MyCSVFile>\w+\.csv)" 

Change the name MyCSVFile to whatever you want to call it. Here you can see it in regex101.com.