I am new with Splunk, I have the following question/issue:
My goal is to parse a raw log file with Splunk and save and download/extract the new generated structured log file as a csv file.
I ran this SPL search:
host="vm-nc-23" index="deeplog" sourcetype="hdfs_log" Content=* | eval rex_template=replace(Content,"[_]*[-]*\d+","*") | cluster t=0.5 labelonly=true labelfield=Template match=termlist field=rex_template | stats count AS Occurences, values(rex_template) AS REGEX_Expressions by Template
Which outputs a set of n(=35) templates (the 'Content' field of logs, where variables/parameters are replaced with wildcards).
The templates can be viewed at the "Statistics" Tab and can be exported as a CSV file.
Instead, I would like to export the structured logs which include the additional field 'Template' among others (Time, PID, Component, Content, Level, etc.) from our SPL search. The structured logs can be vieweb in "Events" Tab.
I am confident that there isn't an option for exporting this from Splunk GUI. Any advice on how I could achieve this?
Thank you in advance!
Do you mean that you can only see the lognames in your events tab? But you want to see the contents of the log files from splunk search commands?
You have to index those logs so that you can extract what you wanted and export into csvs. You can't export log-files as such from splunk.
Not sure if you are looking to outputlookup into csv?
<your base search> | outputlookup structured_logs.csv
After this, you can read your lookupfile using
|inputlookup outputlookup structured_logs.csv
or find it under
That's helpful to know thank you. Still the lookup file i export as you described are the "templates", that is the lines generated in Statistics tab.
Is there a way to export the structured logs from the Events tab, similarly to the way you described above?
Also, since Splunk interface (i use a web browser to use Splunk, i do not have it installed locally) shows in the Event tab the 'structured logs' this means that Splunk saves the logs somewhere. However if i access my VM where Splunk is installed, i try:
grep -r 'string i want to find' in the Splunk's filepath and i get no results with files containing this string, indicating that there is no file with the structured logs to download.
How can i get access to the Event tabs content?
Using the following SPL query:
host="vm-nc-23" index="deeplog" sourcetype="hdfs_log" Content=* | eval rex_template=replace(Content,"[_]*[-]*\d+","*")
i found out that i am able to extract manually the .csv file of the "structured" logs, that is the logs with the fields that Splunk defines.
In these fields, there is also 1 field called "rex_template which extracts the Template of a given log line, as we have defined it here:
This way of selecting the templates looks very basic. What would we do if we want to extract the structured logs, where the rex_template column is better structured?
an example of the current "basic" template extraction would be this:
Received block blk* src: /...😘 dest: /...😘 of size * Receiving block blk* src: /...😘 dest: /...😘
This is "ugly", instead i would like to have for example:
Received block blk<> Receiving block blk<>
(Note: src: /...😘 dest: /...😘 of size * we would like not to have this as well, it is not useful for my purpose)
Thank you all in advance.