Splunk Search

Create field names automatically from line in file

nate1
Explorer

Below are the first 7 lines of a file that I want to index. The additional lines all look like line 7. Can I have it ignore the first 5 lines and then use the names from the 5th line to automatically name the fields for the remaining records?

This would be similar to how you can tell excel to use a certain line for the column names.

Thanks in advance.

#Version: 4
#Producer: xyz.11
#YourID: 36c2df75-2ec7-43b7-870e-f5d38f08be6c
#HTTPErrorCodes: 4,1,401:407
#HTTPErrorCodes: 4,2,404
#Fields: date time c-ip:ip internalClientIP:ip cs-username c-port s-computername s-ip:ip cs-method cs-uri-stem cs-uri-query sc-status sc-win32-status:int:hex httpErrorBucket sc-bytes cs-bytes time-taken s-port sc(Content-Type) cs(User-Agent) cs(Cookie) cs(Referer) transId:long:hex begT pageBegDT reqBegDT reqEndDT resBegDT srvEndDT sslSetupT sRtt cRtt sByte cByte sPkt cPkt sRx cRx sRxPC cRxPC cZeroWin appErr:int:hex appErrFlag:int:hex appErrMsgs:json sContentLen cContentLen appName appThold cCookieLen cCookieCnt sCookieLen sCookieCnt sHdrLen cHdrLen postDataLen sslProtocol sslCipher sslCiphKeySize sslSessID sslCertCliDN sslCertSrvDN hitId:long:hex rdrT recType:int:hex corrId userMetrics:json hdataOff:long:hex


2010-04-29 13:14:59 10.16.135.130 10.16.135.130 - 2281 servicedesk.corp.test.com 10.28.10.12 GET /CAisd/html/deferred_tab.html nbtab=0+tabindex=200 200 40 0 3309 1174 61 80 text/html Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+5.1;+.NET+CLR+2.0.50727;+.NET+CLR+3.0.4506.2152;+.NET+CLR+3.5.30729;+.NET+CLR+1.1.4322) LtpaToken=fuLQqT1au7AEShG5aygAb57rhgExT5qWvI+AL5/kzluoi1cuc31FjgPFMs4GFzMxUlLGSKKK1uK72rLM0mKMlcOP2sQeYGtCFP6LBRWGpGDQchbIdBkWLTvsHoTObO6hiGdX6OTb273yYP5aW+kUAm2t5xKx38i5VvQA/r16DJ9TpFhA39AV6ux2uqdqYC http://servicedesk.corp.test.com/CAisd/pdmweb.exe?SID=664743471+FID=1564+OP=SHOW_DETAIL+FACTORY=cr+P... 4bd9be930035ad70 1272561299944 0 0 0 1 1 0 1 1 3309 1174 3 1 0 0 0 0 0 0 0 - 3029 -1 HTTP 8000 354 1 0 0 280 1174 - - - - - - - 4bd9be93002b6808 - 1 - - -
Tags (1)
1 Solution

sideview
SplunkTrust
SplunkTrust

Our docs on the topic as I understand it are here:

http://www.splunk.com/base/Documentation/latest/Admin/Extractfieldsfromfileheadersatindextime

However I see your problem - they only talk about the simpler cases where the header line is the first line. This might require a scripted input.

some related links:

http://answers.splunk.com/questions/1902/iis-and-exchange-log-header-extraction

http://www.splunk.com/support/forum:SplunkAdministration/3395

View solution in original post

sideview
SplunkTrust
SplunkTrust

Our docs on the topic as I understand it are here:

http://www.splunk.com/base/Documentation/latest/Admin/Extractfieldsfromfileheadersatindextime

However I see your problem - they only talk about the simpler cases where the header line is the first line. This might require a scripted input.

some related links:

http://answers.splunk.com/questions/1902/iis-and-exchange-log-header-extraction

http://www.splunk.com/support/forum:SplunkAdministration/3395

nate1
Explorer

Thanks for your help. These links will at least get me started. It looks like the "Exchange" log is similar to what I want.

0 Karma
Get Updates on the Splunk Community!

Stay Connected: Your Guide to July Tech Talks, Office Hours, and Webinars!

What are Community Office Hours?Community Office Hours is an interactive 60-minute Zoom series where ...

Updated Data Type Articles, Anniversary Celebrations, and More on Splunk Lantern

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

A Prelude to .conf25: Your Guide to Splunk University

Heading to Boston this September for .conf25? Get a jumpstart by arriving a few days early for Splunk ...