Brand new user of Splunk here. I'm currently evaluating Splunk Enterprise. I need a bit of help understanding why Splunk won't let me monitor a file from Microsoft IIS called "web.config". I can see the contents of a file called xxxyyyzzz.log from the same server using a forwarder, so there is not a configuration issue on the client itself.
My guess it has something to do with the source type, but what? One would think that a file such as web.config would be such a common file that source type "automatic" would be able to work? This file never changes of course, except for when we upgrade the system that this file controls. It would save me lots of time if I could see what settings the update destroys for us.
The file I want to see doesn't even show up in "Sources" when trying to search, even though it is entered in the exact same way as the .log file that does show up. What am I doing wrong?
If you have multiple people touching the configuration files then you should really look into some type of version control. An example would include putting all your configs in GIT then committing changes to that and having a consensus on approving the changes before merging them into Splunk.
Using GIT will reduce the number of errors you allow into production and hold people accountable for their changes
That is a good idea, but not what I want to do in this case. My problem is not multiple people touching configuration files, it is that the application itself when it gets updated adds lines that are wrong, or just flat out deletes data. This would be nice to be able to see in splunk.
Dude you're so unhelpful, I really wish this was stackoverflow so your answer could be voted down.
Enforcing your business logic onto others rather than helping them do what they are asking is really annoying, especially for others which have found this post with a similar quandary. I want to do something very similar to what OP was asking, but your perpetual just use git is so unhelpful and adds nothing to the thread.
just so I am understanding you correctly, I am as I said very new to Splunk: You are suggesting to not use Splunk at all in this particular case? Or is it some app that gives the functionality of version control?
If you have the config file in version control, you could use those tools to produce a periodic comparison of what was last committed versus what is currently live. Append that comparison to a log, and have the Universal Forwarder watch that log.
You can then set up an alert to tell you when there were any changes, and both identify undesired changes and changes that need to be committed to source control.
Additionally, did you configure your forwarder to collect "web.config" file as you did with "xxxyyyzzz.log"?
In case you haven't already, please refer to below links to learn more about source types.
yes, from what I can see they are configured exactly the same. I've tried looking at those links before I posted here, it does not shed any light on how this is supposed to be configured, at least not to a rookie Splunker like me. I take it that I will have to create a brand new sourcetype in the props.conf file on the host with the file I want to monitor?
Ah, gotcha. Can you provide sample data so that I can give you basic props.conf structure.
If your file looks similar to below web.config (.NET) file, then this is how your inputs.conf and props.conf should be. If you can explain a little more about your architecture, then I can tell you what all places should these configurations go.
## Non-windows [monitor:///path_to_file/web.config] index=myidx sourcetype=web_config ## Windows [monitor://C:\path\to\file\web.config] index=myidx sourcetype=web_config
[web_config] LINE_BREAKER = ([\r\n]+)\<\?xml\sversion\= SHOULD_LINEMERGE = false KV_MODE = xml
<?xml version="1.0"?> <configuration xmlns="http://schemas.microsoft.com/.NetConfiguration/v2.0"> <appSettings/> <connectionStrings/> <system.web> <compilation debug="false"/> <authentication mode="Windows"/> <customErrors mode="RemoteOnly" defaultRedirect="GenericErrorPage.htm"> <error statusCode="403" redirect="NoAccess.htm"/> <error statusCode="404" redirect="FileNotFound.htm"/> </customErrors> </system.web> </configuration>
Thanks for taking the time to help. Below is a portion of the start of the file, and some part that contain the data that is interesting to monitor:
The start of web.config:
<section name="system.identityModel" type="System.IdentityModel.Configuration.SystemIdentityModelSection, System.IdentityModel, Version=188.8.131.52, Culture=neutral, PublicKeyToken=B77A5C561934E089" /> <section name="system.identityModel.services" type="System.IdentityModel.Services.Configuration.SystemIdentityModelServicesSection, System.IdentityModel.Services, Version=184.108.40.206, Culture=neutral, PublicKeyToken=B77A5C561934E089" /> <!-- DYNAMIC DEBUG COMPILATION Set compilation debug="true" to enable ASPX debugging. Otherwise, setting this value to false will improve runtime performance of this application. Set compilation debug="true" to insert debugging symbols (.pdb information) into the compiled page. Because this creates a larger file that executes more slowly, you should set this value to true only when debugging and to false at all other times. For more information, refer to the documentation about debugging ASP.NET files. --> <compilation defaultLanguage="c#" debug="false" targetFramework="4.5.2"> <assemblies> <add assembly="Microsoft.VisualC, Version=10.0.0.0, Culture=neutral, PublicKeyToken=B03F5F7F11D50A3A" /> <add assembly="System.Data.OracleClient, Version=220.127.116.11, Culture=neutral, PublicKeyToken=B77A5C561934E089" /> <add assembly="System.DirectoryServices, Version=18.104.22.168, Culture=neutral, PublicKeyToken=B03F5F7F11D50A3A" /> <add assembly="System.Windows.Forms, Version=22.214.171.124, Culture=neutral, PublicKeyToken=B77A5C561934E089" /> <add assembly="System.Design, Version=126.96.36.199, Culture=neutral, PublicKeyToken=B03F5F7F11D50A3A" /> <add assembly="System.Runtime.Serialization.Formatters.Soap, Version=188.8.131.52, Culture=neutral, PublicKeyToken=B03F5F7F11D50A3A" /> </assemblies> </compilation>
The data I am interested in monitoring in the file:
<endpoint address="net.tcp://localhost/SECRET/Services/SecurityServices/AuthorizationService.svc" binding="netTcpBinding" bindingConfiguration="NetTcpBinding_IAuthorizationService" contract="AuthorizationService.IAuthorizationService" name="AuthorizationService" /> <endpoint address="https://secret.domain.com/Services/PServices/ESSBasic.svc" binding="wsHttpBinding" bindingConfiguration="secureWsHttpBinding" contract="ESSBasicService.IESSBasic" name="BasicHttpBinding_IESSBasic" /> <endpoint address="https://secret.domain.com/Services/PServices/ESSFlex.svc" binding="wsHttpBinding" bindingConfiguration="secureWsHttpBinding" contract="ESSFlexService.IESSFlex" name="BasicHttpBinding_IESSFlex" /> </client> <clear /> <add key="ConfigInNeptune" value="True" /> <add key="DBAPISection" value="RE_ODBC" /> <add key="DBPath" value="D:\Visma\Programs\Ciceron\" /> <add key="NeptuneConnectionName" value="SECRET" /> <add key="NeptuneSSO" value="False" /> <add key="PathToTravel" value="" /> <add key="TurbyteRastVarning" value="True" /> <add key="ActionLog" value="True" /> <add key="SkickaEmailFrMeddBemRes" value="True" /> <add key="VarningGammalAvvTjg" value="True" /> <add key="ShowRetro" value="True" /> <add key="ShowDoctorInterface" value="True" /> <add key="CheckOverDraft" value="True" /> <add key="ArbtBemFranvorsak" value="False" /> <add key="AlltidKostAvdrag" value="False" /> <add key="CheckCompensatoryBalance" value="-1" /> <add key="ProxyPath" value="" /> <add key="CheckAbsenceHighErrorMsgLevel" value="False" /> <add key="OblArbStalleVidKonto" value="False" /> <add key="WindowsIdentityFoundationEnabled" value="True" /> <add key="VismaWindowOnly" value="False" /> <add key="AMRunInIframePath" value="" /> <add key="AMDashBoardPath" value="/SECRET/Dashboard/Home/Get/1" /> <add key="AMSaldoTileMaxOkomp" value="50" /> <add key="AMSaldoTileShowSem" value="True" /> <add key="AMSaldoTileMaxOtid" value="200" /> <add key="AMFlexTileUseButtons" value="False" /> <add key="ESSBasicServiceLink" value="http://localhost/PWeb.Services/ESSBasic.svc/Web/" /> <add key="ValidationSettings:UnobtrusiveValidationMode" value="None" /> <add key="WIFLogoutLink" value="" /> <add key="TravelShowHigherCompensationWarning" value="False" />
My infrastructure regarding Splunk is basically just a single standalone server that will be collecting data from other servers in the same domain using forwarders that are installed on the hosts. Simplest possible installation in other words.
Try this props.conf:
[web_config] LINE_BREAKER = ([\r\n]+)\s?\<section\sname\= SHOULD_LINEMERGE = false KV_MODE = xml DATETIME_CONFIG = NONE # "NONE" will leave the event time set to whatever time was selected by the input layer, since there is no timestamp in the events
Note: It is difficult to ingest only some part of the data in file. I am not saying it is impossible but if having few extra lines doesn't hurt you, then ingest the whole file.
Place this on both UF and your standalone server and restart splunk. Let me know how it goes.
will try to do this during the week! I noticed there are a lot of props.conf on the server, which one is the one I am supposed to edit?
The documentation say you should edit the file in $SPLUNK_HOME/etc/system/local/
I don't have any props.conf in that location, am I supposed to create a new file at that location? Or should I make a copy of the one in "C:\Program Files\Splunk\etc\system\default" and add the things you suggested in that file?
Tried your suggestions, probably I am doing something wrong because I get no data collected. Here is what I have done:
LINE_BREAKER = ([\r\n]+)\s?\
Hi, thanks for the answer.
In splunk - Add data
Forward data from Splunk forwarder - Selected my host and serverclass
Files & Directories - Enter the path to the file I want to index, using local path on the server that has the file.
next next done.
This works if I put in the path of a file with the extension .log, but if I enter a file with the extension .config, nothing shows up anywhere.