Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for
- About smwirt
smwirt
Path Finder
Member since:
09-19-2012
02-22-2021
Community Statistics
| Posts | 12 |
| Solutions | 1 |
| Karma Given | 0 |
| Karma Received | 3 |
| Member Since | 09-19-2012 |
Activity Feed
- Got Karma for Re: Forwarder is listed in Deployment Monitor but not in Search App. 06-05-2020 12:46 AM
- Got Karma for Re: Seing null (\x00) bytes in indexed data from log file in Windows. 06-05-2020 12:46 AM
- Got Karma for Re: Seing null (\x00) bytes in indexed data from log file in Windows. 06-05-2020 12:46 AM
- Posted Re: Seing null (\x00) bytes in indexed data from log file in Windows on Getting Data In. 10-23-2013 07:06 AM
- Posted Re: Seing null (\x00) bytes in indexed data from log file in Windows on Getting Data In. 10-17-2013 07:42 AM
- Posted Re: Seing null (\x00) bytes in indexed data from log file in Windows on Getting Data In. 10-17-2013 07:04 AM
- Posted Seing null (\x00) bytes in indexed data from log file in Windows on Getting Data In. 10-16-2013 07:21 PM
- Tagged Seing null (\x00) bytes in indexed data from log file in Windows on Getting Data In. 10-16-2013 07:21 PM
- Tagged Seing null (\x00) bytes in indexed data from log file in Windows on Getting Data In. 10-16-2013 07:21 PM
- Tagged Seing null (\x00) bytes in indexed data from log file in Windows on Getting Data In. 10-16-2013 07:21 PM
- Tagged Seing null (\x00) bytes in indexed data from log file in Windows on Getting Data In. 10-16-2013 07:21 PM
- Tagged Seing null (\x00) bytes in indexed data from log file in Windows on Getting Data In. 10-16-2013 07:21 PM
- Posted Re: Forwarder is listed in Deployment Monitor but not in Search App on Getting Data In. 10-16-2013 08:56 AM
- Posted Re: Forwarder is listed in Deployment Monitor but not in Search App on Getting Data In. 10-16-2013 08:20 AM
- Posted Re: Forwarder is listed in Deployment Monitor but not in Search App on Getting Data In. 10-15-2013 05:32 PM
- Posted Re: Forwarder is listed in Deployment Monitor but not in Search App on Getting Data In. 10-15-2013 04:58 PM
- Posted Re: Forwarder is listed in Deployment Monitor but not in Search App on Getting Data In. 10-15-2013 04:36 PM
- Posted Re: Forwarder is listed in Deployment Monitor but not in Search App on Getting Data In. 10-15-2013 04:19 PM
- Posted Re: Forwarder is listed in Deployment Monitor but not in Search App on Getting Data In. 10-15-2013 04:03 PM
- Posted Forwarder is listed in Deployment Monitor but not in Search App on Getting Data In. 10-15-2013 03:54 PM
Topics I've Started
| Subject | Karma | Author | Latest Post |
|---|---|---|---|
| 0 | |||
| 0 |
10-23-2013
07:06 AM
2 Karma
Issue resolved for now: had to set CHARSET = UTF16-LE on props.conf on the forwarders as well as the indexer. I was mistakenly putting the CHARSET line into inputs.conf on the forwarders.
... View more
10-17-2013
07:42 AM
I followed the instructions here (http://answers.splunk.com/answers/83790/how-do-i-remove-x00-characters-from-my-log-message) to remove nulls before indexing (edited props.conf) and the data looks normal now, but for at least one of the hosts so far the timestamp is incorrect. The entry in Splunk that now looks correct has the same timestamp as the previous entries that had \x00 bytes. For another host it is correctly parsing the timestamp from the data.
I'd rather it be correctly processed up front instead of replacing nulls, but if I can get the timestamp correct I can live with it.
... View more
10-17-2013
07:04 AM
Just noticed this line in splunkd.log on the indexer:
WARN UTF8Processor - Using charset UTF-8 for events from 'UTF-16LE', as the monitor is believed over the raw text which may be source:C:\ProgramData\Sophos\Sophos Device Control\logs\DeviceControl.txt|host::my_host|my_sourcetype|remoteport::56789
... View more
10-16-2013
07:21 PM
I have seen several questions regarding null (\x00) bytes in data, but none have helped me resolve my issue so far.
I am trying to read a log file from Sophos using Universal Forwarders. I have done the following so far:
Added a new sourcetype in Splunk Web.
props.conf on the indexer:
[my_sourcetype]
NO_BINARY_CHECK = 1
SHOULD_LINEMERGE = false
TIME_FORMAT = %Y%m%d %H%M%S
TZ = UTC
pulldown_type = 1
CHARSET = UTF-16LE
Modified inputs.conf on the forwarders:
[monitor://C:\ProgramData\Sophos\Sophos Device Control\logs]
sourcetype=my_sourcetype
Sample data from C:\ProgramData\Sophos\Sophos Device Control\logs\DeviceControl.txt:
20131001 150737 Device control has started on this machine.
20131003 131815 Device control has started on this machine.
When I search sourcetype="my_sourcetype", I see data, but it looks like this:
\x002\x000\x001\x003\x00 \x001\x005\x000\x007\x003\x007....
If I copy that data into Notepad, replace \x00 with nothing, then I see the data that I expect.
Before I left tonight, I noticed that the text file I am reading from is blue in Windows Explorer, which indicates the compression bit is set. Every file in this folder is set this way, and removing compression is not an option.
What do I need to do in order to have Splunk index the data without null values? All other data coming from TA-Windows and other apps is fine and does not show null values.
Update 10/17/13:
Wanted to clarify that this is Splunk 4.3.3 on Windows Server 2008 R2 SP1, with Windows 7 SP1 x64 hosts running the Universal Forwarder. Upgrading Splunk is not an option at this time, but we are pushing to do so in the near future.
/etc/system/local/outputs.conf on the forwarder:
[tcpout]
defaultGroup = 1.2.3.4_9997
[tcpout:1.2.3.4_9997]
server = 1.2.3.4:9997
[tcpout-server://1.2.3.4:9997]
/etc/system/local/inputs.conf on the indexer:
[default]
host = my_hostname
[script://$SPLUNK_HOME\bin\scripts\splunk-admon.path]
disabled = 0
[script://$SPLUNK_HOME\bin\scripts\splunk-perfmon.path]
disabled = 0
.... (two more script stanzas)
[monitor://C:\ProgramData\Sophos\Sophos Device Control\logs]
sourcetype=my_sourcetype
Again, all other data coming from the forwarders looks fine without null bytes. Only the data from Sophos is an issue. I am also noticing entries in Splunk with just a single null character as the data (\x00).
... View more
10-16-2013
08:56 AM
1 Karma
Issue resolved!
My apps were not deploying to this system for some reason, so I checked Manager > Deployment > Deployment Server. Per our customer's specification, deployment whitelists were used. This one system had a hostname that did not match the naming convention specified in the whitelist. After adding this system's naming convention to the whitelist, reinstalling the forwarder resolved the issue.
... View more
10-16-2013
08:20 AM
I enabled debugging and haven't found any issues so far in splunkd.log, but I did notice that on the problem system, metrics.log isn't showing any entries with series="winevent" or series="perfmon" (see my edit of the original question). splunkd is running under SYSTEM, so I wouldn't expect there to be permissions issues reading the event log or perfmon.
... View more
10-15-2013
05:32 PM
I have not. I'm not familiar with how to do that, but I should be able to find out easily enough. I am out for the rest of the day, and will try this first thing tomorrow.
... View more
10-15-2013
04:58 PM
The local policies appear to be identical, and the same domain policy is applied to all Windows 7 systems. There are no Group Policy errors shown on any of the systems, working or not working. This is not the first Windows 7 system; there are 3 other Windows 7 systems and 4 other Windows Server 2008 R2 systems that are all forwarding data to Splunk, and all but this one Windows 7 system has indexed data.
Splunk 4.3.3 was released in January 2012, so I don't suspect the age of the forwarder is an issue, especially since other Windows 7 and Windows Server 2008 R2 hosts are working fine.
... View more
10-15-2013
04:36 PM
Permissions on inputs.conf are identical on the Windows host that does have indexed data and the one where I can't find any indexed data. The Universal Forwarder is running under the Local System account, so there shouldn't be any permissions issues there.
The data that I am expecting to see is Windows Event Log data.
I mentioned in the original post that I have reinstalled the forwarder on this machine several times.
Hosts are running Windows 7 Ultimate x64 w/ SP1, and the Splunk indexer is running Windows Server 2008 R2 Enterprise w/ SP1.
... View more
10-15-2013
04:19 PM
The host is brand new, and the time is correct (synced to NTP server).
Also, I ran the search you suggested, and it returns no data. The same search with a different Windows host does return data for that host. I checked outputs.conf and server.conf for this host and a working host, and they are essentially identical other than hostname and GUID.
As I said, data is being received from this host according to Deployment Monitor, but I can't figure out why it isn't indexing any of it.
... View more
10-15-2013
04:03 PM
There is no data being indexed from this host. I have searched for the hostname in all indexes, and the only index that has that hostname or IP is _internal. I see events there where it appears data is being received from this host, and it shows as actively receiving data in the Deployment Monitor app. Events are being generated in Windows and are viewable in Event Viewer, and I have verified using netstat that there is an active connection to the receiving port on the Splunk indexer.
... View more
10-15-2013
03:54 PM
We are running Splunk 4.3.3 (in the process of upgrading but we are stuck on this version for the moment), and one Windows system is not listed in the Search app in the Hosts list. I have searched all internal and non-internal indexes, and can't find any indexed data from this host. It does show in Deployment Monitor, and it shows that it is actively receiving data from this forwarder.
I have reinstalled the Universal Forwarder on this machine several times, but so far nothing changes. The GUID for this forwarder is not duplicated anywhere that I can find.
Any ideas?
Update [10/16/13]:
I compared metrics.log on a forwarder that is working and the one that is not. I noticed that the working system has entries with series="winevent..." (also entries for perfmon) and the system that is not working does not have winevent or perfmon anywhere in metrics.log. At this point, I'm thinking that splunkd can't read the Event Viewer logs or perfmon, but I'm not sure why yet, since splunkd is running as SYSTEM.
... View more
Contact Me
| Online Status |
Offline
|
| Date Last Visited |
02-22-2021
10:49 AM
|
