Getting Data In

Why is the Windows Event Collector not parsing correctly?

lukas_loder
Communicator

Hi Splunkers

I have a problem with my Windows Event Collector (Windows Server 2012 R2). I'm not able to install a Universal Forwarder on every system. So we are collecting data with a Windows Event Collector. On this Server I have installed a Universal Forwarder and the Splunk_TA_windows app. There I've created a new input stanza like this:

[WinEventLog://ForwardedEvents] 
sourcetype=WinEventLog:ForwardedEvents 
disabled = 0 
start_from = oldest 
current_only = 0 
evt_resolve_ad_obj = 1 
checkpointInterval = 5 
index = Test 
renderXml=false

This works for indexing data, but unfortunately, the log in Splunk is wrongly parsed.

03/23/2018 02:01:13 PM
LogName=Security
SourceName=Microsoft Windows security auditing.
EventCode=5061
EventType=0
Type=Microsoft Windows security auditing.
ComputerName=XY.xy.intranet
TaskCategory=Microsoft Windows security auditing.
OpCode=Microsoft Windows security auditing.
RecordNumber=189333
Keywords=Microsoft Windows security auditing.
Message=Microsoft Windows security auditing.

As you can see the fields are wrong because it says everywhere "Microsoft Windows Security auditing". But in the EventViewer I see all the Information correct.
All the other Windows Event Logs from System, Application, Setup and Security are perfectly fine in Splunk.
There is no Special parsing, props, transforms for those logs. I've installed the Splunk_TA_windows on every instance (UF, HF, Indexer, SH).

Does somebody know this issue? Is the format another one of the Windows Event Collector?

Thanks for your help and kind regards,
Lukas

0 Karma

chje
Explorer

Hi,

I just wanted to give my thoughts on this as I have recently experienced the exact same issue.
The thing that solved it for us was that the log format on the WEF server for "Forwarded Events" was in mode "Rendered Text". When changed to mode "Events", the parsing became correct.
Cheers.

splk
Communicator

Great Tip, can you also share what your inputs.conf is looking?

0 Karma

gerald_contrera
Path Finder

Hi Chje,

This sounds very similar to my problem.

Where do you change the log format on the WEF server?

0 Karma

FrankVl
Ultra Champion

You can do that using wecutil, to change the subscription settings. For this specific setting, it would be:

wecutil ss SUBSCRIPTION_ID /cf:Events

https://docs.microsoft.com/en-us/windows/desktop/wec/wecutil

0 Karma

FrankVl
Ultra Champion

Not sure if it will entirely solve your issue, but you will want to rewrite the source and sourcetype of the forwarded events to what they would be when collected locally (so the regular security/system/application sourcetypes for windows data). Otherwise the search time configuration of Splunk_TA_Windows will not apply properly.

For example:

props.conf

[WinEventLog:ForwardedEvents]
TRANSFORMS-force_sourcetype_for_fwd_events = force_sourcetype_for_fwd_events
TRANSFORMS-force_source_for_fwd_events = force_source_for_fwd_events

transforms.conf

[force_sourcetype_for_fwd_events]
DEST_KEY = MetaData:Sourcetype
REGEX = LogName=(\S+)
FORMAT = sourcetype::WinEventLog:$1

[force_source_for_fwd_events]
DEST_KEY = MetaData:Source
REGEX = LogName=(\S+)
FORMAT = source::WinEventLog:$1

richgalloway
SplunkTrust
SplunkTrust

What do you get when you run splunk btool props list WinEventLog://ForwardedEvents?

---
If this reply helps you, Karma would be appreciated.
0 Karma

lukas_loder
Communicator

Hi Rich

Thanks for your answer
With you command i get nothing.
When i change to "WinEventLog:ForwardedEvents" (without the slashes) i get following:
[WinEventLog:ForwardedEvents]
ADD_EXTRA_TIME_FIELDS = True
ANNOTATE_PUNCT = True
AUTO_KV_JSON = true
BREAK_ONLY_BEFORE =
BREAK_ONLY_BEFORE_DATE = True
CHARSET = AUTO
DATETIME_CONFIG = \etc\datetime.xml
DEPTH_LIMIT = 1000
HEADER_MODE =
LEARN_MODEL = true
LEARN_SOURCETYPE = true
LINE_BREAKER_LOOKBEHIND = 100
MATCH_LIMIT = 100000
MAX_DAYS_AGO = 2000
MAX_DAYS_HENCE = 2
MAX_DIFF_SECS_AGO = 3600
MAX_DIFF_SECS_HENCE = 604800
MAX_EVENTS = 256
MAX_TIMESTAMP_LOOKAHEAD = 128
MUST_BREAK_AFTER =
MUST_NOT_BREAK_AFTER =
MUST_NOT_BREAK_BEFORE =
SEGMENTATION = indexing
SEGMENTATION-all = full
SEGMENTATION-inner = inner
SEGMENTATION-outer = outer
SEGMENTATION-raw = none
SEGMENTATION-standard = standard
SHOULD_LINEMERGE = True
TRANSFORMS =
TRANSFORMS-001-sethost_sourcetype = Set-Host-By-ComputerName
TRUNCATE = 10000
detect_trailing_nulls = auto
maxDist = 100
priority =
sourcetype =

Is this the error? Do I may Need to change the input sourcetype or something? Also in Splunk I have the Events without the "//" sourcetype="WinEventLog:ForwardedEvents"

Thanks

0 Karma

richgalloway
SplunkTrust
SplunkTrust

You appear to have the sourcetype correct in your inputs file (I copied the wrong part of your OP into my answer).
The next thing to do is look at some sample events. Get them from a Windows box, not from Splunk. Look at them and compare them to the props.conf settings above. It's possible your WIndows boxes are writing events in an unexpected format that isn't parsed correctly by those props.conf settings.

---
If this reply helps you, Karma would be appreciated.
0 Karma

lukas_loder
Communicator

I just checked the local logs on the system and they look different to the normal Event logs:

<Provider Name="Microsoft-Windows-Security-Auditing" Guid="{54849625-5478-4994-A5BA-3E3B0328C30D}" /> 
<EventID>4624</EventID> 
<Version>1</Version> 
<Level>0</Level> 
<Task>12544</Task> 
<Opcode>0</Opcode> 
<Keywords>0x8020000000000000</Keywords> 
<TimeCreated SystemTime="2018-03-26T07:12:32.452412800Z" /> 
<EventRecordID>2217046350</EventRecordID> 
<Correlation /> 
<Execution ProcessID="700" ThreadID="17588" /> 
<Channel>Security</Channel> 
<Computer>XY.domain.intra</Computer> 
<Security /> 


<Data Name="SubjectUserSid">S-1-0-0</Data> 
<Data Name="SubjectUserName">-</Data> 
<Data Name="SubjectDomainName">-</Data> 
<Data Name="SubjectLogonId">0x0</Data> 
<Data Name="TargetUserSid">S-1-5-21-00000-00000-00000-00000</Data> 
<Data Name="TargetUserName">CompetellaSVCAccount</Data> 
<Data Name="TargetDomainName">DOMAIN</Data> 
<Data Name="TargetLogonId">0x2e33XXXX</Data> 
<Data Name="LogonType">3</Data> 
<Data Name="LogonProcessName">NtLmSsp</Data> 
<Data Name="AuthenticationPackageName">NTLM</Data> 
<Data Name="WorkstationName">XY</Data> 
<Data Name="LogonGuid">{00000000-0000-0000-0000-000000000000}</Data> 
<Data Name="TransmittedServices">-</Data> 
<Data Name="LmPackageName">NTLM V2</Data> 
<Data Name="KeyLength">128</Data> 
<Data Name="ProcessId">0x0</Data> 
<Data Name="ProcessName">-</Data> 
<Data Name="IpAddress">172.1.1.1</Data> 
<Data Name="IpPort">53000</Data> 
<Data Name="ImpersonationLevel">%%1833</Data> 


<Message>An account was successfully logged on. Subject: Security ID: S-1-0-0 Account Name: - Account Domain: - Logon ID: 0x0 Logon Type: 3 Impersonation Level: Impersonation New Logon: Security ID: S-1-5-21-2088249647-00000-00000-00000Account Name: CompetellaSVCAccount Account Domain: DOMAIN Logon ID: 0x2E330000 Logon GUID: {00000000-0000-0000-0000-000000000000} Process Information: Process ID: 0x0 Process Name: - Network Information: Workstation Name: SRVXY Source Network Address: 172.1.1.1 Source Port: 53000 Detailed Authentication Information: Logon Process: NtLmSsp Authentication Package: NTLM Transited Services: - Package Name (NTLM only): NTLM V2 Key Length: 128 This event is generated when a logon session is created. It is generated on the computer that was accessed. The subject fields indicate the account on the local system which requested the logon. This is most commonly a service such as the Server service, or a local process such as Winlogon.exe or Services.exe. The logon type field indicates the kind of logon that occurred. The most common types are 2 (interactive) and 3 (network). The New Logon fields indicate the account for whom the new logon was created, i.e. the account that was logged on. The network fields indicate where a remote logon request originated. Workstation name is not always available and may be left blank in some cases. The impersonation level field indicates the extent to which a process in the logon session can impersonate. The authentication information fields provide detailed information about this specific logon request. - Logon GUID is a unique identifier that can be used to correlate this event with a KDC event. - Transited services indicate which intermediate services have participated in this logon request. - Package name indicates which sub-protocol was used among the NTLM protocols. - Key length indicates the length of the generated session key. This will be 0 if no session key was requested.</Message> 
<Level>Information</Level> 
<Task>Logon</Task> 
<Opcode>Info</Opcode> 
<Channel>Security</Channel> 
<Provider>Microsoft Windows security auditing.</Provider> 
<Keywords>
  <Keyword>Audit Success</Keyword> 
</Keywords>
</RenderingInfo>


you know if it's possible to change the input collection for this sourcetype to match the changes? Or you know how the system collects the data?

thanks again!

0 Karma
Get Updates on the Splunk Community!

Splunk Forwarders and Forced Time Based Load Balancing

Splunk customers use universal forwarders to collect and send data to Splunk. A universal forwarder can send ...

NEW! Log Views in Splunk Observability Dashboards Gives Context From a Single Page

Today, Splunk Observability releases log views, a new feature for users to add their logs data from Splunk Log ...

Last Chance to Submit Your Paper For BSides Splunk - Deadline is August 12th!

Hello everyone! Don't wait to submit - The deadline is August 12th! We have truly missed the community so ...