I am fairly new to Splunk. I am trying to reduce ingestion size of logs that are coming from the Palo Alto Cloud connections. A lot of values come back null or are useless for our applications. I have tried to use the premade Palo Alto filter for log ingestion but it has no effect on pan:threat:cloud or pan:traffic:cloud sourcetypes. I tried eval commands to change the sourcetypes to their noncloud counterparts (pan:threat/pan:traffic) but they still come out unchanged. When I made my own pipeline it does change the data (removing the unused fields) and then sends it off to the default indexer where it is no longer showing under the "Palo Alto Networks FLS LF" source. I assume that since I have converted it to a json splunk doesn't auto recognize it as a Palo Alto log and is indexing it in a different index. I will set up an S3 bucket for the raw logs after this. Traffic Log Pipeline: $pipeline = | from $source | flatten _raw | fields - DestinationDeviceCategory, DestinationDeviceMac, DestinationDeviceOSFamily, DestinationDeviceOSVersion, SourceDeviceProfile, ChunksReceived, ChunksTotal, SDWANSite, LinkChangeCount, DestinationDeviceHost, DestinationUUID, SDWANClusterType, SDWANPolicyName, VirtualSystemName, SourceEDL, _raw, ContainerName, IMSI, SDWANDeviceType, NSSAINetworkSliceType, LinkSwitches, EndpointSerialNumber, DestinationEDL, Tunnel, DestinationDeviceVendor, DestinationDeviceModel, DestinationDynamicAddressGroup, ChunksSent, SourceDeviceVendor, ContainerNameSpace, GPHostID, SourceDeviceOSVersion, NSSAINetworkSliceDifferentiator, SourceDeviceModel, SourceDeviceOSFamily, EndpointAssociationID, SourceDeviceMac, SourceDeviceCategory, SourceDynamicAddressGroup, ParentSessionID, SourceDeviceHost, SDWANCluster, DestinationDeviceProfile, SourceUUID, IMEI, ContainerID, DynamicUserGroupName, HASessionOwner | select _time, RepeatCount, SessionStartTime, DestinationPort, HTTP2Connection, sourcetype, DGHierarchyLevel1, DestinationLocation, NATDestination, _kind, PacketsSent, LogSetting, VirtualLocation, index, _forwarder_channel_info, SourceUser, Rule, FromZone, Application, SourceLocation, SessionDuration, Protocol, DestinationUser, PacketsTotal, DGHierarchyLevel2, DGHierarchyLevel3, DGHierarchyLevel4, DestinationAddress, ToZone, ActionSource, DeviceName, SequenceNo, RuleUUID, source, Bytes, SessionEndReason, host, NATDestinationPort, PacketsReceived, DeviceSN, ConfigVersion, ParentStarttime, BytesSent, SessionID, Subtype, NATSourcePort, Action, InboundInterface, SourcePort, URLCategory, OutboundInterface, SourceAddress, TimeGenerated, TimeGeneratedHighResolution, LogType, TimeReceived, BytesReceived, NATSource | into $destination;
... View more
