@ehudb ok this send trace to splunk enterprise, how it show traces? ... View more

@gcusello  (with rsyslog and Universal Forwarder) as I mention can't use forwarder. ... View more

@gcuselloactually two first solution that come to my mind are 1-mount network disk 2- crontab but unfortunately neither approve by product owner. (they have some issue with these solution). as you mention copy file not good idea for large log file, specially they need realtime collection. I'm curious to know is there any solution like splunk forwarder exit for this scenario?   work like this: server1 > server2 > centralize log server3 > FYI1: rsyslog, filebeat, syslog-ng, fluentd, ... are available solution but I can't decide which one is more suitable for this issue. FYI2: raw data is important , and doesn't be missed. (don't want to clean data exact log file is important for me) FYI3: like splunk forwarder whenever servers or network down, after issue resolve it will continuously send data. (AFAIK rsyslog use tracker file when server stopped and try to send remain file after service start again but while service down new file create with different name and structure can't track) (but splunk forwarder can handle this situation even when forwarder service on servers down, whenever start it can discover any file on that path) FYI4: here is the path of my log /opt/log/* different file, with different name may create here , I need to dynamically everything on this path send to the centralize log. ... View more

@gcusellohard to explain, but it is production environment that I don't have privilege to install splunk forwarder. Now looking for something like splunk forwarder. (easy setup, minimum dependency, in case of disaster try to send all file in path)   1-rsyslog remove from my list as you mention.   2-mount network not acceptable by admin of production.   3-fluentd has different dependency. (and AFAIK send tail of file) https://docs.fluentbit.io/manual/administration/configuring-fluent-bit   4-How about logstash or filebeat or any other log collector solution?   ... View more

@gcusello would you please tell me an example? ... View more

@gcusello as i write in post there is no timestamp in this file. ... View more

@gcusello  1-there is no multiline event. 2- how check correctly events parsed? ... View more

@ITWhisperer  1- there are no blank line in file. 2-vi in linux show line numbers. 3-each line one event in splunk. ... View more

@ITWhisperer actually this is summary index 🙂 and still huge! I try to use transform too but increase index time drastically ! Any other idea? ... View more

@ITWhisperer actually i have performance issue, original post modified, I had typo. ... View more

@ITWhispererThanks for answer. what is the different? ... View more

@yuanliu ok here is the query: | metadata type=sources index="app" | eval _time=relative_time (now(), "-1d@d") | eval time=strftime(_time,"%Y%m%d") | table source time here is the result: /data/app/20221122/CUS/app.log                                                                20221122 /data/app/20221122/CUS/app.log.2022-11-22                                 20221122 /data/app/20221119/CUS2/app-exception.log.2022-11-22    20221122 /data/app/20221119/CUS2/app.log.2022-11-22                             20221122   expected result: /data/app/20221122/CUS/app.log                                                                20221122 /data/app/20221122/CUS/app.log.2022-11-22                                 20221122   any idea? Thanks, ... View more

@bowesmana the reason I use metadata is so fast. i encounter with huge files. any other idea? ... View more