As I had a similar problem - to count the parallel/concurrent HTTP requests grouping by time and host (which means the active threads in each server), I provide my solution: index=jira-prod source="/opt/jira/logs/access_log*"
| rex field=_raw "^(?<IP>\d+\.\d+\.\d+\.\d+) (?<REQUEST_ID>[0-9]+x[0-9]+x[0-9]+) (?<USER>\S+) \[.+\] \"(?<REQUEST>[A-Z]+ \S+)-? HTTP/1.1\" (?<STATUS>[0-9]+) (?<BYTES>[0-9]+) (?<TIME>[0-9]+) \"(?<REFERER>[^\"]+)\".*$"
| eval DURATION=TIME/1000
| eval START_AT=floor(_time-DURATION)
| eval END_AT=floor(_time)
| eval IN_MOMENT=mvrange(START_AT,END_AT,1)
| mvexpand IN_MOMENT
| eval _time=strptime(""+IN_MOMENT,"%s")
| chart count as COUNT, max(DURATION) as MAX_DURATION by _time, host This is parsing a real log file of Atlassian JIRA where: line 2 parses the JIRA access log and determines its elements, including the duration in milliseconds of the request. Note that the request is logged at the moment it is complete thus _time is the end time lies 3-5 calculate the duration in seconds, start second and end second line 6 fills in IN_MOMENT each of the seconds the request is active, having at least one value when the start second is equal to the end second line 7 duplicates the even for each of the seconds listed in IN_MOMENT, setting the event's IN_MOMENT field to the current second as a regular single value line 8 is more a hack - convert the IN_MOMENT from epoch number into a timestamp line 9 calculate as whatever statistics/chart/timechart needed grouping by _time and host This worked fine for me.
... View more
