Contains a summary of thread usage

Question

I have an application that writes logs every time a request is made to it. The date and time information refers to the end time of the function. Using the runtime, we calculate the start time of the request.

Here's what the logs look like:

year | month | day | hour | minute | seconds | Thread | UserName           | ExecTime
2014 | 3     | 26  | 8    | 57     | 300     | 1      | file_download      | 33.86
2014 | 3     | 26  | 8    | 57     | 45      | 1      | file_upload        | 0.359
2014 | 3     | 26  | 8    | 57     | 55      | 1      | folder_browse      | 0.234
2014 | 3     | 26  | 8    | 58     | 11      | 1      | sending_email      | 0.14
2014 | 3     | 26  | 8    | 58     | 20      | 1      | browsing_favorites | 0.985
2014 | 3     | 26  | 8    | 58     | 29      | 1      | file_download      | 0.266
2014 | 3     | 26  | 8    | 58     | 33      | 1      | file_upload        | 0.296
2014 | 3     | 26  | 8    | 58     | 37      | 1      | file_zip           | 0.25
2014 | 3     | 26  | 8    | 58     | 41      | 1      | view_properties    | 0.0
2014 | 3     | 26  | 8    | 58     | 46      | 1      | file_download      | 0.187

For illustration only

As the query can span several minutes or even hours, it becomes difficult to calculate each usage of the threads per minute. The following result shows the result that I get after:

Thread | month | day | hour | minute | % Busy
1      | 3     | 26  | 8    | 57     | 0.5
2      | 3     | 26  | 8    | 57     | 0.1
3      | 3     | 26  | 8    | 57     | 0.9

For illustration only

For example, if the request started at 12:30:30 and ended at 12:32:30, the above table would look like this:

Thread | month | day | hour | minute | % Busy
1      | 3     | 26  | 12   | 29     | 0
1      | 3     | 26  | 12   | 30     | 0.5
1      | 3     | 26  | 12   | 31     | 1
1      | 3     | 26  | 12   | 32     | 0.5
1      | 3     | 26  | 12   | 33     | 0

For illustration only

The number of minutes left will be the number of minutes between the first request and the last request that can be found in the logs.

UPDATE

, @TessellatingHeckler :

• / .
• 100% .
• / .
• 0% .
• , , %

, , , . perl, powershell java.

Answer 1

- . , "", :

use List::Util qw(sum);

use strict;
use warnings;

<DATA>; # Skip Header

my %thread;
while (<DATA>) {
    chomp;
    my @data = split /\s*\|\s*/;
    push @{$thread{$data[6]}}, \@data;
}

print "Thread | month | day | hour | minute | ExecTime\n";
for my $id (sort {$a <=> $b} keys %thread) {
    my $time = sum map {$_->[7]} @{$thread{$id}};
    printf "%-6s | %-5s | %-3s | %-4s | %-6s | %.03f\n", $id, @{$thread{$id}[0]}[1,2,3,4], $time;
}

__DATA__
year | month | day | hour | minute | seconds | Thread |  ExecTime ( seconds )
2014 | 3     | 26  | 8    | 57     | 32      | 1      |  33.86
2014 | 3     | 26  | 8    | 57     | 45      | 2      |  0.359
2014 | 3     | 26  | 8    | 57     | 55      | 3      |  0.234
2014 | 3     | 26  | 8    | 58     | 11      | 1      |  0.14
2014 | 3     | 26  | 8    | 58     | 20      | 2      |  0.985
2014 | 3     | 26  | 8    | 58     | 29      | 3      |  0.266
2014 | 3     | 26  | 8    | 58     | 33      | 3      |  0.296
2014 | 3     | 26  | 8    | 58     | 37      | 1      |  0.25
2014 | 3     | 26  | 8    | 58     | 41      | 2      |  0.0
2014 | 3     | 26  | 8    | 58     | 46      | 1      |  0.187

:

Thread | month | day | hour | minute | ExecTime
1      | 3     | 26  | 8    | 57     | 34.437
2      | 3     | 26  | 8    | 57     | 1.344
3      | 3     | 26  | 8    | 57     | 0.796

Answer 2

, PowerShell. script :

• (, - exectime (seconds))
• ( )
• % Busy ExecTime/60 , 60 ExecTime 100%
• 0% , .

:

• ExecTime . % Busy /60 -. , . , 8.57.32 34 , 56% 56- . , ExecTime ExecTime - , ExecTimes , exectime -.

Script:

$path = c:\test.txt
$data = [System.IO.File]::ReadAllText($path).Replace(" ","") | ConvertFrom-Csv -Delimiter "|"

$data | Group-Object Thread | % {

    #Get start-time
    $_.Group | % { $_ | Add-Member -MemberType NoteProperty -Name StartTime -Value ([datetime]("{0}/{1}/{2} {3}:{4}:{5}" -f $_.month, $_.day, $_.year, $_.hour, $_.minute, $_.seconds)).AddSeconds(-($_.ExecTime)) }

    #Group by minute
    $_.Group | Group-Object {$_.starttime.minute} | % {
        #Report per minute
        New-Object psobject -Property @{
            "Thread" = $_.Group[0].thread
            "Month" = $_.Group[0].starttime.month
            "Day" = $_.Group[0].starttime.day
            "Hour" = $_.Group[0].starttime.hour
            "Minute" = $_.Group[0].starttime.minute
            "% Busy" = [math]::Round((($_.Group | Measure-Object -Property ExecTime -Sum).Sum / 60 * 100),2)
        }
    }

    #Create 0% record for missing minutes
    $min = ($_.Group | Sort-Object StartTime)[0].StartTime
    $max = ($_.Group | Sort-Object StartTime)[-1].StartTime

    for($d = $min; $d -le $max; $d = $d.AddMinutes(1)) {
        if(-not ($_.Group | ? { $_.StartTime.Minute -eq $d.Minute })) {
            New-Object psobject -Property @{
                "Thread" = $_.Group[0].thread
                "Month" = $d.Month
                "Day" = $d.Day
                "Hour" = $d.Hour
                "Minute" = $d.Minute
                "% Busy" = 0
            }
        }
    }


} | Sort-Object Day, Hour, Minute, Thread | Select-Object "Thread", "Month", "Day", "Hour", "Minute", "% Busy"

test.txt

year | month | day | hour | minute | seconds | Thread |  ExecTime
2014 | 3     | 26  | 8    | 57     | 32      | 1      |  33.86
2014 | 3     | 26  | 8    | 57     | 45      | 2      |  0.359
2014 | 3     | 26  | 8    | 57     | 55      | 3      |  0.234
2014 | 3     | 26  | 8    | 58     | 11      | 1      |  0.14
2014 | 3     | 26  | 8    | 58     | 20      | 2      |  0.985
2014 | 3     | 26  | 8    | 58     | 29      | 3      |  0.266
2014 | 3     | 26  | 8    | 58     | 33      | 3      |  0.296
2014 | 3     | 26  | 8    | 58     | 37      | 1      |  0.25
2014 | 3     | 26  | 8    | 58     | 41      | 2      |  0.0
2014 | 3     | 26  | 8    | 58     | 46      | 1      |  0.187
2014 | 3     | 26  | 9    | 00     | 5       | 1      |  0.0
2014 | 3     | 26  | 9    | 00     | 8       | 1      |  0.187

:

Thread Month Day Hour Minute % Busy
------ ----- --- ---- ------ ------
1          3  26    8     56  56,43
1          3  26    8     57      0
2          3  26    8     57    0,6
3          3  26    8     57   0,39
1          3  26    8     58   0,96
2          3  26    8     58   1,64
3          3  26    8     58   0,94
1          3  26    8     59      0
1          3  26    9      0   0,31

, , , , . .:)