Skip to content

T
tsn-measures
Commit 4f65279c authored by Joanne Hugé's avatar Joanne Hugé
Browse files
Options

Replace variance by standard deviation

parent 5b303900
Hide whitespace changes
Inline Side-by-side
Showing with 33 additions and 31 deletions
measure-analysis/measure-analysis.py
View file @ 4f65279c
...@@ -11,6 +11,8 @@ import numpy as np ...@@ -11,6 +11,8 @@ import numpy as np
import matplotlib.mlab as mlab import matplotlib.mlab as mlab
import matplotlib.pyplot as plt import matplotlib.pyplot as plt
from math import sqrt
class MeasureSetHandler: class MeasureSetHandler:
report_description = ("This report was generated with the measure-analysis.py script.\nJSON formated measures " report_description = ("This report was generated with the measure-analysis.py script.\nJSON formated measures "
...@@ -446,7 +448,7 @@ class MeasureSet: ...@@ -446,7 +448,7 @@ class MeasureSet:
self.max = [max(prop) for prop in props] self.max = [max(prop) for prop in props]
self.min = [min(prop) for prop in props] self.min = [min(prop) for prop in props]
self.avg = [statistics.mean(prop) for prop in props] self.avg = [statistics.mean(prop) for prop in props]
self.var = [statistics.variance(prop) for prop in props] self.dev = [sqrt(statistics.variance(prop)) for prop in props]
def add_histogram(self, props_names, props): def add_histogram(self, props_names, props):
...@@ -457,7 +459,7 @@ class MeasureSet: ...@@ -457,7 +459,7 @@ class MeasureSet:
self.max = [] self.max = []
self.min = [] self.min = []
self.avg = [] self.avg = []
self.var = [] self.dev = []
for prop in props: for prop in props:
...@@ -477,7 +479,7 @@ class MeasureSet: ...@@ -477,7 +479,7 @@ class MeasureSet:
var -= avg_centered**2 var -= avg_centered**2
self.avg.append(avg) self.avg.append(avg)
self.var.append(var) self.dev.append(sqrt(var))
def export_to_json(self, path): def export_to_json(self, path):
...@@ -576,7 +578,7 @@ class MeasureSet: ...@@ -576,7 +578,7 @@ class MeasureSet:
if headers: if headers:
# Table headers # Table headers
headers = ["Min", "Max", "Avg", "Var"] headers = ["Minimum", "Maximum", "Average", "Standard deviation"]
# Add a metadata information column if the metadata_mask isn't empty # Add a metadata information column if the metadata_mask isn't empty
if metadata_mask != []: if metadata_mask != []:
...@@ -587,7 +589,7 @@ class MeasureSet: ...@@ -587,7 +589,7 @@ class MeasureSet:
table += [["**" + " - ".join(self.props_names) + "**"] * len(headers)] table += [["**" + " - ".join(self.props_names) + "**"] * len(headers)]
if values: if values:
m = [self.min, self.max, self.avg, self.var] m = [self.min, self.max, self.avg, self.dev]
values = [[ (format(m[i][j], '.4f') + self.units[j]) for j in range(len(m[0]))] for i in range(len(m))] values = [[ (format(m[i][j], '.4f') + self.units[j]) for j in range(len(m[0]))] for i in range(len(m))]
......
measure-analysis/measure-report.md
View file @ 4f65279c
...@@ -26,13 +26,13 @@ Uses cyclictest from the rt-tests test suite to measure wake-up latency. A real- ...@@ -26,13 +26,13 @@ Uses cyclictest from the rt-tests test suite to measure wake-up latency. A real-
**Common test metadata:** Linux kernel version: 4.19, Boot Parameters: isolcpus, Interval: 200us, ETF qdisc delta: 200us, Task priority: 99, qdisc: pfifo_fast, etf_offset: none, route: switch **Common test metadata:** Linux kernel version: 4.19, Boot Parameters: isolcpus, Interval: 200us, ETF qdisc delta: 200us, Task priority: 99, qdisc: pfifo_fast, etf_offset: none, route: switch
Metadata | Min | Max | Avg | Var Metadata | Minimum | Maximum | Average | Standard deviance
------------------------- | ------------------- | ------------------- | ------------------- | ------------------- ------------------------- | ------------------- | ------------------- | ------------------- | -------------------
**board, duration, load** | **wake-up latency** | **wake-up latency** | **wake-up latency** | **wake-up latency** **board, duration, load** | **wake-up latency** | **wake-up latency** | **wake-up latency** | **wake-up latency**
Slate, 21h22, hackbench | 14.0000us | 65.0000us | 21.2779us | 4.7291us Slate, 21h22, hackbench | 14.0000us | 65.0000us | 21.2779us | 2.1746us
Emerald, 21h22, hackbench | 14.0000us | 112.0000us | 19.5352us | 4.0545us Emerald, 21h22, hackbench | 14.0000us | 112.0000us | 19.5352us | 2.0136us
Onyx, 46h, hackbench | 14.0000us | 72.0000us | 20.5580us | 3.9283us Onyx, 46h, hackbench | 14.0000us | 72.0000us | 20.5580us | 1.9820us
Emerald, 24h, ssh | 12.0000us | 80.0000us | 15.4101us | 1.2302us Emerald, 24h, ssh | 12.0000us | 80.0000us | 15.4101us | 1.1091us
![alt text](measures/graphs/cyclictest_wake-up_latencywake-up latency.png "cyclictest_wake-up_latency Graph") ![alt text](measures/graphs/cyclictest_wake-up_latencywake-up latency.png "cyclictest_wake-up_latency Graph")
...@@ -42,13 +42,13 @@ Similar to packet_recv_timestamps, but on the transmitting board. ...@@ -42,13 +42,13 @@ Similar to packet_recv_timestamps, but on the transmitting board.
**Common test metadata:** Linux kernel version: 4.19, Boot Parameters: isolcpus, Task priority: 99, Device and processor load: ssh, etf_offset: none, route: switch **Common test metadata:** Linux kernel version: 4.19, Boot Parameters: isolcpus, Task priority: 99, Device and processor load: ssh, etf_offset: none, route: switch
Metadata | Min | Max | Avg | Var Metadata | Minimum | Maximum | Average | Standard deviance
--------------------------------------- | ----------------------------- | ----------------------------- | ----------------------------- | ----------------------------- --------------------------------------- | ----------------------------- | ----------------------------- | ----------------------------- | -----------------------------
**board, i, duration, delta, qdisc** | **user_space - kernel_space** | **user_space - kernel_space** | **user_space - kernel_space** | **user_space - kernel_space** **board, i, duration, delta, qdisc** | **user_space - kernel_space** | **user_space - kernel_space** | **user_space - kernel_space** | **user_space - kernel_space**
Onyx, 100000us, 0h17, 200us, pfifo_fast | 2.0000us - 42.0000us | 12.0000us - 145.0000us | 2.3099us - 51.8681us | 0.2716us - 56.2809us Onyx, 100000us, 0h17, 200us, pfifo_fast | 2.0000us - 42.0000us | 12.0000us - 145.0000us | 2.3099us - 51.8681us | 0.5212us - 7.5021us
Slate, 1000us, 1h00, 200us, pfifo_fast | 1.0000us - 32.0000us | 12.0000us - 222.0000us | 2.0360us - 36.6218us | 0.0493us - 9.1563us Slate, 1000us, 1h00, 200us, pfifo_fast | 1.0000us - 32.0000us | 12.0000us - 222.0000us | 2.0360us - 36.6218us | 0.2221us - 3.0259us
Slate, 1000us, 0h2, 200us, pfifo_fast | 1.0000us - 32.0000us | 12.0000us - 118.0000us | 1.9341us - 36.6011us | 0.0768us - 3.7223us Slate, 1000us, 0h2, 200us, pfifo_fast | 1.0000us - 32.0000us | 12.0000us - 118.0000us | 1.9341us - 36.6011us | 0.2771us - 1.9293us
Onyx, 1000us, 72h24, 160us, etf | 1.0000us - 33.0000us | 65.0000us - 110.0000us | 1.9916us - 37.1342us | 3.1052us - 13.6433us Onyx, 1000us, 72h24, 160us, etf | 1.0000us - 33.0000us | 65.0000us - 110.0000us | 1.9916us - 37.1342us | 1.7622us - 3.6937us
![alt text](measures/graphs/packet_send_timestampsuser_space.png "packet_send_timestamps Graph") ![alt text](measures/graphs/packet_send_timestampsuser_space.png "packet_send_timestamps Graph")
...@@ -60,12 +60,12 @@ An UDP packet is periodically sent from one board to another using a real time t ...@@ -60,12 +60,12 @@ An UDP packet is periodically sent from one board to another using a real time t
**Common test metadata:** Linux kernel version: 4.19, Boot Parameters: isolcpus, Task priority: 99, Device and processor load: ssh, etf_offset: none, route: switch **Common test metadata:** Linux kernel version: 4.19, Boot Parameters: isolcpus, Task priority: 99, Device and processor load: ssh, etf_offset: none, route: switch
Metadata | Min | Max | Avg | Var Metadata | Minimum | Maximum | Average | Standard deviance
---------------------------------------- | ----------------------------- | ----------------------------- | ----------------------------- | ----------------------------- ---------------------------------------- | ----------------------------- | ----------------------------- | ----------------------------- | -----------------------------
**i, duration, board, delta, qdisc** | **user_space - kernel_space** | **user_space - kernel_space** | **user_space - kernel_space** | **user_space - kernel_space** **i, duration, board, delta, qdisc** | **user_space - kernel_space** | **user_space - kernel_space** | **user_space - kernel_space** | **user_space - kernel_space**
100000us, 0h17, Slate, 200us, pfifo_fast | 0.0000us - 44.0000us | 26.0000us - 142.0000us | 0.1151us - 55.4212us | 0.1646us - 58.2870us 100000us, 0h17, Slate, 200us, pfifo_fast | 0.0000us - 44.0000us | 26.0000us - 142.0000us | 0.1151us - 55.4212us | 0.4057us - 7.6346us
1000us, 16h48, Slate, 200us, pfifo_fast | 0.0000us - 35.0000us | 41.0000us - 233.0000us | 0.0036us - 41.5560us | 0.0521us - 27.0303us 1000us, 16h48, Slate, 200us, pfifo_fast | 0.0000us - 35.0000us | 41.0000us - 233.0000us | 0.0036us - 41.5560us | 0.2282us - 5.1991us
1000us, 72h24, Emerald, 160us, etf | 0.0000us - 41.0000us | 42.0000us - 291.0000us | 0.0235us - 50.7618us | 0.1085us - 55.3431us 1000us, 72h24, Emerald, 160us, etf | 0.0000us - 41.0000us | 42.0000us - 291.0000us | 0.0235us - 50.7618us | 0.3294us - 7.4393us
![alt text](measures/graphs/packet_recv_timestampsuser_space.png "packet_recv_timestamps Graph") ![alt text](measures/graphs/packet_recv_timestampsuser_space.png "packet_recv_timestamps Graph")
...@@ -77,14 +77,14 @@ An UDP packet is periodically sent from one board to another using a real time t ...@@ -77,14 +77,14 @@ An UDP packet is periodically sent from one board to another using a real time t
**Common test metadata:** Linux kernel version: 4.19, Boot Parameters: isolcpus, Task priority: 99, Device and processor load: ssh **Common test metadata:** Linux kernel version: 4.19, Boot Parameters: isolcpus, Task priority: 99, Device and processor load: ssh
Metadata | Min | Max | Avg | Var Metadata | Minimum | Maximum | Average | Standard deviance
------------------------------------------------------- | ----------- | ---------- | ---------- | ---------- ------------------------------------------------------- | ----------- | ---------- | ---------- | -----------------
**i, duration, board, delta, qdisc, etf_offset, route** | **jitter** | **jitter** | **jitter** | **jitter** **i, duration, board, delta, qdisc, etf_offset, route** | **jitter** | **jitter** | **jitter** | **jitter**
100000us, 0h17, Slate, none, pfifo_fast, none, switch | -139.0000us | 147.0000us | 10.4314us | 252.7494us 100000us, 0h17, Slate, none, pfifo_fast, none, switch | -139.0000us | 147.0000us | 10.4314us | 15.8981us
1000us, 16h48, Slate, none, pfifo_fast, none, switch | -434.0000us | 237.0000us | 6.9930us | 81.8093us 1000us, 16h48, Slate, none, pfifo_fast, none, switch | -434.0000us | 237.0000us | 6.9930us | 9.0448us
1000us, 72h24, Emerald, 160us, etf, 1000000, switch | -397.0000us | 369.0000us | 7.4514us | 155.0147us 1000us, 72h24, Emerald, 160us, etf, 1000000, switch | -397.0000us | 369.0000us | 7.4514us | 12.4505us
1000us, 0h15, Onyx, 150us, etf, 400, switch | -535.0000us | 125.0000us | 10.5255us | 166.0334us 1000us, 0h15, Onyx, 150us, etf, 400, switch | -535.0000us | 125.0000us | 10.5255us | 12.8854us
1000us, 0h12, Emerald, 150us, etf, 400, E2E | -692.0000us | 129.0000us | 9.7386us | 155.2697us 1000us, 0h12, Emerald, 150us, etf, 400, E2E | -692.0000us | 129.0000us | 9.7386us | 12.4607us
![alt text](measures/graphs/packet_jitterjitter.png "packet_jitter Graph") ![alt text](measures/graphs/packet_jitterjitter.png "packet_jitter Graph")
...@@ -94,10 +94,10 @@ An UDP packet is periodically sent from one board to another and sent back as so ...@@ -94,10 +94,10 @@ An UDP packet is periodically sent from one board to another and sent back as so
**Common test metadata:** Board name: Onyx, Linux kernel version: 4.19, Boot Parameters: isolcpus, Interval: 10000us, ETF qdisc delta: 200us, Task priority: 99, Device and processor load: ssh, Test duration: 0h4, qdisc: pfifo_fast, etf_offset: none, route: switch **Common test metadata:** Board name: Onyx, Linux kernel version: 4.19, Boot Parameters: isolcpus, Interval: 10000us, ETF qdisc delta: 200us, Task priority: 99, Device and processor load: ssh, Test duration: 0h4, qdisc: pfifo_fast, etf_offset: none, route: switch
Min | Max | Avg | Var Minimum | Maximum | Average | Standard deviance
---------- | ---------- | ---------- | ---------- ---------- | ---------- | ---------- | -----------------
**rtt** | **rtt** | **rtt** | **rtt** **rtt** | **rtt** | **rtt** | **rtt**
636.0000us | 972.0000us | 674.0051us | 241.7037us 636.0000us | 972.0000us | 674.0051us | 15.5468us
![alt text](measures/graphs/packet_rttrtt.png "packet_rtt Graph") ![alt text](measures/graphs/packet_rttrtt.png "packet_rtt Graph")
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7