Phobos Performance Impact
As much as we'd like to make tracing and monitoring available for free, physics and computer organization are going to have something to say about that. Inevitably, Phobos.Tracing and Phobos.Monitoring introduce additional memory and CPU overhead in the course of their normal execution. The purpose of this document is to give Phobos users an idea of the impact of using Phobos inside their applications.
Reference Metrics and CPU
All of these metrics were recorded on a 4-core Intel i7-3630QM CPU clocked at 2.4GHz. The machine was running other workloads at the time, so be it.
For reference purposes, vanilla Akka.NET runs at about 4.0-4.5 million messages per second on this computer on both .NET Core 2.0 and .NET 4.6.1.
All of these metrics were gathered using an NBench 1.2 concurrent benchmark against a single actor running on possibly multiple cores.
What These Numbers Mean
Based on extensive profiling and experimentation carried out by the Petabridge team, the lion's share of the performance impact comes from the tracing system calling ToString on the ActorPath of the actors involved in the sent message and the message content itself. Thus, you shouldn't read too much into these numbers - the bulk of the cost of Phobos comes from string allocation and formatting.
In all likelihood, your actual business and application code will still cost significantly more than the overhead of Phobos.
.NET Core 2.0 Metrics
The following metrics were all recorded using .NET Core 2.0, NBench 1.2, and Phobos v0.5.1.
Fully Instrumented Actor Throughput
These are the data collected when an actor is running with the most verbose actor tracing and monitoring options available.
Totals
| Metric | Units | Max | Average | Min | StdDev |
|---|---|---|---|---|---|
| TotalCollections [Gen0] | collections | 496.00 | 494.69 | 494.00 | 0.63 |
| TotalCollections [Gen1] | collections | 10.00 | 8.85 | 8.00 | 0.69 |
| TotalCollections [Gen2] | collections | 4.00 | 3.31 | 3.00 | 0.48 |
| TotalBytesAllocated | bytes | 125,002,936.00 | 123,703,670.15 | 121,441,688.00 | 1,148,620.85 |
| [Counter] MessageReceived | operations | 3,000,000.00 | 3,000,000.00 | 3,000,000.00 | 0.00 |
Per-second Totals
| Metric | Units / s | Max / s | Average / s | Min / s | StdDev / s |
|---|---|---|---|---|---|
| TotalCollections [Gen0] | collections | 72.48 | 70.60 | 66.86 | 1.59 |
| TotalCollections [Gen1] | collections | 1.46 | 1.26 | 1.12 | 0.11 |
| TotalCollections [Gen2] | collections | 0.58 | 0.47 | 0.41 | 0.07 |
| TotalBytesAllocated | bytes | 18,194,695.01 | 17,652,601.38 | 16,919,040.17 | 387,097.00 |
| [Counter] MessageReceived | operations | 440,137.73 | 428,122.99 | 406,047.43 | 9,551.36 |
Uninstrumented Actor Throughput
These are the data collected when Phobos is installed and setup correctly, but phobos.tracing.enabled = off and phobos.monitoring.enabled = off for this actor.
Totals
| Metric | Units | Max | Average | Min | StdDev |
|---|---|---|---|---|---|
| TotalCollections [Gen0] | collections | 9.00 | 7.00 | 6.00 | 0.71 |
| TotalCollections [Gen1] | collections | 6.00 | 4.00 | 3.00 | 0.71 |
| TotalCollections [Gen2] | collections | 4.00 | 2.00 | 1.00 | 0.71 |
| TotalBytesAllocated | bytes | 59,113,624.00 | 48,688,704.62 | 22,419,888.00 | 11,945,053.11 |
| [Counter] MessageReceived | operations | 3,000,000.00 | 3,000,000.00 | 3,000,000.00 | 0.00 |
Per-second Totals
| Metric | Units / s | Max / s | Average / s | Min / s | StdDev / s |
|---|---|---|---|---|---|
| TotalCollections [Gen0] | collections | 11.71 | 9.14 | 7.93 | 1.09 |
| TotalCollections [Gen1] | collections | 7.81 | 5.22 | 3.97 | 0.95 |
| TotalCollections [Gen2] | collections | 5.20 | 2.60 | 1.32 | 0.92 |
| TotalBytesAllocated | bytes | 78,122,865.80 | 62,855,880.64 | 33,479,424.01 | 13,804,846.43 |
| [Counter] MessageReceived | operations | 4,479,873.94 | 3,923,652.22 | 3,399,262.41 | 344,806.55 |
Tracing-only Actor Throughput
These are the data collected when Phobos is installed and setup correctly, but phobos.tracing.enabled = on and phobos.monitoring.enabled = off for this actor.
Totals
| Metric | Units | Max | Average | Min | StdDev |
|---|---|---|---|---|---|
| TotalCollections [Gen0] | collections | 435.00 | 433.69 | 433.00 | 0.75 |
| TotalCollections [Gen1] | collections | 11.00 | 8.85 | 7.00 | 1.07 |
| TotalCollections [Gen2] | collections | 4.00 | 3.31 | 3.00 | 0.48 |
| TotalBytesAllocated | bytes | 125,015,312.00 | 123,136,178.46 | 121,856,904.00 | 997,482.50 |
| [Counter] MessageReceived | operations | 3,000,000.00 | 3,000,000.00 | 3,000,000.00 | 0.00 |
Per-second Totals
| Metric | Units / s | Max / s | Average / s | Min / s | StdDev / s |
|---|---|---|---|---|---|
| TotalCollections [Gen0] | collections | 75.23 | 73.80 | 69.06 | 1.65 |
| TotalCollections [Gen1] | collections | 1.88 | 1.51 | 1.20 | 0.19 |
| TotalCollections [Gen2] | collections | 0.69 | 0.56 | 0.48 | 0.08 |
| TotalBytesAllocated | bytes | 21,626,363.66 | 20,954,075.56 | 19,614,881.91 | 532,907.49 |
| [Counter] MessageReceived | operations | 520,038.76 | 510,492.28 | 478,468.98 | 11,476.97 |
Monitoring-only Actor Throughput
These are the data collected when Phobos is installed and setup correctly, but phobos.tracing.enabled = off and phobos.monitoring.enabled = on for this actor.
Totals
| Metric | Units | Max | Average | Min | StdDev |
|---|---|---|---|---|---|
| TotalCollections [Gen0] | collections | 60.00 | 58.46 | 57.00 | 0.88 |
| TotalCollections [Gen1] | collections | 11.00 | 9.00 | 8.00 | 1.00 |
| TotalCollections [Gen2] | collections | 5.00 | 3.85 | 3.00 | 0.55 |
| TotalBytesAllocated | bytes | 94,253,216.00 | 92,079,684.31 | 89,600,672.00 | 1,165,156.42 |
| [Counter] MessageReceived | operations | 3,000,000.00 | 3,000,000.00 | 3,000,000.00 | 0.00 |
Per-second Totals
| Metric | Units / s | Max / s | Average / s | Min / s | StdDev / s |
|---|---|---|---|---|---|
| TotalCollections [Gen0] | collections | 41.73 | 40.36 | 37.41 | 1.52 |
| TotalCollections [Gen1] | collections | 7.52 | 6.21 | 5.19 | 0.72 |
| TotalCollections [Gen2] | collections | 3.42 | 2.65 | 1.94 | 0.38 |
| TotalBytesAllocated | bytes | 67,824,238.05 | 63,586,052.55 | 58,727,903.37 | 2,634,062.16 |
| [Counter] MessageReceived | operations | 2,184,351.55 | 2,072,082.28 | 1,902,265.05 | 91,900.12 |
.NET 4.6.1 Metrics
The following metrics were all recorded using .NET 4.6.1, NBench 1.2, and Phobos v0.5.1.
Fully Instrumented Actor Throughput
These are the data collected when an actor is running with the most verbose actor tracing and monitoring options available.
Totals
| Metric | Units | Max | Average | Min | StdDev |
|---|---|---|---|---|---|
| TotalCollections [Gen0] | collections | 547.00 | 545.62 | 544.00 | 0.77 |
| TotalCollections [Gen1] | collections | 11.00 | 10.08 | 9.00 | 0.64 |
| TotalCollections [Gen2] | collections | 2.00 | 1.85 | 1.00 | 0.38 |
| TotalBytesAllocated | bytes | 85,887,792.00 | 84,171,396.31 | 82,236,824.00 | 1,078,249.51 |
| [Counter] MessageReceived | operations | 3,000,000.00 | 3,000,000.00 | 3,000,000.00 | 0.00 |
Per-second Totals
| Metric | Units / s | Max / s | Average / s | Min / s | StdDev / s |
|---|---|---|---|---|---|
| TotalCollections [Gen0] | collections | 94.91 | 89.64 | 77.61 | 5.52 |
| TotalCollections [Gen1] | collections | 1.89 | 1.66 | 1.28 | 0.16 |
| TotalCollections [Gen2] | collections | 0.35 | 0.30 | 0.14 | 0.07 |
| TotalBytesAllocated | bytes | 14,780,162.35 | 13,822,920.45 | 12,231,257.98 | 784,197.04 |
| [Counter] MessageReceived | operations | 522,442.57 | 492,855.65 | 427,229.21 | 30,429.65 |
Uninstrumented Actor Throughput
These are the data collected when Phobos is installed and setup correctly, but phobos.tracing.enabled = off and phobos.monitoring.enabled = off for this actor.
Totals
| Metric | Units | Max | Average | Min | StdDev |
|---|---|---|---|---|---|
| TotalCollections [Gen0] | collections | 17.00 | 16.31 | 16.00 | 0.48 |
| TotalCollections [Gen1] | collections | 8.00 | 6.69 | 6.00 | 0.63 |
| TotalCollections [Gen2] | collections | 2.00 | 2.00 | 2.00 | 0.00 |
| TotalBytesAllocated | bytes | 41,386,392.00 | 25,830,902.15 | 11,912,960.00 | 9,501,344.45 |
| [Counter] MessageReceived | operations | 3,000,000.00 | 3,000,000.00 | 3,000,000.00 | 0.00 |
Per-second Totals
| Metric | Units / s | Max / s | Average / s | Min / s | StdDev / s |
|---|---|---|---|---|---|
| TotalCollections [Gen0] | collections | 21.91 | 20.79 | 19.98 | 0.57 |
| TotalCollections [Gen1] | collections | 10.31 | 8.54 | 7.49 | 0.88 |
| TotalCollections [Gen2] | collections | 2.63 | 2.55 | 2.49 | 0.05 |
| TotalBytesAllocated | bytes | 53,331,027.58 | 32,870,447.44 | 15,484,806.17 | 11,998,983.44 |
| [Counter] MessageReceived | operations | 3,945,746.51 | 3,824,745.63 | 3,729,913.02 | 76,651.74 |
Tracing-only Actor Throughput
These are the data collected when Phobos is installed and setup correctly, but phobos.tracing.enabled = on and phobos.monitoring.enabled = off for this actor.
Totals
| Metric | Units | Max | Average | Min | StdDev |
|---|---|---|---|---|---|
| TotalCollections [Gen0] | collections | 486.00 | 484.15 | 483.00 | 1.14 |
| TotalCollections [Gen1] | collections | 11.00 | 10.23 | 9.00 | 0.60 |
| TotalCollections [Gen2] | collections | 3.00 | 2.00 | 1.00 | 0.41 |
| TotalBytesAllocated | bytes | 85,416,016.00 | 83,577,741.54 | 81,268,736.00 | 1,357,934.21 |
| [Counter] MessageReceived | operations | 3,000,000.00 | 3,000,000.00 | 3,000,000.00 | 0.00 |
Per-second Totals
| Metric | Units / s | Max / s | Average / s | Min / s | StdDev / s |
|---|---|---|---|---|---|
| TotalCollections [Gen0] | collections | 98.46 | 95.03 | 84.57 | 3.85 |
| TotalCollections [Gen1] | collections | 2.21 | 2.01 | 1.72 | 0.16 |
| TotalCollections [Gen2] | collections | 0.60 | 0.39 | 0.19 | 0.09 |
| TotalBytesAllocated | bytes | 17,225,908.74 | 16,405,332.45 | 14,686,748.65 | 744,188.35 |
| [Counter] MessageReceived | operations | 611,535.12 | 588,854.16 | 522,031.92 | 24,561.62 |
Monitoring-only Actor Throughput
These are the data collected when Phobos is installed and setup correctly, but phobos.tracing.enabled = off and phobos.monitoring.enabled = on for this actor.
Totals
| Metric | Units | Max | Average | Min | StdDev |
|---|---|---|---|---|---|
| TotalCollections [Gen0] | collections | 68.00 | 66.62 | 66.00 | 0.65 |
| TotalCollections [Gen1] | collections | 12.00 | 10.69 | 10.00 | 0.75 |
| TotalCollections [Gen2] | collections | 3.00 | 2.38 | 2.00 | 0.51 |
| TotalBytesAllocated | bytes | 70,227,992.00 | 63,645,809.23 | 54,246,664.00 | 5,258,296.00 |
| [Counter] MessageReceived | operations | 3,000,000.00 | 3,000,000.00 | 3,000,000.00 | 0.00 |
Per-second Totals
| Metric | Units / s | Max / s | Average / s | Min / s | StdDev / s |
|---|---|---|---|---|---|
| TotalCollections [Gen0] | collections | 49.53 | 48.47 | 46.16 | 0.99 |
| TotalCollections [Gen1] | collections | 8.96 | 7.78 | 6.89 | 0.60 |
| TotalCollections [Gen2] | collections | 2.24 | 1.73 | 1.38 | 0.37 |
| TotalBytesAllocated | bytes | 51,515,740.83 | 46,308,152.53 | 39,691,138.83 | 3,902,011.40 |
| [Counter] MessageReceived | operations | 2,240,923.64 | 2,183,123.18 | 2,066,936.78 | 49,391.50 |