Automatically Captured Data
Phobos attempts to capture as much data as is relevant automatically, without the need for explicit instrumentation code inside your Akka.NET applications.
The following article articulates all of the data that is captured automatically by individual actors as well as each
ActorSystem that is instrumented with Phobos.
Captured Tracing Data
Tracing data is only captured when an actor is processing a message that either:
- Causes a new trace to be started; i.e it's the first operation that either hasn't been excluded via trace filtering or sampling. Or
- Is part of an going trace that was started by another actor.
Once a trace is active Phobos will automatically attempt to capture the following data each time an actor processes a message:
ActorPathof the actor processing the message will be recorded as the operation name of each individual
.ToString()content of the received message, which will logged as an OpenTracing
- The fully qualified type name of the received message;
- The absolute
ActorPathof the actor who sent this message.
- The fully qualified type name of the actor processing the message; and
ILoggingAdaptercalls made from within your actor will be recorded as
Here's an example of what a single captured trace might look like inside Zipkin using the
Phobos.Tracing.Zipkin driver package:
The results for how this information will be displayed to you will vary a lot by the capabilities of each tracing back-end, but generally speaking this is how the data is captured via Phobos.
N.B. It's worth noting that all traces in Phobos are automatically correlated between actors and between actors running on different nodes via Akka.Remote. This is handled automatically behind the scenes provided that Phobos is installed on all nodes.
Other Trace Events
In addition to the data above, Phobos will also capture the other following types of activities automatically so long as there is an active trace available:
- Child actors being spawned;
- Actors crashing as a result of unhandled exceptions; in this instance we also automatically record relevant data about the crash such as the
Exceptionand its stack trace; and
- Any unhandled messages or deadletters that occurred during an active trace.
Captured Monitoring Data
Monitoring data is captured when actors are processing messages, but data is also captured during other system events that occur independently of actors too.
Phobos will automatically monitor these events per-actor:
- Number of messages received per individual actor (i.e. how many messages has
- Number of messages received per actor implementation class (i.e. how many messages have actors of type
- Number of unprocessed messages sitting in the mailbox per each individual actor (has to be explicitly enabled via
Phobos also captures the following metrics per-
- Number of log events (
- Number of unprocessed messages, i.e.
- Number of actor starts, stops, and restarts; and
- Total number of messages across all actors received (total
Suppose you have a large Akka.NET Cluster - ideally you'd want to be able to monitor all of these values on a per-node basis and a cluster-wide basis. "I want to see how busy node 1 out of 13 is, but I also want to see how busy all 13 nodes are in aggregate."
Phobos.Monitoring supports this as well by reporting all metrics on both a per-node and aggregate basis across the entire cluster.
If you want to capture your own custom metrics in addition to what we capture above, you can read about how to instrument your own Akka.NET monitoring metrics here.
Phobos automatically denormalizes all of these captured metrics into multiple dimensions, keeping track of metrics at both the cluster level and the node-specific level. You can read more about the dimensions actor metrics are recorded in here.