Traces and Metrics for Basic Actor Request-Response Scenario

On this page we will take a look at the most simple scenario: two local actors are sending messages to each other.

Code

Here is a code sample we will use:

var pong = Sys.ActorOf<PongActor>("pong");
var ping = Sys.ActorOf(Props.Create(() => new PingActor(pong, TestActor)), "ping");

for (var i = 0; i < 2; ++i)
    ping.Tell(Request.Instance);
for (var i = 0; i < 2; ++i)
    ExpectMsg<Response>();

private class PingActor : ReceiveActor
{
    public PingActor(IActorRef pongActor, IActorRef testActor)
    {
        Receive<Request>(m => pongActor.Tell(Request.Instance));
        Receive<Response>(m => testActor.Tell(Response.Instance));
    }
}

private class PongActor : ReceiveActor
{
    public PongActor()
    {
        var counter = Context.GetInstrumentation().Monitor.CreateCounter<long>("requests.count");

        Receive<Request>(m =>
        {
            // increment custom counter
            counter.Add(1, new KeyValuePair<string, object>("myTag", "myValue"));

            Context.Sender.Tell(Response.Instance);
        });
    }
}

public class Request
{
    public static readonly Request Instance = new Request();
}

public class Response
{
    public static readonly Response Instance = new Response();
}

What will Phobos collect for us in this case?

Metrics

Phobos will automatically collect some metrics available in this sample:

• akka.actor.created counter - how many actors of given type were created at that moment?
• akka.messages.recv meters - how many messages of given type arrived into actor's mailboxes?

OpenTelemetry exposes its own rich metrics functionality and you can collect your own custom metrics quite easily by creating a new Meter and subscribing to it through the OpenTelemetry.Extensions.Hosting NuGet package's extension methods.

Here we are just incrementing custom requests.count counter.

Context: Akka.NET

Counters

Name	Tags	Value
akka.messages.recv	actortype: Phobos.Actor.End2End.Tests.Standalone.RequestResponseSpec+PongActor messagetype: Phobos.Actor.End2End.Tests.Standalone.RequestResponseSpec+Request	4
akka.messages.recv	actortype: Phobos.Actor.End2End.Tests.Standalone.RequestResponseSpec+PingActor messagetype: Phobos.Actor.End2End.Tests.Standalone.RequestResponseSpec+Request	4
requests.count	myTag: myValue	4
akka.messages.recv	actortype: Phobos.Actor.End2End.Tests.Standalone.RequestResponseSpec+PingActor messagetype: Phobos.Actor.End2End.Tests.Standalone.RequestResponseSpec+Response	4
akka.actor.created	actortype: Phobos.Actor.End2End.Tests.Standalone.RequestResponseSpec+PongActor	2
akka.actor.created	actortype: Phobos.Actor.End2End.Tests.Standalone.RequestResponseSpec+PingActor	2

Histograms

Name	Tags	Value
akka.messages.latency	actortype: Phobos.Actor.End2End.Tests.Standalone.RequestResponseSpec+PongActor messagetype: Phobos.Actor.End2End.Tests.Standalone.RequestResponseSpec+Request	4
akka.messages.latency	actortype: Phobos.Actor.End2End.Tests.Standalone.RequestResponseSpec+PingActor messagetype: Phobos.Actor.End2End.Tests.Standalone.RequestResponseSpec+Request	4
akka.messages.latency	actortype: Phobos.Actor.End2End.Tests.Standalone.RequestResponseSpec+PingActor messagetype: Phobos.Actor.End2End.Tests.Standalone.RequestResponseSpec+Response	4

Traces

Phobos will also keep track of actor's communication, using OpeTelemetry's tracing' to represent message flow.

Here is how one of two collected traces will look like:

sequenceDiagram /system/testActor->>/user/ping: akka.msg.recv Phobos.Actor.End2End.Tests.Standalone.RequestResponseSpec+Request Note over /system/testActor,/user/ping: SpanIndex: 0
ParentIndex: none
Tags:
  akka.actor.recv.msgType:
    Phobos.Actor.End2End.Tests.Standalone.RequestResponseSpec+Request
Logs:
  message
  content:
    Phobos.Actor.End2End.Tests.Standalone.RequestResponseSpec+Request
/user/ping->>/user/pong: akka.msg.recv Phobos.Actor.End2End.Tests.Standalone.RequestResponseSpec+Request Note over /user/ping,/user/pong: SpanIndex: 1
ParentIndex: 0
Tags:
  akka.actor.recv.msgType:
    Phobos.Actor.End2End.Tests.Standalone.RequestResponseSpec+Request
Logs:
  waiting
  
  message
  content:
    Phobos.Actor.End2End.Tests.Standalone.RequestResponseSpec+Request
/user/pong->>/user/ping: akka.msg.recv Phobos.Actor.End2End.Tests.Standalone.RequestResponseSpec+Response Note over /user/pong,/user/ping: SpanIndex: 2
ParentIndex: 1
Tags:
  akka.actor.recv.msgType:
    Phobos.Actor.End2End.Tests.Standalone.RequestResponseSpec+Response
Logs:
  waiting
  
  message
  content:
    Phobos.Actor.End2End.Tests.Standalone.RequestResponseSpec+Response

Here we can see that:

1. Test actor is sending a message of type Request to our Ping actor
2. Ping actor is sending Request to Pong. Note that this two spans are connected (child_of relation), which puts them into single trace
3. Pong actor is replying back with Response message

We have all this exported into your favourite tracing / metrics collection system out of the box!