Configuring Phobos

Phobos 1 and later uses a split configuration setup:

• For mandatory settings these are passed into the ActorSystem.Create method programmatically as part of a PhobosSetup class in C# or F#;
• For optional / detailed settings - these are overridable via HOCON configuration. Any setting for configuring individual tracing / monitoring settings that can be configured inside akka.actor.deployment for individual actors can also bet set on the fluent interface of the Akka.NET Props class.

Note

If you are ready to jump in and see an example of Phobos being configured inside a ASP.NET Core + Akka.NET application, please see the Phobos QuickStart Tutorial.

Let's take a look at how to configure Phobos completely using both of these tools.

PhobosSetup and PhobosConfigBuilder

PhobosSetup takes advantage of the new Setup and ActorSystemSetup types introduced into Akka.NET in v1.4. These types exist to allow a degree of programmatic configuration to be specified prior to creating an ActorSystem. This type of programmatic setup is ideal for Phobos, where we want to externally configure application observability endpoints (metrics, logging, and tracing) once and re-use that configuration across Akka.NET, ASP.NET, SignalR, RabbitMQ, and everywhere else.

In order to create a PhobosSetup we need to initialize both of the following:

1. An OpenTracing ITracer - you can see a list of some of the .NET tracing libraries that support OpenTracing here and
2. An App.Metrics IMetricsRoot, which will be used to pump metrics computed via Phobos out to one of our preferred monitoring solutions. You can see a full list of metrics reporting systems supported by App.Metrics here.

Here's a small example of what a complete example of a fully configured PhobosSetup looks like:

var hocon = ConfigurationFactory.ParseString(File.ReadAllText("Config/app.conf"));

OpenTracing.ITracer tracer = new MockTracer();
App.Metrics.IMetricsRoot metrics = new MetricsBuilder().Configuration.Configure(o =>
    {
        o.GlobalTags.Add("host", Dns.GetHostName());
        o.DefaultContextLabel = "akka.net";
        o.Enabled = true;
        o.ReportingEnabled = true;
    }).Report.ToConsole().Build();

var phobosConfigBuilder = new PhobosConfigBuilder()
    .WithMetrics(m => m.SetMetricsRoot(metrics))
    .WithTracing(t => t.SetTracer(tracer));

// create an ActorSystemSetup class used to configure the ActorSystem
var phobosSetup = PhobosSetup.Create(phobosConfigBuilder)
    .And(BootstrapSetup.Create()
        .WithConfig(hocon)
        .WithActorRefProvider(PhobosProviderSelection.Cluster));

// start ActorSystem
var actorSystem = ActorSystem.Create("myCluster", phobosSetup);

Breaking this down into smaller parts, the first step is to configure an ITracer and an IMetricsRoot

ITracer tracer = new MockTracer();
IMetricsRoot metrics = new MetricsBuilder().Configuration.Configure(opt =>
{
    opt.DefaultContextLabel = "Test";
}).Report.Using(new ConsoleMetricsReporter()).Build();;

Note

Typically the ITracer and IMetricsRoot get bound as required services when setting up dependency injection inside Startup.cs when you're using .NET Core's Generic Host or ASP.NET Core. For a full example of this, please see how PhobosSetup gets constructed in concert with Microsoft.Extensions.DependencyIjnection in the Phobos QuickStart Tutorial.

The ITracer and the IMetricsRoot then need to be passed into an PhobosConfigBuilder, which binds the tracer and metrics to Phobos:

var phobosConfigBuilder = new PhobosConfigBuilder()
    .WithMetrics(m => m.SetMetricsRoot(metrics))
    .WithTracing(t => t.SetTracer(tracer));

From there we need to pass the PhobosConfigBuilder object into the PhobosSetup, which we will merge with an Akka.NET BootstrapSetup which accepts the HOCON configuration we would normally pass into an ActorSystem.Create method:

var bootstrap = BootstrapSetup.Create()
    .WithActorRefProvider(PhobosProviderSelection.Cluster) // configures Phobos for Akka.Cluster
    .WithConfig(HoconConfig);

// need this to launch our ActorSystem
ActorSystemSetup phobosSetup = PhobosSetup.Create(phobosConfigBuilder).And(bootstrap);
ActorSystem actorSystem = ActorSystem.Create("MySys", phobosSetup);

Warning

If the BoostrapSetup is ommitted then Phobos will not run.

This is all we need to launch an Akka.NET ActorSystem that is configured using Phobos.

Enabling Phobos for Akka, Akka.Remote, and Akka.Cluster

Akka

To enable Phobos for stand-alone Akka.NET, i.e. in an environment where you're not using Akka.Remote or Akka.Cluster, then we need to change the call we make to the BootstrapSetup.WithActorRefProvider method:

 var bootstrap = BootstrapSetup.Create()
    .WithActorRefProvider(PhobosProviderSelection.Local) // configures Phobos for Akka only
    .WithConfig(HoconConfig);

The PhobosProviderSelection class includes toggles for Akka, Akka.Remote, and Akka.Cluster. We need to make sure we include the correct one for each environment.

Akka.Remote

To enable Phobos for Akka.Remote only, meaning without Akka.Cluster, we need to use the following code:

 var bootstrap = BootstrapSetup.Create()
    .WithActorRefProvider(PhobosProviderSelection.Remote) // configures Phobos for Akka.Remote
    .WithConfig(HoconConfig);

Akka.Cluster

To enable Phobos for Akka.Cluster only, meaning without Akka.Cluster, we need to use the following code:

 var bootstrap = BootstrapSetup.Create()
    .WithActorRefProvider(PhobosProviderSelection.Cluster) // configures Phobos for Akka.Cluster
    .WithConfig(HoconConfig);

Filtering Traced Messages with PhobosConfigBuilder

One of the other things you can do with the PhobosConfigBuilder is customize which messages can start new traces and which ones cannot. This can be really helpful in reducing the amount of tracing noise produced and memory traffic / bandwidth consumed.

All you need to do to filter trace messages is to pass in a handful of predicate functions to include and exclude messages.

Tip

By default all messages automatically initiate traces in Phobos.

var phobosConfigBuilder = new PhobosConfigBuilder()
    .WithMetrics(m => m.SetMetricsRoot(metrics))
    .WithTracing(t => t.SetTracer(tracer)
        .AddIncludeMessageFilter<string>()
        .AddExcludeMessageFilter<int>()
        .AddMessageFilter(f => f is IActorRef actorRef && !actorRef.IsNobody()));

This configuration will enable Phobos to include string messages, exclude int messages, and exclude any messages that implement the IActorRef inteface where IActorRef.IsNobody() is true.

var actor = actorSystem.ActorOf(act => act.ReceiveAny((obj, ctx) =>
{
    ctx.Sender.Tell(obj);
}), "echo");

// should be traced
var response1 = await actor.Ask<string>("foo", TimeSpan.FromMilliseconds(500));

// should NOT be traced
var response2 = await actor.Ask<int>(1, TimeSpan.FromMilliseconds(500));

// check both results
var finishedSpans = tracer.As<MockTracer>().FinishedSpans();
finishedSpans.Should().Contain(span => span.OperationName.Contains("String"));
finishedSpans.Should().NotContain(span => span.OperationName.Contains("int"));

Phobos HOCON Configuration

Some settings cannot be easily expressed via a stand-alone builder interface and there are also legacy settings from previous versions of Phobos that are still incorporated via HOCON.

Here's the full set of stand-alone HOCON settings that are specific to Akka.NET.

Tip

The phobos HOCON section is a stand-alone section; meaning that it does not get included inside the akka HOCON section that Akka.NET developers typically modify.

##################################
# Phobos Reference Config File   #
##################################

phobos{
	tracing{
		# Setting used to trigger tracing for all built-in /system actors by default.
		# When set to "on," all /system actors will automatically be outfitted with tracing
		# instrumentation.
		trace-all-system-actors = off

		# Setting used to trigger tracing for all /user actors by default.
		# When set to "on," all /user actors will automatically be outfitted with tracing
		# instrumentation.
		trace-all-user-actors = on

		# Debugging settings used for testing the configuration of Phobos.Tracing
		debug{
			# When enabled: will log incoming tracing information
			# on all actors capable of receiving trace events.
			log-traces = off
		}

		# Setting used to instruct Phobos whether or not it should automatically append
		# all captured log messages to the active ISpan during which they were produced.
		#
		# Turning this setting off may help improve performance slightly in some instances,
		# but at the cost of missing useful debugging and diagnostic data.
		# 
		# If you want to throttle down the amount of log messages being appended to outgoing
		# spans, consider simply changing the `akka.loglevel` setting or others inside core Akka.NET.
		append-logs-to-trace = on
	}

	monitoring{
		# Setting used to trigger monitoring for all built-in /system actors by default.
		# When set to "on," all /system actors will automatically be outfitted with monitoring
		# instrumentation.
		monitor-all-system-actors = off

		# Setting used to trigger monitoring for all /user actors by default.
		# When set to "on," all /user actors will automatically be outfitted with monitoring
		# instrumentation.
		monitor-all-user-actors = on

		# Toggles monitoring of mailbox queue depth for any actors who have monitoring enabled
		monitor-mailbox-depth = off

		# When enabled, records all of the default events that pass through the eventstream
		# (log events, dead letters, and unhandled messages) and writes that out to a Global
		# counter for the ActorSystem as a whole.
		monitor-eventstream = on
	}
}