Instrumenting Your Own Metrics with Phobos.Monitoring
One of the biggest features of Phobos is its ability to automatically capture numerous interesting Akka.NET metrics and Phobos does this in order to help minimize the instrumentation cost and complexity on behalf of its users.
However, there are many scenarios where you may want to record your own custom metrics. For instance:
- You want to measure how long it takes an actor to process a specific operation;
- You have domain-specific values you want to track across your cluster; or
- You have operations that occur outside of your actors that you still wish to monitor.
These are all fairly common scenarios, and thus we expose a means of accessing the built-in monitoring and tracing instrumentation directly in your code for this exact purpose.
Here's a small example of how you might record a custom Gauge, for instance:
{
_log.Error("No mailbox monitoring found.");
_log.Error("Config: {0}", Context.GetInstrumentation().InstrumentationSettings);
throw new ApplicationException("FATAL - MAILBOX MONITORING MUST BE ON");
}
_router = Context.ActorOf(Props.Empty.WithRouter(FromConfig.Instance), "router");
}
protected override void PostStop()
{
_cluster.Unsubscribe(Self);
}
}
}
It's the Context.GetInstrumentation() call which exposes Phobos' built-in monitoring, tracing, and any other future planned instrumentation. For the purposes of monitoring though, you'll be most interested in the Context.GetInstrumentation().Monitor.
Using the Monitor
The fundamental unit of work used in Phobos.Monitoring is the IMonitor; this is the tool we use to record metrics which will be propagated to our backend.
The IMonitor is an abstraction built on top of whatever your underlying metrics backend is and all of its methods are intended to behave uniformly across all first party backends.
// -----------------------------------------------------------------------
// <copyright file="IMonitor.cs" company="Petabridge, LLC">
// Copyright (C) 2015 - 2018 Petabridge, LLC <https://petabridge.com>
// </copyright>
// -----------------------------------------------------------------------
using System;
using System.Collections.Generic;
using Phobos.Monitoring.Metrics;
namespace Phobos.Monitoring
{
/// <summary>
/// The Phobos monitor - used to create metrics which themselves can be incremented and transmitted
/// to a reporting source on the other end of the network.
/// </summary>
public interface IMonitor
{
/// <summary>
/// The beginning of the namespace we use to track metrics for this monitor.
/// I.e. if AppName is "petabridge" then all metrics will be named "petabridge.thing1.metric1"
/// </summary>
string AppName { get; }
/// <summary>
/// The underlying <see cref="IMetricRecorder" />, usually only directly accessed for testing purposes.
/// </summary>
IMetricRecorder Recorder { get; }
/// <summary>
/// Record to a <see cref="Gauge" /> directly without having to allocate one.
/// </summary>
/// <param name="name">The specified name of the gauge we're going to measure.</param>
/// <param name="value">The value to be recorded for this gauge.</param>
/// <param name="sampleRate">The sample rate. Defaults to <see cref="Metric.DefaultSampleRate" />.</param>
void RecordGauge(string name, long value, double sampleRate = Metric.DefaultSampleRate);
/// <summary>
/// Creates a new <see cref="Gauge" /> instance.
/// </summary>
/// <param name="name">The specified name of the gauge we're going to measure.</param>
/// <param name="sampleRate">The sample rate. Defaults to <see cref="Metric.DefaultSampleRate" />.</param>
/// <returns>The new gauge instance.</returns>
Gauge CreateGauge(string name, double sampleRate = Metric.DefaultSampleRate);
/// <summary>
/// Creates a new <see cref="Gauge" /> instance.
/// </summary>
/// <param name="name">The specified name of the gauge we're going to measure.</param>
/// <param name="allNames">The set of all names that we can use for this metric.</param>
/// <param name="sampleRate">The sample rate. Defaults to <see cref="Metric.DefaultSampleRate" />.</param>
/// <returns>The new gauge instance.</returns>
Gauge CreateGauge(string name, IReadOnlyList<string> allNames, double sampleRate = Metric.DefaultSampleRate);
/// <summary>
/// Increment a <see cref="Counter" /> directly without having to allocate one.
/// </summary>
/// <param name="name">The specified name of the counter we're going to measure.</param>
/// <param name="value">The value to be recorded for this counter. Defaults to 1.</param>
/// <param name="sampleRate">The sample rate. Defaults to <see cref="Metric.DefaultSampleRate" />.</param>
void IncrementCounter(string name, long value = 1L, double sampleRate = Metric.DefaultSampleRate);
/// <summary>
/// Decrement a <see cref="Counter" /> directly without having to allocate one.
/// </summary>
/// <param name="name">The specified name of the counter we're going to measure.</param>
/// <param name="value">The value to be recorded for this counter. Defaults to -1.</param>
/// <param name="sampleRate">The sample rate. Defaults to <see cref="Metric.DefaultSampleRate" />.</param>
void DecrementCounter(string name, long value = -1L, double sampleRate = Metric.DefaultSampleRate);
/// <summary>
/// Creates a new <see cref="Counter" /> instance.
/// </summary>
/// <param name="name">The specified name of the counter we're going to measure.</param>
/// <param name="sampleRate">The sample rate. Defaults to <see cref="Metric.DefaultSampleRate" />.</param>
/// <returns>The new counter instance.</returns>
Counter CreateCounter(string name, double sampleRate = Metric.DefaultSampleRate);
/// <summary>
/// Creates a new <see cref="Counter" /> instance.
/// </summary>
/// <param name="name">The specified name of the counter we're going to measure.</param>
/// <param name="allNames">The set of all names that we can use for this metric.</param>
/// <param name="sampleRate">The sample rate. Defaults to <see cref="Metric.DefaultSampleRate" />.</param>
/// <returns>The new counter instance.</returns>
Counter CreateCounter(string name, IReadOnlyList<string> allNames,
double sampleRate = Metric.DefaultSampleRate);
/// <summary>
/// Increment a <see cref="Timing" /> directly without having to allocate one.
/// </summary>
/// <param name="name">The specified name of the timing we're going to measure.</param>
/// <param name="value">The value to be recorded for this timing.</param>
/// <param name="sampleRate">The sample rate. Defaults to <see cref="Metric.DefaultSampleRate" />.</param>
void RecordTiming(string name, long value, double sampleRate = Metric.DefaultSampleRate);
/// <summary>
/// Increment a <see cref="Timing" /> directly without having to allocate one.
/// </summary>
/// <param name="name">The specified name of the timing we're going to measure.</param>
/// <param name="value">The value to be recorded for this timing.</param>
/// <param name="sampleRate">The sample rate. Defaults to <see cref="Metric.DefaultSampleRate" />.</param>
void RecordTiming(string name, TimeSpan value, double sampleRate = Metric.DefaultSampleRate);
/// <summary>
/// Creates a <see cref="TimingScope" /> that can be used to automatically track and record
/// a <see cref="Timing" /> upon disposal.
/// </summary>
/// <param name="name">The name of this <see cref="Timing" />.</param>
/// <param name="sampleRate">The sample rate. Defaults to <see cref="Metric.DefaultSampleRate" />.</param>
/// <returns>A disposable <see cref="TimingScope" />.</returns>
TimingScope StartTiming(string name, double sampleRate = Metric.DefaultSampleRate);
/// <summary>
/// Creates a <see cref="TimingScope" /> that can be used to automatically track and record
/// a <see cref="Timing" /> upon disposal.
/// </summary>
/// <param name="allNames">The set of all names that we can use for this metric.</param>
/// <param name="sampleRate">The sample rate. Defaults to <see cref="Metric.DefaultSampleRate" />.</param>
/// <returns>A disposable <see cref="TimingScope" />.</returns>
TimingScope StartTiming(IReadOnlyList<string> allNames, double sampleRate = Metric.DefaultSampleRate);
/// <summary>
/// Creates a new <see cref="Timing" /> instance.
/// </summary>
/// <param name="name">The specified name of the timing we're going to measure.</param>
/// <param name="sampleRate">The sample rate. Defaults to <see cref="Metric.DefaultSampleRate" />.</param>
/// <returns>The new timing instance.</returns>
/// <example>
/// using(myMonitor.StartTiming("foo")){
/// // actual work
/// } // will automatically record a new "foo" Timing instance based on the values recorded.
/// </example>
Timing CreateTiming(string name, double sampleRate = Metric.DefaultSampleRate);
/// <summary>
/// Creates a new <see cref="Timing" /> instance.
/// </summary>
/// <param name="name">The specified name of the timing we're going to measure.</param>
/// <param name="allNames">The set of all names that we can use for this metric.</param>
/// <param name="sampleRate">The sample rate. Defaults to <see cref="Metric.DefaultSampleRate" />.</param>
/// <returns>The new timing instance.</returns>
/// <example>
/// using(myMonitor.StartTiming("foo")){
/// // actual work
/// } // will automatically record a new "foo" Timing instance based on the values recorded.
/// </example>
Timing CreateTiming(string name, IReadOnlyList<string> allNames, double sampleRate = Metric.DefaultSampleRate);
}
}
Performance vs. Flexibility Considerations
The methods described in the comments do what they say, but there is one subtle difference: what's the difference between Record_ and Create_ - the answer is a matter of performance vs flexibility.
If you're in a scenario where you may need to create lots of new metric types on the fly, the IMonitor.Record_ methods allow you to accomplish that easily - you just need to pass in a custom name into the method and Phobos will take care of the rest.
However, suppose you're in a scenario where you have to make a high number of successive calls to the same Counter, Gauge, or Timing over and over again. In this case, if you're concerned about performance you should call the IMonitor.Create_ method and cache the object returned by that method. Re-using the same Metric (the common base class all of the above derive from) instance will reduce the amount of string formatting and allocation used on a per-invocation basis and thus may be a better choice for performance-sensitive contexts.