This post introduces Codecs and the Transcoder. These extract and decouple the encoding / decoding from of the ASP.NET pipeline, making them reusable and testable. Our goal is to overcome the challenges noted in part 2 of this series.
A Few Concerns…
Content-negotiation is too difficult to use in ASP.NET Core.
A few concerns:
- The code to format and parse representations is deep inside the “plumbing” of ASP.NET.
- It is coupled to servers side implementation details, like MVC, HttpContext, and ModelState.
- It is not easily testable.
- The media type matching uses a “subset” logic, which can result in false matches.
Let’s address these concerns.
Codecs = Encoders and Decoders
Codecs are not a new concept. They translate data from one format to another. We rely on the MP4 codecs when we watch videos.
Network APIs need to translate from the format of the representation on the wire to the format of bits in an instance of an resource class. For example, convert a JSON weather forecast to an instance of the WeatherForecast class.
Decoders
Decoders translate from representations like JSON into instances of a resource class. See the IDecoder interface, BaseDecoder, and DecoderContext classes.
Portions of the Microsoft InputFormatter implementation are copied into the BaseDecoder and then customized. One important change is the mediatype matching. ASP.NET core matches mediatype parameters using a “subset” approach. This can result in false matches.
The BaseDecoder uses a full match, rather than subset match. It requires that all mediatype parameters match except the charset and “q” parameters.
IDecoder and DecoderContext
public interface IDecoder
{
IList<MediaTypeHeaderValue> SupportedMediaTypes { get; }
bool CanRead(DecoderContext context);
Task<object> ReadAsync(DecoderContext context);
}
public class DecoderContext
{
public DecoderContext()
{
ReaderFactory = (stream, encoding) => { return new StreamReader(stream, encoding); };
}
/// <summary>
/// Gets the System.Type of the decoded object to return.
/// This is optional.
/// </summary>
public Type ModelType { get; set; }
/// <summary>
/// Gets a delegate which can create a System.IO.TextReader for the input stream body.
/// </summary>
public Func<Stream, Encoding, TextReader> ReaderFactory { get; set; }
/// <summary>
/// The encoding to use for the input stream.
/// </summary>
public Encoding Encoding { get; set; } = Encoding.UTF8;
/// <summary>
/// The media type (aka Content Type) of the data in the InputStream
/// </summary>
public MediaTypeHeaderValue MediaType { get; set; }
/// <summary>
/// The stream of data to decode.
/// </summary>
public Stream InputStream { get; set; }
}
Encoders
Encoders translate from an instance to a representation like JSON. See the IEncoder interface, BaseEncoder, and EncoderContext classes.
Portions of the Microsoft OutputFormatter implementation are copied into the BaseEncoder and then customized. One important change is the mediatype matching. ASP.NET core matches mediatype parameters using a “subset” approach. This can result in false matches.
The BaseEncoder uses a full match, rather than subset match. It requires that all mediatype parameters match except the charset and “q” parameters.
See the MediaTypeHeaderValueSupport class and associated unit tests for details.
IEncoder and EncoderContext
public interface IEncoder
{
IList<MediaTypeHeaderValue> SupportedMediaTypes { get; }
bool CanWrite(EncoderContext context);
Task WriteAsync(EncoderContext context);
}
public class EncoderContext
{
public EncoderContext()
{
WriterFactory = (stream, encoding) => { return new StreamWriter(stream, encoding); };
}
/// <summary>
/// Gets or sets the media type that should be encoded.
/// Optional. If null, the ActualMediaType will be set to the SupportedMediaType of the encoder.
/// </summary>
public virtual MediaTypeHeaderValue DesiredMediaType { get; set; }
/// <summary>
/// Gets or sets the media type that the encoder wrote to the stream.
/// </summary>
public virtual MediaTypeHeaderValue ActualMediaType { get; set; }
/// <summary>
/// Gets or sets the object to encode to the stream.
/// </summary>
public virtual object Object { get; set; }
/// <summary>
/// Gets or sets the <see cref="Type"/> of the object to encode to the stream.
/// </summary>
public virtual Type ObjectType { get; set; }
/// <summary>
/// Gets a delegate which can create a System.IO.TextWriter for the output stream body.
/// </summary>
public Func<Stream, Encoding, TextWriter> WriterFactory { get; set; }
/// <summary>
/// The encoding to use in the output stream.
/// </summary>
public Encoding Encoding { get; set; } = Encoding.UTF8;
/// <summary>
/// The stream for writing encoded data.
/// </summary>
public Stream OutputStream { get; set; }
}
Example WeatherForcast Codecs
The CodecExample refactors the code from the formatters of part 2 into a new set of custom codecs and serialized codecs. These codecs are leaner, simpler, and focus on a single responsibility. They are not coupled to server side MVC concepts or classes.
The CodecExample includes both flavors (custom and serialized) to mimic the Part 2 formatters. Each relies on a base class to handle some common details.
WeatherForecastCustomV1Encoder
public class WeatherForecastCustomV1Encoder : BaseNewtonsoftJsonEncoder
{
public const string WeatherForecastJsonV1MediaType = "application/json; Domain=Example.WeatherForecast.Custom; Version=1";
public WeatherForecastCustomV1Encoder()
{
AddSupportedMediaType(WeatherForecastJsonV1MediaType);
}
protected override bool CanWriteType(Type type)
{
return typeof(WeatherForecast).IsAssignableFrom(type);
}
public override JToken EncodeToJToken(EncoderContext context)
{
WeatherForecast forecast = (WeatherForecast)context.Object;
JToken jobject = new JObject();
// UTC date format. Full control. No ambiguity.
jobject["date"] = forecast.Date.ToString("yyyy-MM-ddTHH:mm:ss.fffZ");
// Structure the json as works best for json. Doesn't have to match object exactly.
var temperatureNode = jobject["temperature"] = new JObject();
temperatureNode["celcius"] = forecast.TemperatureC;
temperatureNode["farenheight"] = forecast.TemperatureF;
jobject["summary"] = forecast.Summary;
return jobject;
}
}
Transcoder
The Transcoder holds a list of encoders and a list of decoders. It locates and then invokes the proper codec based on the specified mediatype.
The Transcoder uses an approach similar to the formatters where it asks each codec if it can read or write the given context. It selects the first codec that returns true.
The Transcoder also exposes methods to get a codec by mediatype. This can be useful when composing codecs. Multiple representations may contain the same structure. The formatting and parsing of this could be encapsulated into separate codecs for reuse. Then codecs can get that instance from the Transcoder by asking for it by mediatype.
See the Transcoder implementation in the CodecExample.Common project.
Transcoder
public class Transcoder
{
public IList<IDecoder> Decoders { get; } = new List<IDecoder>();
public IList<IEncoder> Encoders { get; } = new List<IEncoder>();
public bool CanRead(DecoderContext context)
{
foreach (var decoder in Decoders)
{
if (decoder.CanRead(context))
{
return true;
}
}
return false;
}
public async Task<object> ReadAsync(DecoderContext context)
{
foreach (var decoder in Decoders)
{
if (decoder.CanRead(context))
{
return await decoder.ReadAsync(context);
}
}
throw new FormatException($"Unable to find a decoder for {context.MediaType}.");
}
public bool CanWrite(EncoderContext context)
{
foreach (var encoder in Encoders)
{
if (encoder.CanWrite(context))
{
return true;
}
}
return false;
}
public async Task WriteAsync(EncoderContext context)
{
foreach (var encoder in Encoders)
{
if (encoder.CanWrite(context))
{
await encoder.WriteAsync(context);
return;
}
}
throw new FormatException($"Unable to find an encoder for {context.DesiredMediaType} that can encode an object of type {context.Object.GetType().Name}.");
}
public IDecoder GetDecoder(DecoderContext context)
{
foreach (var decoder in Decoders)
{
if (decoder.CanRead(context))
{
return decoder;
}
}
return null;
}
public IEncoder GetEncoder(EncoderContext context)
{
foreach (var encoder in Encoders)
{
if (encoder.CanWrite(context))
{
return encoder;
}
}
return null;
}
}
Wiring Into ASP.NET
In Part 2, we created multiple new InputFormatters and OutputFormatters. Here, instead of multiple formatters we will use one TranscoderInputFormatter and one TranscoderOutputFormatter. Each of these will leverage the Transcoder and multiple codecs.
Startup and Configuration
The Startup.cs class shows the key steps to setting up the server side use of the Transcoder and codecs.
- Create a single Transcoder.
- Create all relevant codecs and add to the Transcoder.
- Register the Transcoder with the ServiceCollection as a singleton.
- Create the TranscoderInputFormatter and register it as a formatter for ASP.NET
- Create the TranscoderOutputFormatter and register it as a formatter for ASP.NET
- Remove the framework’s built in JSON and XML formatters. This is optional, but prevents unintended or unexpected formatting.
- Set MVC formatter options.
Startup.cs
// This method gets called by the runtime. Use this method to add services to the container.
public void ConfigureServices(IServiceCollection services)
{
// -----------------------
// Create and hydrate the transcoder.
// -----------------------
var transcoder = new Transcoder();
// V1 Custom Codecs
transcoder.Encoders.Add(new WeatherForecastCustomV1Encoder());
transcoder.Encoders.Add(new WeatherForecastCollectionCustomV1Encoder());
transcoder.Decoders.Add(new WeatherForecastCustomV1Decoder());
transcoder.Decoders.Add(new WeatherForecastCollectionCustomV1Decoder());
// V2 Custom Codecs
transcoder.Encoders.Add(new WeatherForecastCustomV2Encoder());
transcoder.Encoders.Add(new WeatherForecastCollectionCustomV2Encoder());
transcoder.Decoders.Add(new WeatherForecastCustomV2Decoder());
transcoder.Decoders.Add(new WeatherForecastCollectionCustomV2Decoder());
// V1 Serilization-based Codecs
transcoder.Encoders.Add(new WeatherForecastSerializedV1Encoder());
transcoder.Encoders.Add(new WeatherForecastCollectionSerializedV1Encoder());
transcoder.Decoders.Add(new WeatherForecastSerializedV1Decoder());
transcoder.Decoders.Add(new WeatherForecastCollectionSerializedV1Decoder());
// Other
transcoder.Encoders.Add(new ValidationProblemsEncoder());
services.AddSingleton<Transcoder>(transcoder);
services.AddControllers(options =>
{
// -----------------------
// Output formatters / Encoders
// -----------------------
// For the example, remove the built-in formatter.
options.OutputFormatters.RemoveType<SystemTextJsonOutputFormatter>();
options.OutputFormatters.Add(new TranscoderOutputFormatter(transcoder));
// -----------------------
// Input formatters / Decoders
// -----------------------
// For the example, remove the built-in formatter.
options.InputFormatters.RemoveType<SystemTextJsonInputFormatter>();
options.InputFormatters.Add(new TranscoderInputFormatter(transcoder));
// -----------------------
// Relevant settings
// -----------------------
// Gets or sets the flag which causes content negotiation to ignore Accept header
// when it contains the media type */*.
// false by default.
//
// Set to true if the server wants to comprehend the valid case of a */* media type.
// Recommended value: true
options.RespectBrowserAcceptHeader = true;
// Gets or sets the flag which decides whether an HTTP 406 Not Acceptable response
// will be returned if no formatter has been selected to format the response.
// false by default.
//
// Set to true if the server wants to be strict about supported media types.
// Recommended value: true
options.ReturnHttpNotAcceptable = true;
});
}
Unit Testing Codecs
An important benefit of extracting the formatting is to make it easily testable. With the logic in encoders and decoders, this is straight-forward and powerful.
The suggested pattern is to write two “round-trip” tests for each pair of codecs:
- Representation -> Resource -> Representation
- Resource -> Representation -> Resource
For both of these tests, start with a fully populated structure. Then the test should use the both the encoder and decoder. Then use a tool like FluentAssertions and FluentAssertions.Json to compare the original with the transcoded round-tripped output. If they are identical, then the codec pair is comprehensive and correct.
See CodecExample.Tests project and the CodecTests class for examples of the above.
Note: When comparing two JSON representations, you MUST use the FluentAssertions.Json package to compare two JObjects. Otherwise, the regular FluentAssertions library will return a false positive due to the nuances of the JObject. However the FluentAssertions.Json package handles JObjects correctly and semantically.
What Did Not Change
The rest of the sample code is relatively unchanged.
The controllers exposes the same routes as Part 2 and uses the same attributes.
The curl commands are the same as Part 2.
Opportunities for Improvement
The code in these examples is a good starting point for production use, but could benefit from a few investments:
- More Tests
- Logging
- Configurable or overridable MediaType parameter matching rules
- Startup configuration helper method
Conclusions
The CodecExample introduced the Transcoder and Codecs. These decouple the formatting logic, which makes for simpler, testable, and reusable encoders and decoders. The ASP.NET plumbing is encapsulated into two formatters, rather than many.
In Part 4, we’ll use the Transcoder and Codecs from the client side to encode HTTP requests and decode HTTP responses. This doubles the ROI of the new concepts.
This post is part of a series:
the alnernative 
Leave a Reply