Add built-in support for generating OpenAPI document from APIs

[captainsafia]

This issue outlines background content, proposed design, and proposed API to add support for built-in OpenAPI document generation to minimal APIs and MVC in the Microsoft.AspNetCore.OpenApi package. It also captures some open questions and future plans for work in this space. Some of the implementation details are subject to change as work evolves.

Background

The OpenAPI specification is a standard for describing HTTP APIs. The standard allows developers to define the shape of APIs that can be plugged into client generators, server generators, testing tools, documentation and more. Despite the universality and ubiquity of this standard, ASP.NET Core does not provide support for OpenAPI by default within the framework.

ASP.NET Core does not provide first-class, built-in support for OpenAPI. Instead, ASP.NET Core has shipped with support for ApiExplorer (not to be confused with Visual Studio's API Explorer) for quite some time. The ApiExplorer is a helpful abstraction that provides metadata about the routes that are registered in an application. This metadata is accessible via the DI container and is used by tools in the ecosystem like Asp.Api.Versioning, NSwag, and Swashbuckle to introspect and query the metadata aggregated by ApiExplorer.

In .NET 6, minimal APIs was introduced and support for minimal APIs was added to ApiExplorer via the EndpointMetadataApiDescriptionProvider which allowed querying the ApiExplorer metadata to introspect registered minimal endpoints in an application.

In .NET 7, the Microsoft.AspNetCore.OpenApi package was introduced (note: this package ships via NuGet and is not part of the shared framework). It exposed the WithOpenApi extension method for modifying the OpenApiOperation associated with a single endpoint in minimal APIs. The package takes a dependency on the Microsoft.OpenApi package which provides an object model and deserializers/serializers for interacting with various versions of the OpenAPI specification.

The evolution of our OpenAPI "support" has resulted in a large quantity of bugs and feature gaps (ref). To resolve this and provide a more seamless experience for users, we're incorporating OpenAPI document generation as a first-class feature in ASP.NET Core.

Future Implementation

Implementation Overview

The flow diagram outlines the proposed implementation. New components are in a bordered box. The OpenApiComponentService is responsible for managing state that will be serialized to the top-level components field in the OpenAPI document. At the moment, it is largely responsible for generating and managing JSON schemas associated with application types. The OpenApiDocumentService exposes a GetOpenApiDocument method for resolving the OpenApiDocument associated with an application. These new components build on top of the metadata that is produced by the ApiExplorer, allowing us to take advantage of a time-tested and well-established component.

flowchart LR
	mvc[Controller-based API]
	minapi[Minimal API]
	defdescprov[DefaultApiDescriptionProvider]
	endpdescprov[EndpointMetadataDescriptionProvider]
	desccoll[IApiDescriptionCollection]
	compservice[[OpenApiComponentService]]
	docsvc[[OpenApiDocumentService]]
	idocprovider[[IDocumentProvider]]
	meas[Microsoft.Extensions.ApiDescription.Server]
	mvc --> defdescprov 
	mvc --> endpdescprov
	minapi --> endpdescprov
	defdescprov --> desccoll
	endpdescprov --> desccoll
	desccoll --> docsvc
	idocprovider --> docsvc
	meas --> idocprovider
	compservice --> docsvc

Loading

Document Generation

The OpenAPI document contains a well-defined set of fields to be populated with established meanings. To make it easier to understand why documents are generated the way they are, we intend to document and implement the following semantics are used when generating the OpenAPI document.

• info.name: Derived from the entry-point assembly name.
• info.version: Defaults to 1.0.
  • Future: integrate with versioning information in Asp.Versioning.
• servers: Derived from the host info registered in the IServer implementation.
• paths: Aggregated from ApiDescriptions defined in the ApiExplorer
  • paths.operations.parameters: captures all routes understood by the model binding systems of MVC and minimal except inert parameters like those coming from the DI container
    • Information in route constraints and validation attributes will be added onto the schemas associated with each parameter type
• components: Aggregates JSON schemas from the OpenApiComponentService (see below)
• security: No autogeneration by default.
• tags: Aggregates all tags discovered during the construction of the document.

JSON Schema Generation

OpenAPI definitions rely on a derivative implementation of JSON Schema to describe the shapes of types that are used in request parameters and responses. .NET does not contain a built-in solution for generating or validating JSON schemas from .NET types (although this work is outlined here). To fill this gap, this implementation will ship with an OpenApiComponentService that uses System.Text.Json's type resolver APIs to generate the underlying OpenAPI schemas. This gives us the opportunity to address some of the gaps that exist with how certain types are currently implemented as seen in the following issues:

• Correct handling duplicate FromForm parameters in a given API (ref)
• Correct handling nullability annotations for response types (ref)
• Correctly handling IFormFile and IFormFileCollection inputs
• Correctly handling polymorphic serialization metadata from STJ in the generated schema using oneOf and type discriminaotrs
• Correctly handling inheritance hierarchies in types using allOf
• Applying all supported route constraints to generated schemas
• Applying all supported ComponentModel annotations to generated schemas

Note: The version of OpenAPI.NET that we intend to target uses the JsonSchema.NET to handle JSON schema representation in the OpenAPI document.

Question: As part of this work, we'll build a test bed to validate schema generation behavior. Please share examples of APIs you'd like to make sure appropriate schemas are generated for so they can be included in the test set.

Generating Operation IDs

The OpenAPI specification consists of operations that uniquely identify an endpoint by it's operation type (HTTP method) and path. Each of these operations can optionally include an operation ID, a unique string that identifies the operation. Operation IDs play an important role in OpenAPI integrations with client generators, OpenAI plugins, and more. Users can define operation IDs for each endpoint in their application themselves, but ideally we should be able to generate high-quality operation IDs by default to make the process more seamless for the user. Operation IDs should be deterministic so it's not sufficient to generate an arbitrary GUID for each operation in an application. The proposed semantics for generated operation IDs are as follows:

• If a name is provided on an action via the route method, use that name.
• If a name is provided on an action or endpoint using the EndpointName metadata, use that name.
• If neither is available, attempt to create an operation ID using information available in the route template and endpoint metadata in sequential order.
  • The HTTP method associated with the operation stringified (GET, POST, etc.)
  • If route segments exist on the application, the route segment values concatenated by _.
  • If the route segments contain parameters, use the parameter name sans any constraints.
• If duplicate operation IDs are generated with these semantics, disambiguate them with a monotonically increasing integer (Get_1, Get_2, etc.)

Swagger UI (Or Lack Thereof)

The web templates currently expose two endpoints by default in relation to OpenAPI: one that serves the OpenAPI document as a JSON file and another that serves an Swagger UI against the backing JSON file. At the moment, we don't intend to ship with Swagger UI as a supported UI in ASP.NET Core. Although it provides the benefit of an accessible UI for ad-hoc testing, it introduces engineering overhead around shipping (need to bundle web assets), has some security implications (it's easy to accidently leak client secrets for certain authentication configurations), and introduces maintenance overhead (need to make sure that we upgrade swagger-ui as needed).

Since swagger-ui is independent of the OpenAPI document, users can independently incorporate into their applications if needed via third-party packages or their own code (swagger-ui just needs a pointer to the served OpenAPI JSON file). Users can also take advantage of other ad-hoc testing tools that plug in to OpenAPI, like ThunderClient.

Customizing document generation

The automatic document generation will make use of metadata exposed via ApiExplorer to generate the OpenAPI document. There are currently several avenues that exist in the framework for influencing the generation of this document:

• Accepts and Produces metadata allow limited support for customizing the content-types and object types associated with an endpoints parameters and responses
• EndpointName, EndpointTags, EndpointSummary, and EndpointDescription metadata and their associated attributes/extension methods allow customization of the tags, summary, and description fields associated with a request
• WithOpenApi extension method supports overriding the OpenApiOperation associated with an endpoint in its entirety

The customization story is disparate at the moment, and it's largely a result of the way the system evolved. As we move to support generating entire documents, there are certain aspects we don't provide APIs for customizing, like the version number specified in the info of the OpenAPI document or the supported security schemes. This effort provides a nice avenue for unifying the various strategies that have proliferated in the codebase for customizing these aspects.

The current customization options that we provide are largely endpoint-focused, mostly because we've never had the need to manage document-level settings like properties in the info property of the OpenAPI document.

XML Documentation Support

One of the most upvoted issues with regards to OpenAPI/ApiExplorer in our repo is around supporting XML code comments (ref). Being able to generate the OpenAPI document as standard functionality pushes us towards support for this feature. Work here will requiring reading the generated XML documentation at runtime, mapping members to the appropriate operations, and merging information from the XML comment into the target operation.

Ecosystem Integration

OpenAPI plays an important role into several existing technologies in the space. At the center of this effort is the goal to produce a high-quality OpenAPI document that provides strong integrations with existing tools in the ecosystem including:

• Kiota client generation
• NSwag generators
• Asp.Api.Versioning
• OpenAI plugins
• Swagger UI/Redoc and other ad-hoc testing tools

Question: Are there other components we should validate integration with?

Build Time OpenAPI Document Generation

As far as build time generation goes, we'll plug-in to the existing integration provided by the dotnet-getdocument command line tool. dotnet-getdocument enforces a loose, string-based contract that requires an implementation of Microsoft.Extensions.ApiDescription.IDocumentProvider to be registered in the DI container. This means that users will be able to generate OpenAPI files at build-time using the same strategy they currently do by enabling the following MSBuild flags.

<OpenApiGenerateDocumentsOnBuild>true</OpenApiGenerateDocumentsOnBuild>
<OpenApiDocumentsDirectory>.\</OpenApiDocumentsDirectory>

Behind the scenes, this functionality works by booting up the entry point assembly behind the scenes with an inert IServer implementation, querying the DI container on the entry point's host for the IDocumentProvider interface, then using reflection to invoke the appropriate methods on that interface.

Note: although I've explored strategies for generating OpenAPI documents at build-time using static analysis, this is out-of-scope for the time being.

Runtime OpenAPI Document Generation

Templates will be updated to include the following code in Program.cs when OpenAPI is enabled. The AddOpenApi method registers the appriopriate OpenAPI-related services and MapOpenApi exposes an endpoint that serves the serialized JSON document.

var builder = WebApplication.CreateBuilder();

builder.Services.AddOpenApi();

var app = builder.Build();

app.MapOpenApi();

app.MapGet("/", () => "Hello world!");

app.Run();

Underlying API

See #54600 for full API review.

// Assembly: Microsoft.AspNetCore.OpenApi
namespace Microsoft.AspNetCore.Builder;

public static class IEndpointRouteBuilderExtensions
{
  public static IEndpointRouteBuilder MapOpenApi(this IEndpointRouteBuilder builder);
}

// Assembly: Microsoft.AspNetCore.OpenApi
namespace Microsoft.Extensions.DependencyInjection;

public static class IServiceCollectionExtensions
{
  public static IServiceCollection AddOpenApi(this IServiceCollection serviceCollection);
  public static IServiceCollection AddOpenApi(this IServiceCollection serviceCollection, Action<OpenApiOptions> configureOptions);
}

// Assembly: Microsoft.AspNetCore.OpenApi
namespace Microsoft.AspNetCore.OpenApi;

public class OpenApiOptions
{
  public string JsonFilePath { get; set; }
  public OpenApiSpecVersion OpenApiVersion { get; set; }
}

Tasks

preview4

• Support generating OpenAPI document using ApiExplorer metadata
• Support generating JSON schemas using System.Text.Json schema generation support and Microsoft.OpenAPI
• Support serving JSON OpenAPI document from server
• Support generating OpenAPI document at build with Microsoft.Extensions.ApiDescription.Server infrastructure
• Support customization of OpenAPI document (document and operations)

preview5

• Replat WithOpenApi on top of operation transformers
• Update web API templates to use new APIs
• Verify native AoT comapt for minimal APIs scenarios

preview6

• Add support for schema transformers
• Add support for reference IDs and schema reference

preview7

• Support integrating XML comments into generated code
• Support configuring discriminator mappings on polymorphic types
• Update gRPC JSON transcoding implementation

P1: Ecosystem Integration and Enhancements

• Support incorporating API versions from Asp.Versioning
• Validate generation experience with Kiota, NSwag

[cphorton]

Apologies if this is the wrong place to ask these questions, but will Yaml file generation be supported out of the box?

It would also be great to have the ability to perform a schema validation if you are doing schema first I.e. load a schema from a file and validate that the generated OpenAPI spec matches the loaded schema.

[captainsafia]

Apologies if this is the wrong place to ask these questions, but will Yaml file generation be supported out of the box?

I haven't thought about YAML generation as something we'd ship as part of preview4. However, it should be trivial to support and something that we've discussed in the API review of the first batch of APIs (ref).

If folks will get a lot of value from supporting YAML generation by default, I'd be happy to add support for it.

[cphorton]

If folks will get a lot of value from supporting YAML generation by default, I'd be happy to add support for it.

Thanks for your swift response.

I understand that the objective here is not necessarily a like for like replacement for Swashbuckle, but Yaml was one of the formats that was supported by that library, you just needed to request the swagger.json spec from the endpoint with a .yaml extension

[captainsafia]

I understand that the objective here is not necessarily a like for like replacement for Swashbuckle, but Yaml was one of the formats that was supported by that library, you just needed to request the swagger.json spec from the endpoint with a .yaml extension

Yep, the serialization functionality in both places is actually supported by the underlying Microsoft.OpenAPI package so support for YAML is largely a manner of wiring it up and calling the right APIs. That being said, I want to be careful about introducing too many configurability toggles early on hence the ask for user feedback here. Let's use the thumbs up reaction on this comment to signal that having YAML serialization before GA would be valuable.