Surprising behavior in references: component is resolved only when in a different module?

[sphaso]

Hello,
we stumbled upon a behavior which we find surprising and would like to know if this is expected and if we are breaking any intended pattern or best practice.
I created a small phoenix project to reproduce: /~https://github.com/sphaso/api_spec_misunderstanding

/~https://github.com/sphaso/api_spec_misunderstanding/blob/main/lib/apispecmisunderstanding_web/schema/abc.ex Contains two Schemas pointing to what's effectively the same reference (Submodule) except that Working references it from another Elixir module, while NotWorking references it from a submodule of Abc.
Testing this in /~https://github.com/sphaso/api_spec_misunderstanding/blob/main/test/apispecmisunderstanding_web/schema_test.exs , cast_value of NotWorking gives an UndefinedFunctionError. The following warning is output:

....warning: Unresolved schema module: nil.
Use OpenApiSpex.resolve_schema_modules/1 to resolve modules ahead of time.

  (open_api_spex 3.20.1) lib/open_api_spex.ex:407: OpenApiSpex.resolve_schema/2
  (open_api_spex 3.20.1) lib/open_api_spex/cast/any_of.ex:47: OpenApiSpex.Cast.AnyOf.cast_any_of/3
  (elixir 1.15.4) lib/enum.ex:1693: Enum."-map/2-lists^map/1-1-"/2
  (open_api_spex 3.20.1) lib/open_api_spex/cast/array.ex:59: OpenApiSpex.Cast.Array.cast_array/1
  (open_api_spex 3.20.1) lib/open_api_spex/cast/array.ex:10: OpenApiSpex.Cast.Array.cast/1
  (ex_unit 1.15.4) lib/ex_unit/assertions.ex:785: ExUnit.Assertions.assert_raise/2
  test/apispecmisunderstanding_web/schema_test.exs:10: ApispecmisunderstandingWeb.SchemaTest."test cast_value/2 Not working"/1
  (ex_unit 1.15.4) lib/ex_unit/runner.ex:463: ExUnit.Runner.exec_test/2
  (stdlib 5.0.2) timer.erl:270: :timer.tc/2
  (ex_unit 1.15.4) lib/ex_unit/runner.ex:385: anonymous fn/5 in ExUnit.Runner.spawn_test_monitor/4

Trying to follow the advice gives the same result (see "Also Not Working" test)

However, using the plug (see /~https://github.com/sphaso/api_spec_misunderstanding/blob/main/lib/apispecmisunderstanding_web/controllers/test_controller.ex and related test) the schema seems to be correctly resolved.

Questions:

• is it intended behavior that some references types are not resolved depending if they are in the same module or in another one?
• why is then the plug working? Is there any step we are missing to ensure all references are resolved ahead of time?

I'm happy to help clarifying this issue further if needed. Thank you for your time and attention.

[mbuhot]

Hi @sphaso 👋

The short answer to your questions:

1. No, the reason it is not working is because the caller isn't passing enough information into cast_value for it to resolve the reference. I'm not sure why we have OpenApiSpex.cast_value/2, it will fail as soon as it encounters a %Reference{} to another schema. Prefer to use OpenApiSpex.cast_value/3 which includes the OpenApi struct.

2. The plug is working because it uses OpenApiSpex.cast_and_validate/5, including the full OpenApi struct, allowing references to be looked up in the components.

(Edited this comment to update the details on point 2)

---

With more explaining:

In this code:
/~https://github.com/sphaso/api_spec_misunderstanding/blob/main/lib/apispecmisunderstanding_web/schema/abc.ex#L16
An explicit %Reference{} is being used.

In this code: /~https://github.com/sphaso/api_spec_misunderstanding/blob/main/lib/apispecmisunderstanding_web/schema/abc.ex#L29
The Submodule schema is being in-lined into the Working schema, it isn't using a reference.

The third way is to reference the module name (not calling the .schema() function on it), and leave it to OpenApiSpex.resolve_schema_modules to convert the module name to a %Reference{}

  defmodule Option3 do
    @moduledoc false

    OpenApiSpex.schema(%{
      type: :array,
      items: %Schema{
        anyOf: [
          ApispecmisunderstandingWeb.Schemas.Abc.Submodule
        ]
      }
    })
  end

---

This test: /~https://github.com/sphaso/api_spec_misunderstanding/blob/main/test/apispecmisunderstanding_web/schema_test.exs#L10 can never work because the schema contains an explicit reference, but the calling code isn't passing in the other schemas for it to lookup. Try using OpenApiSpex.cast_value/3, passing in the whole OpenApi struct, so that any references can be looked up in the OpenApi.components map.
/~https://github.com/open-api-spex/open_api_spex/blob/master/lib/open_api_spex.ex#L72-L78

This test: /~https://github.com/sphaso/api_spec_misunderstanding/blob/main/test/apispecmisunderstanding_web/schema_test.exs#L14 is working because the Submodule schema was in-lined. No schema lookups are required to cast the value.

---

To help understand what's going on, you can run: mix openapi.spec.json --spec ApispecmisunderstandingWeb.ApiSpec and see the JSON output:

{
  "info": { "title": "Api Spex Misunderstanding API", "version": "1.0" },
  "openapi": "3.0.0",
  "security": [],
  "servers": [{ "url": "http://localhost:4000", "variables": {} }],
  "tags": [],
  "components": {
    "responses": {},
    "schemas": {
      "NotWorking": {
        "items": { "anyOf": [{ "$ref": "#/components/schemas/Submodule" }] },
        "title": "NotWorking",
        "type": "array"
      },
      "Submodule": { "title": "Submodule", "type": "integer" },
      "Working": {
        "items": { "anyOf": [{ "title": "Submodule", "type": "integer" }] },
        "title": "Working",
        "type": "array"
      }
    }
  },
  "paths": {
    "/api_spec/test": {
      "post": {
        "callbacks": {},
        "description": "",
        "operationId": "ApispecmisunderstandingWeb.TestController.create",
        "parameters": [],
        "requestBody": {
          "content": {
            "application/json": {
              "schema": { "$ref": "#/components/schemas/NotWorking" }
            }
          },
          "description": "NotWorking",
          "required": false
        },
        "responses": {},
        "summary": "Plug works but cast_value does not?",
        "tags": []
      }
    }
  }
}

[sphaso]

Thanks a lot for your detailed explanation!