Introduce the ability to create "Virtual" circuits

[jsenecal]

NetBox version

v3.5.4

Feature type

Change to existing functionality

Proposed functionality

Allow a new Circuit Termination variant which would link to a "Parent" circuit and optionally allow the user to select a virtual interface of either Parent side connected interface.

Currently this is not possible as Circuits require a physical port, which prevent the assignment of a sub-interface for instance.

Use case

As the operator of a service provider network, we often lease connections from third parties.
These connections often have multiple components, some of which run "on top" of other components, think many different vlans on a parent link, with each vlan being terminated to a different location or "Z" side.

This was discussed in #10558, as well as during community meetings and multiple slack conversations.
#10558 (comment) is a very good description of the use case.

Allowing netbox to properly document these is a must for service providers such as myself and would be of great benefit to the netbox community.

Database changes

Amongst other things, the CircuitTermination model would need to have an additional ForeignKey to circuits.CircuitTermination.

Additionally, a ForeignKey to dcim.Interface would allow to track the sub-interface related to the Parent circuit termination connected interface (and thus enable traces from sub-interfaces as well).

External dependencies

None

Additional information

I would be more than happy to provide a PR with those changes and related UI/Tests/Docs should the feature be accepted.

[shatt79]

I'd like to expand upon this.

In reality, a physical circuit would have two endpoints but multiple intermediate path elements. The circuits module in Netbox is practically useless because I can't see anything but the two endpoints. I want to see the DLR of the entire path including all patch panels, WDMs, etc along the way.

In real networks, "services" are multiplexed on a physical path. I can carry voice, video, data, etc on the same cable. But within Netbox, I have no way to document that. The proper way to model this would be to have a "parent" or "infrastructure" physical circuit (with a circuit ID) that documents the entire physical path from A to Z, much like the cable trace view shows. The problem with the cable trace is that I can't assign a circuit ID to that or reference it in any way. The second part to properly modeling this would be to allow the creation of "child" or "service" circuits underneath the physical parent/infrastructure circuit. These circuits would/could be different VLANs, PPP groups, etc, possibly terminating to different sub-interfaces, using different DWDM channels, etc. These child circuits would also have their own circuit ID, and would associate with both a tenant and parent circuit. The "overlay" feature was a welcome addition, but its missing a crucial piece that would actually make it usable. When documenting an L2VPN, you don't attach interfaces to them. You would attach EVCs... Virtual circuits. Those EVCs would then have 2 or more UNIs or EFPs (interfaces)

Proper circuit modeling should be the cornerstone of any DCIM/F&E system. Netbox does not allow one to document the physical path of a circuit. If there is a problem on a physical or logical circuit, there is no way to generate an A to Z DLR to hand to a tech for troubleshooting. I have to use a different (and expensive) system for that. This creates swivel chairing and introduces data inconsistency. We would no longer have a source of truth.

[kmorin18]

I agree that from a service provider perspective this is super useful

[jsenecal]

The circuits module in Netbox is practically useless because I can't see anything but the two endpoints. I want to see the DLR of the entire path including all patch panels, WDMs, etc along the way.

In real networks, "services" are multiplexed on a physical path.

The use case you are explaining here is something entirely different @shatt79.

What you want is the ability to define Services components and track those items. This can be achieved in a plugin (this is what I did).

In NetBox, Circuits are there to document third-party "links" to be considered as gray/black boxes where the consumer aka the operator is unaware of the details of such circuits besides their purpose or their A and Z sides.

[gblanchard]

I was also just looking for this feature. Happy to see that familiar names are showing up here :)

[fguigou-dstny]

So... Not sure this is the place to ask this, but is anyone close to an implementation of sorts? Because I'm looking for the same feature, and if not, I'll do it myself, but it would be a shame to write the entire thing just to have it pop up in the next release of Netbox ;)

[jsenecal]

So... Not sure this is the place to ask this, but is anyone close to an implementation of sorts? Because I'm looking for the same feature, and if not, I'll do it myself, but it would be a shame to write the entire thing just to have it pop up in the next release of Netbox ;)

I havent worked on it since its implementation only really makes sense in the core models, and I dont usually work on core models unless there is an accepted FR. Having this designed as a plugin would make it really cumbersome to use.

[dejantep]

I just stumbled upon a case where i was not able to connect port to a circuit because whole port is used as l2vpn termination.
There is no multiplexing on this particualar port but i would still like to document physical connection as a circuit.

Does this adress my issue so i dont create similar PR?

[fguigou-dstny]

As a heads up, I'm currently working on a virtual circuit implementation (packaged in core, not as a plugin). Virtual circuits run on physical circuits or provider networks, but connect to interfaces and can have any arbitrary number of terminations, the idea being to allow cases like multiple endpoints connected to the same provider network.

[dejantep]

As a heads up, I'm currently working on a virtual circuit implementation (packaged in core, not as a plugin). Virtual circuits run on physical circuits or provider networks, but connect to interfaces and can have any arbitrary number of terminations, the idea being to allow cases like multiple endpoints connected to the same provider network.

Does this mean that circuits as we know them today will not change but there will be addition of virtual circuit on top of that which can be connected to any type of interface?

[fguigou-dstny]

As a heads up, I'm currently working on a virtual circuit implementation (packaged in core, not as a plugin). Virtual circuits run on physical circuits or provider networks, but connect to interfaces and can have any arbitrary number of terminations, the idea being to allow cases like multiple endpoints connected to the same provider network.

Does this mean that circuits as we know them today will not change but there will be addition of virtual circuit on top of that which can be connected to any type of interface?

That's the idea.

[bsavardmsk]

As a heads up, I'm currently working on a virtual circuit implementation (packaged in core, not as a plugin). Virtual circuits run on physical circuits or provider networks, but connect to interfaces and can have any arbitrary number of terminations, the idea being to allow cases like multiple endpoints connected to the same provider network.

Does this mean that circuits as we know them today will not change but there will be addition of virtual circuit on top of that which can be connected to any type of interface?

That's the idea.

Hello, i'm also looking for that feature have you been able to implement a plugin or write one that does the job ?

[DanSheps]

As a heads up, I'm currently working on a virtual circuit implementation (packaged in core, not as a plugin). Virtual circuits run on physical circuits or provider networks, but connect to interfaces and can have any arbitrary number of terminations, the idea being to allow cases like multiple endpoints connected to the same provider network.

This sound like a different use case then @jsenecal mentioned. I think you would need an entirely new FR for this to be accepted.

[SteveRodrigue]

Just adding that I'm currently "blocked".

I can't model/automate L2vpn properly within Netbox at the moment, because I can't set virtual interfaces as terminations for a EPL/EVPL (any type of L2vpn basically).

We provision most our L2vpn on virtual interfaces (sub-interfaces), they end being interconnected/meshed together through the L2vpn construct.

At the moment, we could only model Physical L2vpn circuits, which is rarely used: most of the time a single physical interface carries multiple services (DIA, L3vpn, L2vpn...) through virtual/sub interfaces.

To add to the discussion: we can't rely on the VLANs, because in most carrier grade devices, the VLANs are meaningless in the chassis, they are assigned per interfaces and can be reused. So, we can and are delivering different services on the same VLAN ID, but on different ports.

[DanSheps]

We provision most our L2vpn on virtual interfaces (sub-interfaces), they end being interconnected/meshed together through the L2vpn construct.

The termination is not meant to be the VTEP, but the individual interface or VLAN the EVPN "pops out" of. I highly doubt your L2VPN dumps out at a Virtual Interface, it is more likely it drops out at a real physical interface.

Would be glad to be wrong if you want to share the config of your device.

[bcollet]

I highly doubt your L2VPN dumps out at a Virtual Interface, it is more likely it drops out at a real physical interface.

That's actually the case whenever you provision an EVPL/EVP-LAN termination (at least on Cisco and Juniper); your circuit/xconnect is tied to a sub-interface/unit.

Without the ability to connect a (virtual) circuit to a virtual interface, modelling any type of service delivered on an ENNI is impossible, regardless of it its type (EVPL/EVP-LAN/DIA/regular transit/etc.).

[dejantep]

i would also like to add that l2vpn or VPLS/VPWS types of service can be connected to both physical or virtual interfaces.

[SteveRodrigue]

We provision most our L2vpn on virtual interfaces (sub-interfaces), they end being interconnected/meshed together through the L2vpn construct.

The termination is not meant to be the VTEP, but the individual interface or VLAN the EVPN "pops out" of. I highly doubt your L2VPN dumps out at a Virtual Interface, it is more likely it drops out at a real physical interface.

Would be glad to be wrong if you want to share the config of your device.

Read any documentation on L2vpn, EPL, MPLS x-connect for JunOS or IOS-XR platforms and you terminate services un units or sub-interfaces where the VLAN (id) only has local meaning (only to the interface).

So, you can't model around the VLAN in the chassis, because you would connect resources together while they aren't in real life.

A good example is a customer who have a hub/spoke topology: you terminate many l2 circuits from multiple locations to a single interface. You locally map each remote location to a different VLAN id which is only meaningful for this particular interface. You could reuse the same VLAN (id) on any other interface of the same chassis.

[jsenecal]

We provision most our L2vpn on virtual interfaces (sub-interfaces), they end being interconnected/meshed together through the L2vpn construct.

The termination is not meant to be the VTEP, but the individual interface or VLAN the EVPN "pops out" of. I highly doubt your L2VPN dumps out at a Virtual Interface, it is more likely it drops out at a real physical interface.
Would be glad to be wrong if you want to share the config of your device.

Read any documentation on L2vpn, EPL, MPLS x-connect for JunOS or IOS-XR platforms and you terminate services un units or sub-interfaces where the VLAN (id) only has local meaning (only to the interface).

So, you can't model around the VLAN in the chassis, because you would connect resources together while they aren't in real life.

A good example is a customer who have a hub/spoke topology: you terminate many l2 circuits from multiple locations to a single interface. You locally map each remote location to a different VLAN id which is only meaningful for this particular interface. You could reuse the same VLAN (id) on any other interface of the same chassis.

All these are good examples of other ways to improve netbox but totally out of scope of this FR. @SteveRodrigue I believe you should open another FR with your own implementation details / use case for this.

[jeremystretch]

We're revisiting the topic of virtual circuits as a potential candidate for NetBox v4.2. After reviewing the discussion in #10558 and this FR, here's what I've been able to come up with so far.

First, we should tackle FR #9604, which proposes the ability to attach a CircuitTermination to a Location. Although not directly relevant to this FR, I believe that this can be implemented by replacing the current site and provider_network foreign keys on the CircuitTermination model with a single generic foreign key which could point to any of the three models. This could then be extended under this initiative to alternatively reference a virtual interface.

I believe that enabling a CircuitTermination to "attach" to a virtual interface would allow us to efficiently model point-to-point virtual circuits without compromising our ability to also model the underlying physical circuits. I'll use @jsenecal's topology from #10558 as an example:

We would model this as four discrete circuits in NetBox. The first, at the hub site, would be modeled as a physical circuit between the site and the provider network:

[Interface] <-- [Cable] --> [CircuitTerm] --> [Circuit] <-- [CircuitTerm] --> [ProviderNetwork]

Then, we would define three virtual circuits, one for each of the spoke sites:

[Interface] <-- [Cable] --> [CircuitTerm] --> [Circuit] <-- [CircuitTerm] --> [Interface]
                                                                               (virtual)

I should note that this approach does preclude us from defining a separate underlying physical circuit at a spoke site, but maybe that's an acceptable degree of abstraction. This also intentionally does not broach the topic of multipoint circuits: It simply allows for the virtual hand-off of a point-to-point circuit.

[gblanchard]

When looking at individual virtual circuits, where is the "tie" in point from the virtual to the hub circuit? Ideally we want to be able to do this 1. View all underlying circuits in the NNI/UNI 2. View the parent circuit when looking at the virtual one I can see how this is challenging here, ideally the hub site objects are shared between each virtual circuit and bifurcates from there. [Main Int] <- [SameCable] -> [SameCircuitTerm] -> [SameCircuit] <- [SameCircuitTerm] -> [SameProviderNetwork] [Virtual Int1] <- [SameCable] -> [SameCircuitTerm] -> [SameCircuit] <- [SameCircuitTerm] -> [SameProviderNetwork] -> [UNI Circuit 1] -> [CircuitTerm] etc... [Virtual Int2] <- [SameCable] -> [SameCircuitTerm] -> [SameCircuit] <- [SameCircuitTerm] -> [SameProviderNetwork] -> [UNI Circuit 2] -> [CircuitTerm] etc... [Virtual Int3] <- [SameCable] -> [SameCircuitTerm] -> [SameCircuit] <- [SameCircuitTerm] -> [SameProviderNetwork] -> [UNI Circuit 2]

…

On Thu, Sep 26, 2024 at 10:56 AM Jeremy Stretch ***@***.***> wrote: We're revisiting the topic of virtual circuits as a *potential* candidate for NetBox v4.2. After reviewing the discussion in #10558 <#10558> and this FR, here's what I've been able to come up with so far. First, we should tackle FR #9604 <#9604>, which proposes the ability to attach a CircuitTermination to a Location. Although not directly relevant to this FR, I believe that this can be implemented by replacing the current site and provider_network foreign keys on the CircuitTermination model with a single generic foreign key which could point to any of the three models. This could then be extended under this initiative to alternatively reference a virtual interface. I believe that enabling a CircuitTermination to "attach" to a virtual interface would allow us to efficiently model point-to-point virtual circuits without compromising our ability to also model the underlying physical circuits. I'll use @jsenecal </~https://github.com/jsenecal>'s topology from #10558 <#10558> as an example: [image: Virtual circuit topology] <https://user-images.githubusercontent.com/1564902/193906217-3b03fa53-31b3-4540-ad99-6249f25de7a1.png> We would model this as four discrete circuits in NetBox. The first, at the hub site, would be modeled as a physical circuit between the site and the provider network: [Interface] <-- [Cable] --> [CircuitTerm] --> [Circuit] <-- [CircuitTerm] --> [ProviderNetwork] Then, we would define three virtual circuits, one for each of the spoke sites: [Interface] <-- [Cable] --> [CircuitTerm] --> [Circuit] <-- [CircuitTerm] --> [Interface] (virtual) I should note that this approach does preclude us from defining a separate underlying physical circuit at a spoke site, but maybe that's an acceptable degree of abstraction. This also intentionally does not broach the topic of multipoint circuits: It simply allows for the virtual hand-off of a point-to-point circuit. — Reply to this email directly, view it on GitHub <#13086 (comment)>, or unsubscribe </~https://github.com/notifications/unsubscribe-auth/AAA4DSAP7QXVFVZQOTUTI33ZYQOAXAVCNFSM6AAAAAAZ55LOKGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNZXGIYTCOBYGA> . You are receiving this because you commented.Message ID: ***@***.***>

[jsenecal]

Thanks @jeremystretch for giving this a look/try. I've got a few questions to make sure we're on the same page:

1. Parent-Child Relationship: Where exactly is the link between the virtual circuits and the "parent" circuit in your proposed setup? I'm not seeing how we'd track that connection.

2. Physical vs Logical: I get using ProviderNetworks (which I'm using right now, for lack of a better way to document these), but for us service providers, it'd be super helpful to keep the physical (where to/from) and logical parts of the topology separate (Many UNI/NNI can have some sort of full mesh, multiple vlans in mutliple directions on the same physical circuit...). Any thoughts on how we could do that?

3. Virtual Interface Ties: In my current setup, I can't define the virtual interfaces linked to a child circuit. How would your proposal handle tying the virtual interfaces of the UNI/NNI to their respective circuits? I think you elaborated a way, but this goes against the point above (2)

4. End-to-End Traces: How can we make sure this setup lets us trace a customer's virtual circuit all the way back to the physical hardware?

5. Idea: What if we added a "parent_circuit" field to CircuitTermination, or Circuit, and allow virtual interfaces in Circuit Terminations? That way we could directly link child circuits to their physical parent, while still using ProviderNetwork for the big picture. This would allow very detailed topologies (see 2)

I think nailing down these points will help us build something that really fits what service providers need. Thoughts?

[BrunoBlanes]

I don't know if anyone have elaborated on this yet, but I have a use case scenario where I provide an L2 circuit to a partner using LAG interfaces for bandwidth and right now I have no adequate way to document this in Netbox since LAG interfaces cannot be used as circuit terminators.

[jsenecal]

I don't know if anyone have elaborated on this yet, but I have a use case scenario where I provide an L2 circuit to a partner using LAG interfaces for bandwidth and right now I have no adequate way to document this in Netbox since LAG interfaces cannot be used as circuit terminators.

Technically, Circuits are meant to be things that you purchase from third parties, not that you provide them. They are meant to represent a point-to-point or means to access a service or network, which represents a balck-box (you dont know how its done, but you know where to connect and where it ends...).

Services/Circtuits that you are "Providing" are of a different kind and not in the realm of things that netbox tracks. I have resorted to custom plugins for that.

[BrunoBlanes]

It doesn't matter though, if I sign a contract for an L2 transit and terminate it on a LAG interface I'd still have no way to acurrately map it into Netbox.

[davidgwatkins]

I agree with the need for parent/child relationships between User-Network Interface (UNI) and the specific Virtual Circuit.

In the case of a multiplexed handoff circuit with a child virtual circuits:

Physical Circuit Parent

[Interface] <-- [Cable] --> [CircuitTerm] --> [Circuit] <-- [CircuitTerm] --> [ProviderNetwork]

Virtual Circuit Child

[Interface] <-- [CircuitTerm] --> [Circuit] <-- [CircuitTerm] --> [Interface]
 (virtual)                                                         (virtual)

To address @BrunoBlanes point, in practice, the UNI handoff at one or both ends of a circuit may be a LAG; from the consumer perspective, the circuit order to the provider would include N interfaces of M rate that would form an NxM LAG. It is common practice to order and configure a handoff as a LAG (even with a single member in initial orders), and grow the LAG based on utilization.

For modeling purposes, this could be achieved by having a Virtual Circuit with no Physical Circuit Parent:

Virtual Circuit Parent (or Childless)

[Interface] <-- [CircuitTerm] --> [Circuit] <-- [CircuitTerm] --> [ProviderNetwork]
 (virtual)

Virtual Circuit Child

[Interface] <-- [CircuitTerm] --> [Circuit] <-- [CircuitTerm] --> [Interface]
 (virtual)                                                         (virtual)

Virtual circuits should terminate to virtual interfaces, and physical circuits should terminate to physical interfaces. The following should be true:

• Virtual Circuits must support a Parent-Child relationship with both Physical Circuits and Virtual Circuits
• Virtual Circuits may have one Virtual Circuit Parent or a Physical Circuit Parent
• Physical Circuits may have many Virtual Circuit Children
• Virtual Circuits may have many Virtual Circuit Children
• Virtual Circuits may not have any Physical Circuit Children
• Virtual Circuit CircuitTerms may only terminate to virtual interfaces or provider networks

[ajackson79]

...
What you want is the ability to define Services components and track those items. This can be achieved in a plugin (this is what I did).

I would be interested in knowing about this plugin for documenting service components. Is this something available publicly or did you create something for your own use case? This was one of my requests for the 4.2 SP working group, but if something already exists as a plugin that would be great.

[ajackson79]

For the virtual circuit topic at hand one thing I thought we could consider is creating a new connection type that would be called "Type 2 Network Provider". This would essentially be a cloud. This cloud could have ENNI and UNI ports. We would then have all the physical devices/ports connect to that cloud with cables. This would be the physical tie between all the devices/ports connected to this Type 2 Network Provider. You then have virtual circuits connect to EFPs or sub-interfaces on each port. Below is a quick and basic drawing.

With the current multiconnection cable you can represent this today in Netbox, but it looks like everything is connected with a cable. This is why I suggested making a new connection type so we can distinguish between a physical cable and a physical Type 2 network. Below is what you can do now today with the mulitconnection cable in Netbox. If we can just replace the cable 84 connections with a "Type 2 Network Provider" connection (either ENNI or UNI) we would get the functionality we want from a physical perspective.

You can even represent a dual ENNI today with the multiconnection Cable. So again that would be nice to represent but replacing the cable with a Type 2 network provider, Termination A with ENNI Interfaces and Termination B with UNI Interfaces.

Next you just need to have sub-interfaces (efps) as children to the parent physical interfaces and you create virtual circuits between those sub-interfaces showing your actual circuits with the type 2 network provider. You might have 2+ EFPs that use the same VC so you would need to make sure you can map multiple sub-interfaces to each side of the VC. You might also have 2+ VCs at a given UNI.

If we get the QinQ and Vlan Translation asks in other FRs we should be able to then model these circuits and all their Vlan encapsulations, rewrites, and translations at each point in the network.

The only thing you really need to track on the VC that isn't tracked on a circuit today is what Vlans are agreed on at the ENNI and UNI and if the ENNI is using S-Vlan (.1q or .1ad) or 1-to-1 mapping with the UNI. If the ENNI is using S-Vlan the agreed upon Vlan(s) could be both inner and outer Vlans or just outer Vlan(s)

[jsenecal]

...
What you want is the ability to define Services components and track those items. This can be achieved in a plugin (this is what I did).

I would be interested in knowing about this plugin for documenting service components. Is this something available publicly or did you create something for your own use case? This was one of my requests for the 4.2 SP working group, but if something already exists as a plugin that would be great.

I have a custom built plugin (very alpha quality, aka work in progress) to achieve this in our organisation's Gitlab. It's purpose is to track customer services components and the services themselves, as long as tracking their dependancy on third party contracts. My org doesnt see me opensourcing it as a problem but it also brings the problem that I would now have to maintain it for others.

Also very unrelated to this FR 😄

[rcon-git]

I know a lot of conversations about this have been had regarding service provider networks, but would virtual circuits also serve as a use case for SD-WAN?

Right now, we model our SD-WAN circuits as physical circuits, but we aren't able to associate them to the same physical interface as the ISP circuit.
Would this allow us to have an ISP circuit associated with a physical interface, then associate a 'virtual' circuit onto the same physical interface as the ISP circuit?
Even better if we can model the SD-WAN virtual circuit on multiple physical interfaces of different equipment. For example, our site might have (2) separate ISP circuits each terminating on a different piece of equipment, but we only have (1) SD-WAN circuit because that SD-WAN virtual circuit rides on top of both ISP circuits.