Orb Best Practices For Enterprise Customers

Ecosystem Orbs
1

It’s my understanding that on-premise deployments of Circle CI enterprise customers will include Circle CI CERTIFIED and PARTNER Orbs, along with the ability for enterprise customers to publish their own Orbs to their on-premise registry instance.

The organization I belong to is evaluating Circle CI Enterprise as an option for an on-premise CI solution.

During our early meeting with Circle CI we discussed how best to work with Orbs so that we could quickly, and easily, migrate projects using an on-premise Circle CI instance, to CircleCI_com.

For a little background, our organization is looking to shift a large quantity of source code repositories to GitHub_com from our internal SCM platform. Some repositories will be kept private, while others will be made public under an Open Source license.

If we adopt Circle CI Enterprise as our on-premise CI solution, our internal source code repositories would be restricted to Orbs published to the on-premise Orb registry.

If an internal repository is migrated to GitHub_com and CircleCI_com, Orb references would need to be updated.

For CERTIFIED and PARTNER Orbs this may not be an issue at all, assuming those Orbs are kept in sync with the CircleCI_com registry.

For Orbs developed by our company, the issue is that we’ll need to develop a process that publishes our Orbs to both the public Circle CI Orb registry, and an on-premise registry. This process can be automated. It’ll just require more overhead to maintain, especially if we want to keep the source code for our Orbs publicly available on GitHub_com.

Lastly, there’s the question of how third-party Orbs published on the public Circle CI Orb registry can be made available to an on-premise Circle CI instance (or even, whether they should be, given the security implications).

If they are not made available, our organization may find itself forking publicly available Orbs and re-publishing them internally. If we are able to publish the Orb internally using the same namespace, then projects that migrate to GitHub_com would not need to update those Orbs, or Orb versions. Otherwise, it would require switching the Orb from an internal one to an external one.

The point of this thread is to explore the difficulties we may encounter for projects that first on-board with an on-premise instance of Circle CI, before then migrating to GitHub_com and CircleCI_com. Furthermore, it’s my hope to use this thread to explore practices that could minimize the difficulties in migrating projects from an on-premise Circle CI instance to CircleCI_com.

2 Likes
2

I think exactly what Orbs/registries will look like for hosted CircleCI customers is still being worked out—@ndintenfass or @Marty may have more thoughts—but I imagine that some of the following will be true:

  • Hosted CircleCI customers will have access to the public CircleCI Orb Registry
  • Hosted CircleCI administrators may be able to restrict access to the public registry if desired

My guess is that you’ll find the above aspect of the process to be fairly simple. The CLI commands to publish the orbs will likely be close-to-identical, just with different registry targets. Or they might even be the exact same commands, but will read an environment variable or parameter something like that to tell the CircleCI CLI to use a different registry target. (I’m just thinking out loud here and am not a Product Manager, so bear with me…)

We’ll try to keep folks updated re: Orbs + hosted CircleCI as these details start to firm up!

1 Like
3

All great questions. In general, our approach is likely to be that each install’s registry stands alone from the perspective of the build system. We will then provide some amount of tooling and/or scripts for doing importing and syncing across orb registries. There are a host of complexities we need to solve as part of getting orbs to our server installations of CircleCI, not the least of which is that if you are running an “older” version of the config processing code in your server install, and you pull a “new” orb from the public registry that uses some new feature (eg: if we add a new parameter type) that your older install doesn’t support that will not be a happy path. How to mitigate that risk is tricky, especially with Certified orbs that we want to be able to represent as stable, safe interfaces. One option is we get more granular with config versioning, but that has various complications for everyone. Another option is we put in various flavors of error catching that anticipate getting orbs with unknown features (the problem, of course, is that the difference between a typo and a feature not supported is not clear for a given version of CircleCI). Another option is ship a known-good set of orbs with each server version and don’t encourage lots of cross-registry sharing, so your orbs are mostly either ones we have tested on that version or that you have published to that version or earlier. Publishing to multiple registries is not, in theory, particularly complicated. That said, once there are multiple registries we’ll need to expand the parameters in the orb-tools publishing helpers.

2 Likes
4

I’m really glad to hear y’all are considering stability as a core requirement for re-using Orbs across registries.

In this scenario, I hope bug fixes to a CERTIFIED orb could still be back-ported and uploaded to an on-premise deployment independent of the upgrade cycle for the Circle CI instance itself.

That may be alright given the high degree of consistency we try to achieve in CI workflows across our organization. Therefore, it’s unlikely we would be burdened by regular requests for uploading public Orbs to our internal registry.