macOS Image Users: Homebrew Brownout 2021-04-26

The Problem

On Monday, April 26th, packages/formulas installed via Homebrew will either fail or take a long time to manually compile on CircleCI’s macOS images. These are the images you specify in your config using the xcode key. This issue will occur for a span of 6 hours.

When

The brownout will occur: April 26, 2021 13:00 - 19:00 UTC

Why

This brownout is in preparation for Homebrew’s hosting provider, Bintray, shutting down for good. Bintray, which hosts Homebrew’s bottles (binary packages), is being retired and is being replaced by GitHub Packages hosting. To point to the new host requires an update to Homebrew itself - at least v3.1. CircleCI’s macOS images don’t contain a Homebrew version this new as of today.

Workaround for Monday

During the 6-hour brownout, customers have two options:

  1. Run brew update in your CI build before using any other brew commands. This can take around 5-8 minutes to run, not ideal in a CI environment but this will prevent your builds from failing.
  2. Wait it out. Homebrew/Bintray will return to normal after the 6-hour brownout.

Future Solution

This brownout is a lead up to Bintray support stopping completely on May 1st. We are already working to get our entire macOS fleet updated before then to prevent any lasting impact from the shutdown of Bintray.

Resources

Questions?

Please ask them below. Thanks.

1 Like

Update: May 4th

We’re passed May 1st and the Bintray shutdown. As we know, CircleCI’s macOS images contain an older version of Homebrew, which is affected by the Bintray shutdown. We’re addressing the issue in two ways.

A Patch

To address the situation immediately, a temporary fix to automatically update Homebrew has been put in place for existing macOS images to prevent failed pipelines due to this issue. Some users, depending on the age of the image, may notice slower startup times. This is temporary as we continue to roll out the permanent fix.

Permanent Fix

The macOS Team is working diligently to get our entire macOS fleet updated, prioritizing by image usage. These will ship updated versions of Homebrew.

1 Like