Set ruby version for XCode 11.1 not working

Build Environment
, ,
1

I’m trying to switch the Ruby version like documented here: https://circleci.com/docs/2.0/testing-ios/#images-using-macos-1014-mojave--xcode-111-and-earlier
by adding

- run:
    name: Set Ruby Version
    command:  echo "ruby-2.6.4" > ~/.ruby-version

to the config.yml.
But when running bundle install, I always get “Your Ruby version is 2.3.7, but your Gemfile specified 2.6.4”

ruby -v prints ruby 2.3.7p456 (2018-03-28 revision 63024) [universal.x86_64-darwin18]

I also tried sudo gem install bundler -v 2.1.0 but bundler doesn’t seem to be the problem.
bundle env prints (shortened)

Environment

Bundler       2.1.0
  Platforms   ruby, universal-darwin-18
Ruby          2.3.7p456 (2018-03-28 revision 63024) [universal.x86_64-darwin18]
  Full Path   /System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/bin/ruby
  Config Dir  /Library/Ruby/Site
RubyGems      2.5.2.3
  Gem Home    /Library/Ruby/Gems/2.3.0
  Gem Path    /Users/distiller/.gem/ruby/2.3.0:/Library/Ruby/Gems/2.3.0:/System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/gems/2.3.0
  User Home   /Users/distiller
  User Path   /Users/distiller/.gem/ruby/2.3.0
  Bin Dir     /usr/local/bin
Tools         
  Git         2.23.0
  RVM         not installed
  rbenv       not installed
  chruby      0.3.9

Gemfile


source 'https://rubygems.org'

ruby '2.6.4'

RUBY VERSION
   ruby 2.6.4p104

BUNDLED WITH
   2.1.0

Autoswitcher and chruby are supposed to be enabled by default and Ruby 2.6.4 should be available with the XCode 11.1 image. Any advice or ideas why the ruby version is not switched here?