Friday, March 1, 2013

Knife Bootstrap Failure - no such file to load -- rubygems/format (LoadError)

I just experienced an issue when trying to bootstrap a new Chef node in EC2 using the "knife ec2 server create" command. Everything seemed to be proceeding normally, until the actual node attempted to run Chef to register with the server; instead I was greeted with the following error:

/usr/lib/ruby/site_ruby/1.8/rubygems/core_ext/kernel_require.rb:45:in `gem_original_require': no such file to load -- rubygems/format (LoadError)
ec2-54-228-10-198.eu-west-1.compute.amazonaws.com from /usr/lib/ruby/site_ruby/1.8/rubygems/core_ext/kernel_require.rb:45:in `require'
ec2-54-228-10-198.eu-west-1.compute.amazonaws.com from /usr/lib64/ruby/gems/1.8/gems/chef-10.24.0/bin/../lib/chef/provider/package/rubygems.rb:34
ec2-54-228-10-198.eu-west-1.compute.amazonaws.com from /usr/lib/ruby/site_ruby/1.8/rubygems/core_ext/kernel_require.rb:45:in `gem_original_require'
ec2-54-228-10-198.eu-west-1.compute.amazonaws.com from /usr/lib/ruby/site_ruby/1.8/rubygems/core_ext/kernel_require.rb:45:in `require'
ec2-54-228-10-198.eu-west-1.compute.amazonaws.com from /usr/lib64/ruby/gems/1.8/gems/chef-10.24.0/bin/../lib/chef/providers.rb:60
ec2-54-228-10-198.eu-west-1.compute.amazonaws.com from /usr/lib/ruby/site_ruby/1.8/rubygems/core_ext/kernel_require.rb:45:in `gem_original_require'
ec2-54-228-10-198.eu-west-1.compute.amazonaws.com from /usr/lib/ruby/site_ruby/1.8/rubygems/core_ext/kernel_require.rb:45:in `require'
ec2-54-228-10-198.eu-west-1.compute.amazonaws.com from /usr/lib64/ruby/gems/1.8/gems/chef-10.24.0/bin/../lib/chef.rb:25
ec2-54-228-10-198.eu-west-1.compute.amazonaws.com from /usr/lib/ruby/site_ruby/1.8/rubygems/core_ext/kernel_require.rb:45:in `gem_original_require'
ec2-54-228-10-198.eu-west-1.compute.amazonaws.com from /usr/lib/ruby/site_ruby/1.8/rubygems/core_ext/kernel_require.rb:45:in `require'
ec2-54-228-10-198.eu-west-1.compute.amazonaws.com from /usr/lib64/ruby/gems/1.8/gems/chef-10.24.0/bin/chef-client:23
ec2-54-228-10-198.eu-west-1.compute.amazonaws.com from /usr/bin/chef-client:23:in `load'
ec2-54-228-10-198.eu-west-1.compute.amazonaws.com from /usr/bin/chef-client:23

I tried modifying my bootstrap template (thanks to Steven Danna, of Opscode for providing a CentOS 6 template!) to force the installation of the format gem, but the problem persisted. A quick search online brought up a topic of discussion on the Chef mailing list that seemed to blame the latest release of RubyGems (2.0). The suggested workaround was to downgrade the RubyGems install to the previous release; 1.8.25. To ensure that I would be able to bootstrap machines without a manual intervention, I opened up the relevant template (residing in ~/.rvm/gems/ruby-1.9.3-p385/gems/chef-10.24.0/lib/chef/knife/bootstrap/centos6-gems.erb) and changed line 15 from:

gem update --system

To the following:

gem update --system 1.8.25

This allowed my EC2 instantiation to complete successfully! I'm sure Opscode will release an update to fix this issue shortly, but hopefully this will help a few people in the meantime. Now to debug some outstanding issues with my cookbooks...