Chef-solo: setting the recipe needed to create, before postgresql

Question

I am trying to install postgresql on my server using chef-solo. I had this work by making bootstrap myself and manually incorporating the presentation package into the script. Now I want to use a knife solo to handle bootstrapping, and I had a problem determining the execution order of recipes.

I want to install the build-essential package before postgresql. Here is my json file for the server that I buidling

{ "dbuser": "myuser", "dbpass": "mypassword", "postgresql":{"password":{"postgres":"mypassword"}}, "passenger":{"version":"3.0.14", "max_pool_size":"2"}, "default_attributes":{"build_essential":{"compiletime": "true"}}, "run_list": [ "recipe[build-essential]", "role[web_server]", "recipe[sqlite]", "recipe[postgresql::server]", "recipe[pope::default]" ] } 

When I issue a command

 knife cook <user>@<my server ip> 

I get this error

 WARNING: No knife configuration file found Checking cookbook syntax... Checking Chef version Enter the password for test@ <my server ip>: [2012-10-29T16:14:44+11:00] INFO: *** Chef 10.16.2 *** [2012-10-29T16:14:45+11:00] INFO: Setting the run_list to ["recipe[build-essential]", "role[web_server]", "recipe[sqlite]", "recipe[postgresql::server]", "recipe[mysettings::default]"] from JSON [2012-10-29T16:14:45+11:00] INFO: Run List is [recipe[build-essential], role[web_server], recipe[sqlite], recipe[postgresql::server], recipe[pope::default]] [2012-10-29T16:14:45+11:00] INFO: Run List expands to [build-essential, apache2, passenger_apache2, sqlite, postgresql::server, mysettings::default] [2012-10-29T16:14:45+11:00] INFO: Starting Chef Run for test-VirtualBox [2012-10-29T16:14:45+11:00] INFO: Running start handlers [2012-10-29T16:14:45+11:00] INFO: Start handlers complete. [2012-10-29T16:14:46+11:00] INFO: Processing package[postgresql-client] action install (postgresql::client line 37) [2012-10-29T16:14:47+11:00] INFO: Processing package[libpq-dev] action install (postgresql::client line 37) [2012-10-29T16:14:47+11:00] INFO: Processing package[make] action install (postgresql::client line 37) [2012-10-29T16:14:47+11:00] INFO: Processing gem_package[pg] action install (postgresql::client line 42) ================================================================================ Error executing action `install` on resource 'gem_package[pg]' ================================================================================ Mixlib::ShellOut::ShellCommandFailed ------------------------------------ Expected process to exit with [0], but received '1' ---- Begin output of /opt/chef/embedded/bin/gem install pg -q --no-rdoc --no-ri -v "0.14.1" ---- STDOUT: Building native extensions. This could take a while... STDERR: ERROR: Error installing pg: ERROR: Failed to build gem native extension. /opt/chef/embedded/bin/ruby extconf.rb checking for pg_config... yes Using config values from /usr/bin/pg_config checking for libpq-fe.h... yes checking for libpq/libpq-fs.h... yes checking for pg_config_manual.h... yes checking for PQconnectdb() in -lpq... no checking for PQconnectdb() in -llibpq... no checking for PQconnectdb() in -lms/libpq... no Can't find the PostgreSQL client library (libpq) *** extconf.rb failed *** Could not create Makefile due to some reason, probably lack of necessary libraries and/or headers. Check the mkmf.log file for more details. You may need configuration options. Provided configuration options: --with-opt-dir --with-opt-include --without-opt-include=${opt-dir}/include --with-opt-lib --without-opt-lib=${opt-dir}/lib --with-make-prog --without-make-prog --srcdir=. --curdir --ruby=/opt/chef/embedded/bin/ruby --with-pg --without-pg --with-pg-dir --without-pg-dir --with-pg-include --without-pg-include=${pg-dir}/include --with-pg-lib --without-pg-lib=${pg-dir}/lib --with-pg-config --without-pg-config --with-pg_config --without-pg_config --with-pqlib --without-pqlib --with-libpqlib --without-libpqlib --with-ms/libpqlib --without-ms/libpqlib Gem files will remain installed in /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/pg-0.14.1 for inspection. Results logged to /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/pg-0.14.1/ext/gem_make.out ---- End output of /opt/chef/embedded/bin/gem install pg -q --no-rdoc --no-ri -v "0.14.1" ---- Ran /opt/chef/embedded/bin/gem install pg -q --no-rdoc --no-ri -v "0.14.1" returned 1 Cookbook Trace: --------------- /tmp/chef-solo/cookbooks/postgresql/recipes/client.rb:44:in `from_file' /tmp/chef-solo/cookbooks/postgresql/recipes/server.rb:24:in `from_file' Resource Declaration: --------------------- # In /tmp/chef-solo/cookbooks/postgresql/recipes/client.rb 42: gem_package "pg" do 43: action :nothing 44: end.run_action(:install) Compiled Resource: ------------------ # Declared in /tmp/chef-solo/cookbooks/postgresql/recipes/client.rb:42:in `from_file' gem_package("pg") do provider Chef::Provider::Package::Rubygems action [:nothing] retries 0 retry_delay 2 package_name "pg" version "0.14.1" cookbook_name :postgresql recipe_name "client" gem_binary "/opt/chef/embedded/bin/gem" end ================================================================================ Recipe Compile Error in /tmp/chef-solo/cookbooks/postgresql/recipes/server.rb ================================================================================ Mixlib::ShellOut::ShellCommandFailed ------------------------------------ gem_package[pg] (postgresql::client line 42) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1' ---- Begin output of /opt/chef/embedded/bin/gem install pg -q --no-rdoc --no-ri -v "0.14.1" ---- STDOUT: Building native extensions. This could take a while... STDERR: ERROR: Error installing pg: ERROR: Failed to build gem native extension. /opt/chef/embedded/bin/ruby extconf.rb checking for pg_config... yes Using config values from /usr/bin/pg_config checking for libpq-fe.h... yes checking for libpq/libpq-fs.h... yes checking for pg_config_manual.h... yes checking for PQconnectdb() in -lpq... no checking for PQconnectdb() in -llibpq... no checking for PQconnectdb() in -lms/libpq... no Can't find the PostgreSQL client library (libpq) *** extconf.rb failed *** Could not create Makefile due to some reason, probably lack of necessary libraries and/or headers. Check the mkmf.log file for more details. You may need configuration options. Provided configuration options: --with-opt-dir --with-opt-include --without-opt-include=${opt-dir}/include --with-opt-lib --without-opt-lib=${opt-dir}/lib --with-make-prog --without-make-prog --srcdir=. --curdir --ruby=/opt/chef/embedded/bin/ruby --with-pg --without-pg --with-pg-dir --without-pg-dir --with-pg-include --without-pg-include=${pg-dir}/include --with-pg-lib --without-pg-lib=${pg-dir}/lib --with-pg-config --without-pg-config --with-pg_config --without-pg_config --with-pqlib --without-pqlib --with-libpqlib --without-libpqlib --with-ms/libpqlib --without-ms/libpqlib Gem files will remain installed in /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/pg-0.14.1 for inspection. Results logged to /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/pg-0.14.1/ext/gem_make.out ---- End output of /opt/chef/embedded/bin/gem install pg -q --no-rdoc --no-ri -v "0.14.1" ---- Ran /opt/chef/embedded/bin/gem install pg -q --no-rdoc --no-ri -v "0.14.1" returned 1 Cookbook Trace: --------------- /tmp/chef-solo/cookbooks/postgresql/recipes/client.rb:44:in `from_file' /tmp/chef-solo/cookbooks/postgresql/recipes/server.rb:24:in `from_file' Relevant File Content: ---------------------- /tmp/chef-solo/cookbooks/postgresql/recipes/client.rb: 37: package pg_pack do 38: action :nothing 39: end.run_action(:install) 40: end 41: 42: gem_package "pg" do 43: action :nothing 44>> end.run_action(:install) 45: [2012-10-29T16:15:03+11:00] ERROR: Running exception handlers [2012-10-29T16:15:03+11:00] ERROR: Exception handlers complete [2012-10-29T16:15:03+11:00] FATAL: Stacktrace dumped to /tmp/chef-solo/chef-stacktrace.out [2012-10-29T16:15:03+11:00] FATAL: Mixlib::ShellOut::ShellCommandFailed: gem_package[pg] (postgresql::client line 42) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1' ---- Begin output of /opt/chef/embedded/bin/gem install pg -q --no-rdoc --no-ri -v "0.14.1" ---- STDOUT: Building native extensions. This could take a while... STDERR: ERROR: Error installing pg: ERROR: Failed to build gem native extension. /opt/chef/embedded/bin/ruby extconf.rb checking for pg_config... yes Using config values from /usr/bin/pg_config checking for libpq-fe.h... yes checking for libpq/libpq-fs.h... yes checking for pg_config_manual.h... yes checking for PQconnectdb() in -lpq... no checking for PQconnectdb() in -llibpq... no checking for PQconnectdb() in -lms/libpq... no Can't find the PostgreSQL client library (libpq) *** extconf.rb failed *** Could not create Makefile due to some reason, probably lack of necessary libraries and/or headers. Check the mkmf.log file for more details. You may need configuration options. Provided configuration options: --with-opt-dir --with-opt-include --without-opt-include=${opt-dir}/include --with-opt-lib --without-opt-lib=${opt-dir}/lib --with-make-prog --without-make-prog --srcdir=. --curdir --ruby=/opt/chef/embedded/bin/ruby --with-pg --without-pg --with-pg-dir --without-pg-dir --with-pg-include --without-pg-include=${pg-dir}/include --with-pg-lib --without-pg-lib=${pg-dir}/lib --with-pg-config --without-pg-config --with-pg_config --without-pg_config --with-pqlib --without-pqlib --with-libpqlib --without-libpqlib --with-ms/libpqlib --without-ms/libpqlib Gem files will remain installed in /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/pg-0.14.1 for inspection. Results logged to /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/pg-0.14.1/ext/gem_make.out ---- End output of /opt/chef/embedded/bin/gem install pg -q --no-rdoc --no-ri -v "0.14.1" ---- Ran /opt/chef/embedded/bin/gem install pg -q --no-rdoc --no-ri -v "0.14.1" returned 1 

Answer 1

The easiest solution is to update your local postgresql cookbook. The problem you are observing has already been fixed in the latest versions of the cookbook (all versions> = 1.0.0).

Now how did they decide that? First, you will notice that the client’s recipe has been split into client.rb and ruby.rb The former only installs OS packages, the latter is a Ruby gem for use in a subsequent release of Chef. As you noticed, to install this gem, you need to install the components necessary for assembly. This is controlled by the new attribute used by the build-essential cookbook called node['build-essential']['compile_time'] . This ensures that the necessary packages are installed at compile time, that is, directly when the compound recipe is included.

To understand why this is important, you need to understand the “Anatomy of the Chef” a little bit, namely the difference between the compilation time when the source code of the recipes is executed and the collection of resources is created for the subsequent conversion and the next convergence, in which the resources are executed to move the system to the desired state.

Typically, most of the actual changes are made in the convergence phase. Sometimes, however, it is necessary to build something before, at the compilation stage, in order to be able to use it to determine which resources need to be precisely determined, as is the case here. Since we install gem on pg at the beginning, we can use it for an early conversation with the database. decide whether to create a database or not.