更新日: 2014年7月31日
Vagrant & Chef

Chef CookbookをServerspecテスト駆動で作成（Elasticsearchインストール）

serverspec の使い方を大体覚えたので、練習を兼ねて elasticsearch をインストールする Chef Cookbook をテスト駆動スタイルで作成してみました。基本的に以下ページの作業をなぞる手順となります。

CentOS6にElasticsearchをインストールしMySQLからデータをインポート | EasyRamble

serverspec については、以下の Resource Types のページが大変役に立ちます。

Serverspec – Resource Types

【お知らせ】英単語を画像イメージで楽に暗記できる辞書サイトを作りました。英語学習中の方は、ぜひご利用ください！

画像付き英語辞書 Imagict | 英単語をイメージで暗記
【開発記録】
英単語を画像イメージで暗記できる英語辞書サービスを作って公開しました

このエントリーは、CentOS サーバー設定用 Chef Cookbook/Recipe の目次の一部です。

elasticsearch 用の serverspec テストコード

以下の内容をチェックするテストコードです。6以降は、MySQL から Elasticsearch のインデックスにデータをインポートするための内容となります。

1. java-1.7.0-openjdk-devel がインストールされていること
2. elasticsearch インストール用のリポジトリ設定
3. elasticsearch がインストールされていること
4. elasticsearch が自動起動のサービスに追加されていて、起動中であること
5. elasticsearch のプラグインである kuromoji がインストールされていること
6. mysql-connector-java がインストールされていること
7. CLASSPATH に /usr/share/java/mysql-connector-java.jar を追加
8. elasticsearch のプラグインである elasticsearch-river-jdbc がインストールされていること

spec/centos.vagrant/app_elasticsearch_spec.rb

require 'spec_helper'

# install java-1.7.0-openjdk-devel
describe package('java-1.7.0-openjdk-devel') do
  it { should be_installed }
end

# elasticsearch repository setting 
describe file('/etc/yum.repos.d/elasticsearch.repo') do
  it { should be_file }
  it { should be_owned_by 'root' }
  it { should be_grouped_into 'root' }
  it { should be_mode 644 }
  its(:content) { should match(/^\[elasticsearch/) }
  its(:content) { should match(/^enabled=1$/) }
end

# install elasticsearch
describe package('elasticsearch') do
  it { should be_installed }
end

# service
describe service('elasticsearch') do
  it { should be_enabled   }
  it { should be_running   }
end

# install kuromoji plugin
describe command('ls /usr/share/elasticsearch/plugins/ | grep analysis-kuromoji') do
  it { should return_stdout 'analysis-kuromoji' }
end

# --------------------------------------------------
# set up to import data from mysql into elasticsearch index
# --------------------------------------------------
# install JDBC driver for MySQL
describe package('mysql-connector-java') do
  it { should be_installed }
end

# set classpath for mysql-connector-java.jar
describe file('/etc/profile.d/mysql-connector-java.sh') do
  it { should be_file }
  regexp = Regexp.escape("export CLASSPATH=$CLASSPATH:/usr/share/java/mysql-connector-java.jar")
  its(:content) { should match(/^#{regexp}$/) }
end

# CLASSPATH include '/usr/share/java/mysql-connector-java.jar'
describe command('echo $CLASSPATH') do
  regexp = Regexp.escape("/usr/share/java/mysql-connector-java.jar")
  its(:stdout) { should match(/#{regexp}/) }
end

# install elasticsearch-river-jdbc plugin
describe command('ls /usr/share/elasticsearch/plugins/ | grep jdbc') do
  it { should return_stdout 'jdbc' }
end

require 'spec_helper'

# install java-1.7.0-openjdk-devel

describe package('java-1.7.0-openjdk-devel') do

it { should be_installed }

end

# elasticsearch repository setting

describe file('/etc/yum.repos.d/elasticsearch.repo') do

it { should be_file }

it { should be_owned_by 'root' }

it { should be_grouped_into 'root' }

it { should be_mode 644 }

its(:content) { should match(/^\[elasticsearch/) }

its(:content) { should match(/^enabled=1$/) }

end

# install elasticsearch

describe package('elasticsearch') do

it { should be_installed }

end

# service

describe service('elasticsearch') do

it { should be_enabled }

it { should be_running }

end

# install kuromoji plugin

describe command('ls /usr/share/elasticsearch/plugins/ | grep analysis-kuromoji') do

it { should return_stdout 'analysis-kuromoji' }

end

# --------------------------------------------------

# set up to import data from mysql into elasticsearch index

# --------------------------------------------------

# install JDBC driver for MySQL

describe package('mysql-connector-java') do

it { should be_installed }

end

# set classpath for mysql-connector-java.jar

describe file('/etc/profile.d/mysql-connector-java.sh') do

it { should be_file }

regexp = Regexp.escape("export CLASSPATH=$CLASSPATH:/usr/share/java/mysql-connector-java.jar")

its(:content) { should match(/^#{regexp}$/) }

end

# CLASSPATH include '/usr/share/java/mysql-connector-java.jar'

describe command('echo $CLASSPATH') do

regexp = Regexp.escape("/usr/share/java/mysql-connector-java.jar")

its(:stdout) { should match(/#{regexp}/) }

end

# install elasticsearch-river-jdbc plugin

describe command('ls /usr/share/elasticsearch/plugins/ | grep jdbc') do

it { should return_stdout 'jdbc' }

end

テストを実行。


$ bundle exec rspec spec/centos.vagrant/app_elasticsearch_spec.rb
FFFFFFFFFFFFFFFF

Failures:
...

Finished in 5.49 seconds
16 examples, 16 failures

Failed examples:
...

$ bundle exec rspec spec/centos.vagrant/app_elasticsearch_spec.rb

FFFFFFFFFFFFFFFF

Failures:

...

Finished in 5.49 seconds

16 examples, 16 failures

Failed examples:

...

予定通り、テストが全部失敗してます。

elasticsearch インストール用の Chef Cookbook を作成

続いて、上述のテストを成功させるための elasticsearch インストール用の Chef Cookbook を作成していきます。


$ bundle exec knife cookbook create -o site-cookbooks app_elasticsearch_cookbook

$ bundle exec knife cookbook create -o site-cookbooks app_elasticsearch_cookbook

Recipe 作成。

site-cookbooks/app_elasticsearch_cookbook/recipes/default.rb

# Cookbook Name:: app_elasticsearch_cookbook
# Recipe:: default

# install jdk (java)
package "java-1.7.0-openjdk-devel" do
  action :install
end

# add elasticsearch repository
bash 'add_elasticsearch_repo' do
  user 'root'
  code <<-EOC
    rpm --import http://packages.elasticsearch.org/GPG-KEY-elasticsearch
  EOC
  creates "/etc/yum.repos.d/elasticsearch.repo"
end

# elasticsearch repository file
cookbook_file "/etc/yum.repos.d/elasticsearch.repo" do
  owner "root"
  group "root"
  mode 0644
end

# install elasticsearch
package "elasticsearch" do
  action :install
end

service "elasticsearch" do
  supports :status => true, :restart => true, :reload => true
  action [:enable, :start]
end

# install kuromoji plugin
bash 'install_kuromoji_plugin' do
  code <<-EOC
    /usr/share/elasticsearch/bin/plugin -install elasticsearch/elasticsearch-analysis-kuromoji/2.1.0
  EOC
  not_if "ls /usr/share/elasticsearch/plugins/ | grep analysis-kuromoji"
end

# --------------------------------------------------
# set up to import data from mysql into elasticsearch index
# --------------------------------------------------
# JDBC driver for MySQL
package "mysql-connector-java" do
  action :install
end

# set classpath for mysql-connector-java.jar
cookbook_file "/etc/profile.d/mysql-connector-java.sh" do
  owner "root"
  group "root"
  mode 0644
  notifies :restart, 'service[elasticsearch]'
end

bash "reflesh_classpath" do
  code <<-EOC
    source /etc/profile.d/mysql-connector-java.sh
  EOC
  not_if "echo $CLASSPATH | grep /usr/share/java/mysql-connector-java.jar"
end

# install elasticsearch-river-jdbc plugin
bash 'install_elasticsearch-river-jdbc_plugin' do
  code <<-EOC
    /usr/share/elasticsearch/bin/plugin --install jdbc --url http://xbib.org/repository/org/xbib/elasticsearch/plugin/elasticsearch-river-jdbc/1.3.0.0/elasticsearch-river-jdbc-1.3.0.0-plugin.zip
    ln -s /usr/share/java/mysql-connector-java.jar /usr/share/elasticsearch/plugins/jdbc/
    service elasticsearch restart
  EOC
  not_if "ls /usr/share/elasticsearch/plugins/ | grep jdbc"
end

# Cookbook Name:: app_elasticsearch_cookbook

# Recipe:: default

# install jdk (java)

package "java-1.7.0-openjdk-devel" do

action :install

end

# add elasticsearch repository

bash 'add_elasticsearch_repo' do

user 'root'

code <<-EOC

rpm --import http://packages.elasticsearch.org/GPG-KEY-elasticsearch

EOC

creates "/etc/yum.repos.d/elasticsearch.repo"

end

# elasticsearch repository file

cookbook_file "/etc/yum.repos.d/elasticsearch.repo" do

owner "root"

group "root"

mode 0644

end

# install elasticsearch

package "elasticsearch" do

action :install

end

service "elasticsearch" do

supports :status => true, :restart => true, :reload => true

action [:enable, :start]

end

# install kuromoji plugin

bash 'install_kuromoji_plugin' do

code <<-EOC

/usr/share/elasticsearch/bin/plugin -install elasticsearch/elasticsearch-analysis-kuromoji/2.1.0

EOC

not_if "ls /usr/share/elasticsearch/plugins/ | grep analysis-kuromoji"

end

# --------------------------------------------------

# set up to import data from mysql into elasticsearch index

# --------------------------------------------------

# JDBC driver for MySQL

package "mysql-connector-java" do

action :install

end

# set classpath for mysql-connector-java.jar

cookbook_file "/etc/profile.d/mysql-connector-java.sh" do

owner "root"

group "root"

mode 0644

notifies :restart, 'service[elasticsearch]'

end

bash "reflesh_classpath" do

code <<-EOC

source /etc/profile.d/mysql-connector-java.sh

EOC

not_if "echo $CLASSPATH | grep /usr/share/java/mysql-connector-java.jar"

end

# install elasticsearch-river-jdbc plugin

bash 'install_elasticsearch-river-jdbc_plugin' do

code <<-EOC

/usr/share/elasticsearch/bin/plugin --install jdbc --url http://xbib.org/repository/org/xbib/elasticsearch/plugin/elasticsearch-river-jdbc/1.3.0.0/elasticsearch-river-jdbc-1.3.0.0-plugin.zip

ln -s /usr/share/java/mysql-connector-java.jar /usr/share/elasticsearch/plugins/jdbc/

service elasticsearch restart

EOC

not_if "ls /usr/share/elasticsearch/plugins/ | grep jdbc"

end

メインの Recipe は以上です。以降は、Recipe 中で利用しているファイル。

elasticsearch インストール用のリポジトリ設定ファイル。

site-cookbooks/app_elasticsearch_cookbook/files/elasticsearch.repo


[elasticsearch-1.3]
name=Elasticsearch repository for 1.3.x packages
baseurl=http://packages.elasticsearch.org/elasticsearch/1.3/centos
gpgcheck=1
gpgkey=http://packages.elasticsearch.org/GPG-KEY-elasticsearch
enabled=1

[elasticsearch-1.3]

name=Elasticsearch repository for 1.3.x packages

baseurl=http://packages.elasticsearch.org/elasticsearch/1.3/centos

gpgcheck=1

gpgkey=http://packages.elasticsearch.org/GPG-KEY-elasticsearch

enabled=1

続いて、mysql-connector-java.jar のパスを CLASSPATH に追加するためのファイル。

site-cookbooks/app_elasticsearch_cookbook/files/mysql-connector-java.sh


export CLASSPATH=$CLASSPATH:/usr/share/java/mysql-connector-java.jar

export CLASSPATH=$CLASSPATH:/usr/share/java/mysql-connector-java.jar

以上で Cookbook の作成は終了です。プロビジョニング実行。


$ vagrant provision

$ vagrant provision

プロビジョニング実行後にテストが通ることを確認

プロビジョニング実行後に、もう一回テスト。


$ bundle exec rspec spec/centos.vagrant/app_elasticsearch_spec.rb
................

Finished in 5.34 seconds
16 examples, 0 failures

$ bundle exec rspec spec/centos.vagrant/app_elasticsearch_spec.rb

................

Finished in 5.34 seconds

16 examples, 0 failures

おおお！全部テストが通りました！これは快感。

以下、手動での確認。serverspec のテストに書いてある内容を手動で行っています。


$ vagrant ssh
$ su -
# yum list installed | grep java-1.7.0-openjdk-devel
java-1.7.0-openjdk-devel.x86_64
# cat /etc/yum.repos.d/elasticsearch.repo
[elasticsearch-1.3]
name=Elasticsearch repository for 1.3.x packages
baseurl=http://packages.elasticsearch.org/elasticsearch/1.3/centos
gpgcheck=1
gpgkey=http://packages.elasticsearch.org/GPG-KEY-elasticsearch
enabled=1
# yum list installed | grep elasticsearch
elasticsearch.noarch 1.3.1-1            @elasticsearch-1.3
# chkconfig --list elasticsearch
elasticsearch  	0:off	1:off	2:on	3:on	4:on	5:on	6:off
# service elasticsearch status
elasticsearch (pid  5379) is running...
# ls /usr/share/elasticsearch/plugins/ | grep analysis-kuromoji
analysis-kuromoji
# yum list installed | grep mysql-connector-java
mysql-connector-java.noarch
# cat /etc/profile.d/mysql-connector-java.sh
export CLASSPATH=$CLASSPATH:/usr/share/java/mysql-connector-java.jar
# echo $CLASSPATH
:/usr/share/java/mysql-connector-java.jar
# ls /usr/share/elasticsearch/plugins/ | grep jdbc
jdbc