CBF - Build RPMs automatically

Motivation

CBF - Community Build Framework - is an amazing tool for being able to maintain several projects, compile pentaho and even doing remote deploys using rsync.

However, there are cases where rsync is not the best way to maintain a server installation. At Mozilla, IT uses Puppet to maintain everything related to automated server maintenance. The way CBF is structured allows us to define rules to automate solution checkouts, but we need to automate the installation of 3 extra bits:

• Pentaho webapp
• Pentaho solution (the system/ and admin/ directories)
• Administration / Enterprise console

Approach

The cluster uses RHEL6 distribution, so we chose to pack the distributions using rpm format. This should work for any rpm-based distribution. If someone wants to do the same for deb or something else, just get in touch or simply fork CBF on github and send a pull request.

Building RPMs through CBF

CBF now has 2 extra options (be sure to grab the latest version from github):

 create-rpm-dist - Creates a tgz file with pentaho, pentaho-style and ROOT folders - to be used with a puppet script
 create-rpms - Creates rpms based on the specs found on the config/rpm folder for the solution

 In order for this to be usable, we need to create a directory called rpm under config, and place there the .spec files for the rpm's we're creating. We will also need a specific makefile.

This is the structure:

project-metrics
├── config
│   ├── build-ee-pedro.properties
│   ├── build-pedro.properties
│   ├── build.properties
│   └── rpm
│       ├── makefile
│       ├── webdetails-pentaho-adminconsole.spec
│       ├── webdetails-pentaho-adminconsole-stage.spec
│       ├── webdetails-pentaho-solution.spec
│       ├── webdetails-pentaho-solution-stage.spec
│       ├── webdetails-pentaho-webapp.spec
│       └── webdetails-pentaho-webapp-stage.spec
├── etl
├── patches
│   ├── project-metrics
│   ├── target-build
│   └── target-dist
├── patches-ee
│   ├── enterprise-console
│   ├── server
│   └── solution
└── solution -> ../project-metrics-solution

 You'll see that I'm building here 6 RPMs, for a production and stage environment.

You can download the entire set of files from here.

Here's a sample of the file webdetails-pentaho-solution.spec:

Name:  Webdetails-Pentaho-Webapp
Version: @VERSION@
Release: 1%{?dist}
Summary: Pentaho webapp customized for Mozilla metrics
Group:  Applications/Engineering
License: MPL-1.0
Source:     webdetails-pentaho.tgz


%description

%prep
rm -rf %{buildroot}
%build
tar zxvf ../SOURCES/webdetails-pentaho.tgz 

%install
mkdir %{buildroot}
make DESTDIR=%{buildroot} install

%clean
rm -rf %{buildroot}

%files
%defattr(-,root,root,-)
%doc

/opt/pentaho/pentaho-server/server/webapps/pentaho
/opt/pentaho/pentaho-server/server/webapps/pentaho-style
/opt/pentaho/pentaho-server/server/webapps/ROOT

%changelog

And here's the makefile that is called by the specs. I chose to install the pentaho server under /opt/pentaho/, but this can be changed to any desired path:

all: install

install:
 mkdir $(DESTDIR)/opt
 mkdir $(DESTDIR)/opt/pentaho
 mkdir $(DESTDIR)/opt/pentaho/pentaho-server
 mkdir $(DESTDIR)/opt/pentaho/pentaho-server/server/
 mkdir $(DESTDIR)/opt/pentaho/pentaho-server/server/webapps
 cp -r pentaho $(DESTDIR)/opt/pentaho/pentaho-server/server/webapps/pentaho
 cp -r pentaho-style $(DESTDIR)/opt/pentaho/pentaho-server/server/webapps/pentaho-style    
 cp -r ROOT $(DESTDIR)/opt/pentaho/pentaho-server/server/webapps/ROOT    

install-solution:
 mkdir $(DESTDIR)/opt
 mkdir $(DESTDIR)/opt/pentaho
 mkdir $(DESTDIR)/opt/pentaho/pentaho-solution
 cp *.* $(DESTDIR)/opt/pentaho/pentaho-solution/
 cp -r system $(DESTDIR)/opt/pentaho/pentaho-solution/system
 cp -r admin $(DESTDIR)/opt/pentaho/pentaho-solution/admin


install-stage:
 mkdir $(DESTDIR)/opt
 mkdir $(DESTDIR)/opt/pentaho
 mkdir $(DESTDIR)/opt/pentaho/pentaho-server-stage
 mkdir $(DESTDIR)/opt/pentaho/pentaho-server-stage/server  
 mkdir $(DESTDIR)/opt/pentaho/pentaho-server-stage/server/webapps
 cp -r pentaho $(DESTDIR)/opt/pentaho/pentaho-server-stage/server/webapps/pentaho
 cp -r pentaho-style $(DESTDIR)/opt/pentaho/pentaho-server-stage/server/webapps/pentaho-style    
 cp -r ROOT $(DESTDIR)/opt/pentaho/pentaho-server-stage/server/webapps/ROOT    

install-solution-stage:
 mkdir $(DESTDIR)/opt
 mkdir $(DESTDIR)/opt/pentaho
 mkdir $(DESTDIR)/opt/pentaho/pentaho-solution-stage
 cp *.* $(DESTDIR)/opt/pentaho/pentaho-solution-stage/ 
 cp -r system $(DESTDIR)/opt/pentaho/pentaho-solution-stage/system
 cp -r admin $(DESTDIR)/opt/pentaho/pentaho-solution-stage/admin

Dependencies 

Note that this is depends on having rpmbuild installed on your system. Would work on linux and mac, and even eventually on windows with cygwin.

After running the create-rpms ant target, you should get the RPMs in the dist/rpm/RPMS/ directory.

Next steps

You probably know what you want to do with the RPMs. You can either install them directly, or if you're using puppet you may want to set up a yum repository. Lots of tutorials on how to do this, here's one.

Packaging Enterprise Edition

By default, this targets know how to handle the CE version. We can change a few options and tell it to pack the Enterprise Edition, just by setting some extra options pointing to a different location where we unpacked the EE version:

######################################
## EE BASED RPM BUILD
######################################

rpm.source.webapp = /home/pedro/tex/pentaho/ee/4.8/biserver-ee/tomcat/webapps/
rpm.source.solution = /home/pedro/tex/pentaho/ee/4.8/biserver-ee/pentaho-solutions/
rpm.source.administration-console = /home/pedro/tex/pentaho/ee/4.8/enterprise-console