Recently I had the “opportunity” (read: challenge) to setup a continuous integration build for a project with a Flex frontend and a Ruby/Rails backend. The project had been converted from Java-Hibernate-Tomcat to Ruby on Rails. Cucumber was introduced for GUI testing using the FunFX adapter. It was my first experience with Cucumber, so the combination of Cucumber, FunFX, Flex, and Firefox was a little daunting at first. But we got through it and, actually, it turned out to be quite an effective tool in our testing suite.
But the fun really started when I was tasked with building the TeamCity CI server for our project. I scoured the blogs and tech sites for anything about CI builds of a Flex-Ruby/Rails project with Cucumber and FunFX. I found bits and pieces, like how to setup Firefox in a headless environment and how to setup TeamCity in a headless environment. So I filled in the rest around that to get the TeamCity server up and running with a continuous integration build for our project.
So why is a headless environment required? Our TeamCity CI server was a remote server- there is no input device or display device. So all of the setup was done via SSH through a VPN. But more importantly, we needed a headless implementation since FunFX requires browser interaction. FunFX “attaches” itself to a browser (via JSSH) so we needed a windowing emulator. We use the X virtual frame buffer of the X Window System (Xvfb) for this purpose (more about Xvfb at http://en.wikipedia.org/wiki/Xvfb).
After it was all over, I felt I needed to document it, if not just for posterity, but also so others might benefit. Since setting up the environment is as much an effort as setting up TeamCity, I decided to break it into two parts. So we’ll cover the basics of setting up the server in this first part and then in the second part we’ll tackle getting TeamCity set up. I hope you enjoy the trip as much as I did.
We’re using a Debian-based system in this prensentation. We’re going to assume you’re familiar with your operating system and know what most of these commands are. Hopefully you can translate to your operating system, if different.
Basically, this is what we need to do:
- Install Java
- Install Ruby and Rails
- Install Flex SDK
- Install and Configure Firefox
- Test JSSH
- Install FlashPlayer
- Install the SCM Client
- Install TeamCity
- Create the CI project
- Create the build steps
- Run the build
So Let’s Get Started…
Because our Flex-Ruby/Rails project was a bit more complicated than what’s needed, plus it involved other requirements that do not need to be documented here, we’re going to use a small sample project, FC_Convert, an interface to convert Fahrenheit to Celsius. I have uploaded the project to my public github account and we reference it in these procedures.
First create a user account on the TeamCity server for the ‘teamcity’ user. This will create a /home/teamcity directory which we will be using. All downloads are to a local workstation and then copied to this home directory of the ‘teamcity’ user on the TeamCity server. It is assumed you will also have your own account on the TeamCity server with sudo capability that you can use to SSH for all of these tasks.
Install Java
Since this is a Ruby/Rails and Flex project, we do not need the JDK. For our project we are installing the JRE since TeamCity requires either the JRE or a JDK. If your build process requires Java, you’ll need to install the JDK.
Visit the Java website and download either the JRE or the JDK for the operating platform of your TeamCity server. Install the JRE or JDK per the installation instructions. In addition to adding the Java bin directory to the PATH environment variable, don’t forget to set the JAVA_HOME environment variable to point to the Java installation directory- TeamCity requires it.
Install Ruby and Rails
As of this writing, the latest version of FunFX (0.2.2) had a conflict with Ruby 1.9.1. Ruby 1.9.1 now includes FasterCSV but with renamed variables and methods. If you install Ruby 1.9.1 and then run FunFX, you will get the following message:
Please switch to Ruby 1.9's standard CSV library. It's FasterCSV plus support for Ruby 1.9's m17n encoding engine.
FunFX uses the FasterCSV library and requires modification to run with Ruby 1.9.1. Since there are no features of Ruby 1.9.1 that we need for this project, we are installing Ruby 1.8.7 instead.
Installing Ruby and Rails is pretty straightforward. Depending on the platform of the server, Ruby can be installed as a set of pre-compiled executables (Windows) or compiled for the specific platform (Linux, Unix). The distribution files for Ruby installation can be found on the Ruby downloads web page.
And if your server’s operating system has a package installer, i.e., apt-get or yum, it’s even easier to install the latest version of Ruby 1.8:
$ sudo apt-get install ruby-full
Otherwise, download the source tar, unpackage it, and then configure, build, and install Ruby.
$ cd /usr/local/src $ sudo cp /home/teamcity/ruby-1.8.7-p302.tgz . $ sudo tar -xzvf ruby-1.8.7-p302.tgz $ cd ruby-1.8.7-p302 $ sudo ./configure $ sudo make $ sudo make install
After Ruby is installed, you need to install the Ruby Gems package manager. Ruby Gems binaries can be downloaded from the project’s download page. Again, check your server package installer first.
$ sudo apt-get install rubygems
If the package installer is not available, download the RubyGems distribution file from the site, unpackage it, and install the package manager by running setup.rb in the install directory:
$ cd /usr/local/src
$ sudo cp /home/teamcity/rubygems-1.3.7.tgz . $ sudo tar -xzvf rubygems-1.3.7.tgz $ cd rubygems-1.3.7 $ sudo ruby setup.rb
Installing Rails is a snap using the gems package manager.
$ sudo gem install rails
Next you need to install any gems required by your application. Add your required gems to the list. NOTE: FunFX requires version 1.6.2 of the firewatir gem. Install it separately specifying the version.
$ sudo gem sources -a http://gems.github.com $ sudo gem install cucumber rake webrat funfx rspec rspec-rails cucumber-rails test-unit $ sudo gem install –v 1.6.2 firewatir
Install Flex SDK
The Flex SDK and Flex compiler come in several flavors: Flex 3 or Flex 4, and Adobe Flex or Open Source Flex (not quite as feature rich as Adobe Flex). For purposes of this presentation, we’re installing the open source edition of Flex 4 and the Flex compiler. Visit the Adobe Open Source Flex web site Flex 4 downloads page and download the Flex 4 Open Source SDK zip file (we are installing Flex 4.1.0 SDK) as well as the Adobe Add Ons zip file.
Create the flex_4_sdk directory, unpackage the zip files, and set execute permissions on the bin, lib, and frameworks directories. If prompted to replace any files, reply yes.
$ cd /usr/local
$ sudo mkdir flex_4_sdk $ cd flex_4_sdk $ sudo unzip /home/teamcity/flex_sdk_4.1.0.16076_mpl.zip $ sudo unzip /home/teamcity/flex_sdk_4.1.0.16076_add_on.zip $ sudo chmod –R 755 /usr/local/flex_4_sdk/bin $ sudo chmod –R 755 /usr/local/flex_4_sdk/lib $ sudo chmod –R 755 /usr/local/flex_4_sdk/frameworks
Install and Configure Firefox
Now it gets a little more difficult. Firefox must run as a headless application (no display device). We use the X virtual framebuffer of the X Window System (Xvfb) for this purpose (more about Xvfb at http://en.wikipedia.org/wiki/Xvfb).
First, insure Xvfb is installed on your system.
$ which Xvfb
If this command does not return the location of Xvfb, it is not installed. Install it with your system’s package installer.
$ sudo apt-get install Xvfb xinit
Start Xvfb and create a virtual display. This only needs to be done once on the server and after every reboot. Ideally, it could be started in a script that is run whenever the server is restarted.
$ startx – ‘which Xvfb’ :1 -screen 0 1024x768x24 &
And since we will not have a browser window to install the JSSH plugin, we must build Firefox from source as opposed to installing binaries. But prior to building Firefox, you must insure your system has all the necessary development tools required to build an application. Here is a list of the development libraries I needed to load prior to executing a successful build of Firefox from source. I used the apt-get package manager for my Debian-based system.
$ sudo apt-get install libgtk2.0-dev $ sudo apt-get install libdbus-glib-1-dev $ sudo apt-get install libasound2-dev $ sudo apt-get install libIDL-dev $ sudo apt-get install build-essential gcc-3.4 $ sudo apt-get install libxt-dev $ sudo apt-get build-dep mozilla-thunderbird
As of this writing we are installing the latest release of Firefox 3.5, 3.5.11. The Firefox 3.5.11 source code is available from the Mozilla repo at ftp://ftp.mozilla.org/pub/mozilla.org/firefox/releases/3.5.11/source/. Download the firefox-3.5.11-source.tar.bz2 file from the site, copy it to the TeamCity server, and unpackage it.
$ cd /usr/local/src $ sudo mkdir firefox-3.5.11 $ cd firefox-3.5.11
$ sudo cp /home/teamcity/firefox-3.5.11-source.tar.bz2 . $ sudo tar -xjvf firefox-3.5.11.source.tar.bz2
Before building Firefox, we need to add configuration parameters to the mozilla configuration by creating a .mozconfig file if it does not exist. If it does exist, it should be modified to look like this:
$ cd mozilla $ sudo vi .mozconfig . $topsrcdir/browser/config/mozconfig mk_add_options MOZ_CO_PROJECT=browser,xulrunner mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/firefox-jssh ac_add_options --enable-extensions=default,jssh,webservices ac_add_options --enable-webservices
ac_add_options --enable-application=browser
This file is used in configuring the options to be used in building the Firefox application. Set the MOZCONFIG environment variable to point to this file and build Firefox.
$ export MOZCONFIG=/usr/local/src/firefox-3.5.11/mozilla/.mozconfig $ sudo make -f client.mk build
Create a symlink to the firefox binary in /usr/local/bin:
$ sudo ln -s /usr/local/src/firefox-3.5.11/mozilla/firefox-jssh/dist/bin/firefox /usr/local/bin/firefox
Test JSSH
We use FunFX for Flex testing. FunFX is a Watir / SafariWatir / FireWatir extension that lets you talk to an Adobe Flex application straight from Ruby. Firewatir is a Ruby gem that uses the Firefox JSSH connection. We need to first create a Firefox profile for use by the Cucumber FunFX tests so we can modify the default settings. We’ll name it firewatir.
SSH into the TeamCity server as the ‘teamcity’ user and create the firewatir profile.
$ DISPLAY=localhost:1.0 firefox -CreateProfile firewatir
This creates a profile directory in the /home/teamcity/.mozilla/firefox directory for firewatir. The directory name for the profile is a two-part name: the first part is a randomly generated string of 8 letters and numbers; the second is the profile name specified in the CreateProfile command- in our case, firewatir.
A skeleton file named prefs.js is created in this directory. The default configuration settings can be overridden for a particular profile by including them in the prefs.js file in this directory. Normally this file is maintained by Firefox. Typically, configuration changes are made via the about:config URL. But we need to modify this file since we do not have access to the about:config URL in a headless environment.
Modify the prefs.js file to contain the following:
$ cat .mozilla/firefox/vrvkdf9e.firewatir/prefs.js # Mozilla User Preferences /* Do not edit this file. If you make changes to this file while the application is running, the changes will be overwritten when the application exits. To make a manual change to preferences, you can visit the URL about:config For more information, see http://www.mozilla.org/unix/customizing.html#prefs */ user_pref("app.update.auto", false); user_pref("app.update.enabled", false); user_pref("app.update.lastUpdateTime.addon-background-update-timer", 1266853577); user_pref("app.update.lastUpdateTime.background-update-timer", 1266853576); user_pref("app.update.lastUpdateTime.blocklist-background-update-timer", 1266853576); user_pref("app.update.lastUpdateTime.microsummary-generator-update-timer", 1266853577); user_pref("browser.places.importBookmarksHTML", true); user_pref("browser.places.importDefaults", false); user_pref("browser.places.leftPaneFolderId", -1); user_pref("browser.places.smartBookmarksVersion", 1); user_pref("browser.sessionstore.enabled", false); user_pref("browser.sessionstore.resume_from_crash", false); user_pref("browser.warnOnRestart", false); user_pref("extensions.lastAppVersion", "3.5.11"); user_pref("extensions.newAddons", "jssh@extensions.mozilla.org"); user_pref("extensions.update.enabled", false); $
You will need two SSH sessions to test JSSH. SSH into the TeamCity host server as yourself for this test. Start Firefox in the teamcity user SSH session (Note: Xvfb must have been started prior to this. See above.):
$ DISPLAY=localhost:1.0 firefox -jssh 9997 -P firewatir
Then use telnet to connect to JSSH in the other SSH session (enter Ctrl-] to exit the shell):
$ telnet localhost 9997 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. Welcome to the Mozilla JavaScript Shell! > ^] telnet> quit Connection closed.
Terminate Firefox in the first SSH session via Ctrl-C. If you do not do this and Firefox remains executing, you may have difficulty running your first Cucumber test.
Install FlashPlayer
Since we’re using Flex for the client interface, we need to install the Adobe Flash Player as a plugin to Firefox. Unlike a development system that has a browser window with which to interact requiring the debug version of Flash Player, we do not need the debug version for the headless version of Firefox.
Flash Player can be downloaded from here. The installation instructions can be found here but are simple and are documented as follows.
Note: Firefox should not be executing when these installation procedures are executed.
Download the tar.gz file and copy it to the TeamCity host server. SSH into the host server as the ‘teamcity’ user and unpackage the tar.gz file. Create a plugins directory in the ‘teamcity’ user’s .mozilla directory (if it does not exist) and move the unpackaged libflashplayer.so file to the plugins directory.
$ tar -xzvf libflashplayer-10.0.45.2.linux-x86_64.so.tar.gz $ mkdir .mozilla/plugins $ mv libflashplayer.so .mozilla/plugins/.
Install the SCM Client
We are using Git for our sample project. Git binaries can be downloaded from here. You can also install git using your system’s package manager.
$ sudo apt-get install git
If you are using your own project for this exercise, install your SCM client as appropriate.
That’s it for this session. Next time we’ll setup TeamCity and create a sample project.