Executive Summary

The client, MyClient, is in need of a more automatic, integrated, and productive Java web application build and release processes due to its growth of business and organization.

Currently, the Development team builds from their own IDE and deploys them manually. Since the process is manually done and not standardized, it can be time-consuming and high risk. Human errors could easily bring down the company’s revenue-source site.

Based on the information collected, I propose to standardize the build process, streamline the delivery flow, and utilize existing open source tools to improve the business bottom line. Three major goals will be achieved:

  • Build of a code management repository – Growing team members have a shared place to branch, merge, back, and collaborate their work.

  • Creation of a Continuous Integration (CI) Environment – The Development team can use this environment to develop project features and fixes without being distracted by building and deploying overhead.

  • Integration of Controlled build/release promotion process – With the help of this integration interface, QA and Operation can save time and avoid human errors by using the same deploy process to package and promote code to the next target environments.

After evaluating some available tools and options on the market, I recommend the following open source tools to implement this improvement. By selecting market available software package, the company can focus more on seamlessly functional integration that saves MyClient significant amount of time and money.

  • Web Server – Apache Server

  • Version Control System – Subversion

  • Java Web Server – Tomcat

  • Build – Maven

  • CI – Jenkins

Three phases are proposed. Initial phase, Constant Build & Deploy, is to install and customized the selected technologies to continuously build code from the designated code repository on the build server. The builds are then deployed to target environments. The standardized tools, controlled by QA, will be automatically triggered immediately after developers’ check-in. The milestone of this phase is measured by time of the whole code deployment cycle. Possible delay might come from Prevayler deployment, if it is not streamlined. (This is what has been implemented so far)

Second phase, system integration, is to improve (fine-tune) the integration among control accessing, ticketing, knowledge sharing (i.e., Wiki), and notification, etc. The milestone is determined by index of data hopping between departments and systems.

To complete the project, the third phase is to add efficiency, connectivity and intelligent modules into the ecosystem. Since the developed interface keeps tracking “most active areas”, “iteration trends”, “dependency”, and “impact/risk analysis”, re-evaluation of the applied technologies becomes an iterative yet effective process for the company.

MyClient, will see impacts on three areas:

  • Code development velocity increases as well as new employment training flexibility.

  • Standardized code deployment process saves time and resources.

  • Feedback/intelligence cycle can be established to identify risk pro-actively and improve quality.

However, the current system layout is based on the development of Continuous Integration and Improvement. Upon all connections being integrated, the efficiency can be significantly improved. Major practice adaption can generate problems, if the team ignore the importance of the communication and policy enforcement. Scaling up can be another issue, if we can not project the expanding of the organization and growing components of the applications. Moving into cloud storage and computing can be a solution for hardware scaling up.

Most Impacted Areas

Version Control

  • Source code repository management

  • Branching strategy

  • Check in policy

  • Access Control

  • Backup & Recovery

Build

  • Build process (beyond IDE)

  • Build environment

  • Continuous Integration (Check-In->Build->Deploy)

  • Build once, deploy multiple places

  • Notification

  • Dependency

  • 3-party tools/libraries

Deployment

  • Continuous Integration

  • Deploy on demand

  • Deploy and rollback

  • Hosted, on premise, or cloud

  • Prepare/reset the target environments or not

Audit

  • Hot list of excluded, monitored, and filtered items

  • Out-of-scope check up

  • Quality records

  • Alerts

  • Progress reports.

Intelligence

  • High traffic areas

  • Impact analysis

  • Risk Analysis

  • Continuous improvement

Communication

  • Push

  • Query

  • Dashboard

  • Mailing lists

  • Email templates

  • Tracking records

Automation & Integration

  • Vertical

  • Horizontal

  • Knowledge sharing

Preliminary Assessment

  • Products

    • Front End – Java Web Application

    • Back End – Prevayler

  • Environments

    • INT

    • QA

    • Production

  • Hosting

    • Cloud – Amazon EC2 with EBS

  • Platform

    • Linux – Ubuntu 8.0.4 LTS

  • IDE

    • Eclipse

  • Language

    • Java EE

  • Organization

    • Dev

    • QA

    • Operation

  • Version Control

    • None

  • Build

    • Inside IDE – Eclipse

  • Deployment

    • Manual

  • Ticketing

    • None

Observations & Visions

Background

A development team has been working on a Java web application in their IDE (Eclipse for Java EE).  At present they build their code and run the application only within the IDE.

Basic problem

This process is being stretched as the team grows in number and location.  It will also run into serious problems when testers start working on the application and when IT Operations wants to deploy it into production.  As a build and release manager, we ask you to assist this team in delivering a reliable application to production.

Assumptions

The application needs to be signed off by a QA team before it is deployed to production by the IT Operations team.

As a release manager you have free rein to incorporate any software tools and hardware you need to streamline development, QA and Ops work.

Description of environments

Developers work on Windows XP workstations.  Developers are free to download and trial any third party tools to assist in creating the features for this application.

The QA and production environments are Ubuntu 8.0.4 LTS Server.

User Stories

Prepare the virtual machine for client’s local
  • Observation

    • Developers are using MS Windows platform

    • They might have the needs to have a local environment (Dev) for local testing on Linux/Ubuntu

  • Analysis

    • VirtualBox is an open source

    • VirtualBox is commonly approved on Windows platform and most organization

    • Recommend to install Guest Addition on its guest (Ubuntu)

    • Recommend to setup Shared Folders so we can share files with the host

    • This is just to help the effectiveness of developers work

  • Proposed Action

    • Devices -> Install Guest Additions

    • sudo mount /dev/cdrom /media/VBOXADDITIONS

    • cd /media/BOXADDITIONS

    • sudo sh ./VBoxLinuxAdditions.run

    • (HOST): VBoxManage sharedfolder Shared

    • sudo mount -t vboxsf Shared /home/username/myDesktop

Need a Web server to interact with tools and environments
  • Observation

    • This can be good for the static web sites

    • And also can be used for SVN access

    • Should be part of out-of-box, but just to double check

  • Analysis

    • Enable the Apache on this Ubuntu VM

  • Proposed Action

    • sudo apt-get install apache2

    • httpd.conf or ports.conf

    • http://localhost – It Works

Need a Version Control System for code repository
  • Observation

    • No version control is used to share code with the team

    • Source code repository is needed for Continuous Integration

  • Analysis

    • Recommend to use Subversion as it is

      • Open source

      • Easy to setup & install

      • Support the need of MyClient

    • Recommend simple user control

      • Use standalone user file to manage access

      • Web access through Apache

      • Can integrate with company wide access control in the future

  • Proposed Action

    • Install

      • sudo apt-get install subversion libapache2-svn

      • sudo svnadmin create /svn

    • Setup

      • Edit config file – sudo gedit /etc/apache2/mods-enabled/dav_svn.conf

      • Uncomment

        • <Location /svn>

        • DAV svn

        • SVNParentPath /var/svn

        • AuthType Basic

        • AuthName “Subversion Repository”

        • AuthUserFile /etc/apache2/dav_svn.passwd

        • </Location>

      • sudo htpasswd -cm /etc/apache2/dav_svn.passwd <username>

      • sudo /etc/init.d/apache2 restart

    • Repository

Need an IDE to act as client side for demo purpose
  • Observation

    • This is optional in the real world. However, for demo (on this single VM) purpose, I would like to have an IDE so we can test the whole trip of CI

  • Analysis

    • Eclipse

    • Maven Plug in

    • Subversion Plug in – Subclipse

  • Proposed Action

    • Eclipse

    • Download – http://www.eclipse.org/downloads/?osType=linux

    • tar xzf eclipse-SDK-4.2.2-linux-gtk-x86_64.tar.gz

    • mv eclipse /opt/

    • sudo chown -R root:root eclipse

    • sudo chmod -R +r eclipse

    • sudo touch /usr/bin/eclipse

    • sudo chmod 755 /usr/bin/eclipse

    • sudo nano /usr/bin/eclipse

#!/bin/sh

#export MOZILLA_FIVE_HOME=”/usr/lib/mozilla/”

export ECLIPSE_HOME=”/opt/eclipse”

$ECLIPSE_HOME/eclipse $*

  • sudo nano /usr/share/applications/eclipse.desktop

[Desktop Entry]

Encoding=UTF-8

Name=Eclipse

Comment=Eclipse IDE

Exec=eclipse

Icon=/opt/eclipse/icon.xpm

Terminal=false

Type=Application

Categories=GNOME;Application;Development;

StartupNotify=true

  • /opt/eclipse/eclipse -clean &

  • Maven Plug in

    • Eclipse -> Help -> Software Updates -> Find and Install

    • Search for new features to install -> New Remote Site

    • Enter Name:”Maven2Plugin” and home URL:“http://download.eclipse.org/technology/m2e/releases“

  • Subversion Plug in

    • sudo apt-get install eclipse

    • sudo apt-get install subversion

    • sudo apt-get install libsvn-java

    • Install New Software

      • Name: “Subclipse 1.6.x (Eclipse 3.2+)”

      • URL: “http://subclipse.tigris.org/update_1.6.x

    • sudo find / -name “libsvnjavahl-1.so”

    • sudo vi /usr/lib/eclipse/eclipse.ini

-Djava.library.path=/usr/lib/jni

Need a build environment for Java web app

  • Observation

    • Can’t build from IDE for CI

  • Analysis

    • Maven

  • Proposed Action

    • apt-cache search maven

    • sudo apt-get install maven

    • mvn -version

    • ls -ls /usr/share/mav

Need a web server for the target environment of Java web application

export JAVA_HOME=/usr/lib/jvm/default-java

export CATALINA_HOME=~/path/to/tomcat

  • . ~/.bashrc

  • vi /etc/tomcat7/conf/tomcat-users.xml

<role rolename=”manager-gui”/>

 <role rolename=”manager-script”/>

 <user username=”tomcat” password=”tomcat” roles=”manager-gui”/>

 <user username=”managerscript” password=”tomcat” roles=”manager-script”/>

Need a Hello World project for demo/verification/POC
  • Observation

  • Analysis

    • Code Location: ~/MyClient

  • Proposed Action

SVN URL: http://MyClient.com/svn/MyClient/Web

Implement the Continuous Integration
  • Observation

    • Let’s pick an open source tool instead of creating one from scratch

  • Analysis

    • Jenkins

    • Subversion plug in

    • Maven plug in

    • Deploy plug in

    • Release plug in

  • Proposed Action

HTTP_PORT = 9090

    • Subversion plug in

    • Maven plug in

    • Deploy plug in

    • Release plug in

  • CI Job

    • New Job -> “Build a maven ⅔ project”

    • Name: MyClient

    • Source Code Management: Subversion

    • Build Triggers

      • Build whenever a SNAPSHOT dependency is built

      • Poll SCM

      • Schedule: H/5 * * * *

      • */5 * * * *

    • Build

      • Root POM: pom.xml

    • Post Steps: Run regardless of build result

    • Build Settings

      • E-Mail Notification: notification@MyClient.com

    • Post-build Actions

      • Deploy war/ear to a container

      • WAR/EAR files: target/Maven.WebApps.war

      • Container: Tomcat 7.x

      • Manager user name: managerscript

      • Manager password: tomcat

      • Tomcat URL: http://localhost:8080

Demo of the Continuous Integration (CI)
Advertisements