Peter Mac And Associates

Java/Hibernate ant build example

This is the contents of a typical ant build file I use for Java coding projects. It’s up here in case it can help anyone else, and so I can copy and paste it when working off-site.

The process is fairly simple, I have my working area in a ~/projects/myapp folder (where myapp is name of current project). I have lib files in ~/projects/java/lib/

I run the build in the myapp file, the ant file reads the build.properties file for some config info and dumps the build outputs to the $targetdir folder (~/projects/bin)

Firstly, the build.properties file

Now for the actual ant file. This is specific to a GWT application, but you can easily knock out the google bits and use the rest without any changes.

Feel free to suggest any ways to make this simpler and more automated.

Moving from Subversion to Git

I have been using subversion for a number of years and it wasn’t until I had seen and sampled the simplicity and power of git that I decided once and for all to bite the bullet and migrate all my subversion repositories to git.

I’d done the same from cvs to subversion in the early 2000′s and still think the move worked out well with a bit of preparation.

An unfortunate side effect of subversion is that it takes a good bit of effort to set up and manage a new repository so I ended up with two somewhat monolithic repositories…

1. archives
2. projects

The ‘archives ‘ repository contained all old project work and the ‘projects’ one contained relevant or current project work.

The result of this was that each archive had a bundle of sub-folders, each containing a project in its’ own right. Although git can handle the concept of sub-modules it’s not the best way to structure your project. In fact, it was a pretty lazy to structure my subversion projects in the first place but convenience overcame system administrative chores at the time.

This article explains the following:

1. Converting a monolithic subversion repository to a git repository
2. Breaking out the new git repository into a set of discrete repositories.
3. General backup process and scripts to copy the new repositories across to a backup system.

First off, a couple of conventions I keep are…

My server stores a central set of repositories with the name of myproject.git
My workstations work with their local version of repositories called myproject (no .git)
My server exports it’s .git repositories to a backup server using the same .git naming convention.
I have created a git.git user.group on my server to run all git processes. File ownership is given to this user so other users can’t fowl things up at least without thinking about it first.

Step 1 – Migrating to subversion

I store all my git repositories on a server in the directory

/usr/local/share/gitrepos

All of my subversion repositories are stored in

/usr/local/share/svn

Even though each folder is accessible using standard paths and commands, I have to go through the subversion door using the same mechanism I would access it from a remote box, namely http://. Other people may use svn:// if that’s their way of working.

Migration is taken care of with a single command.

An explanation of all the bits follow:

sudo – because I run everything as unprivileged user, sudo gives me the rights i need to create new files and folders.

git svn – this is the git subcommand that manages conversion of subversion repositories.

clone – make a copy of the repositories that I’m pointing at.

https://localhost/repository/projects – I access all my subversion repositories using secure http. This allows me to securely browse the content across the Internet and track things easily using Trac.

Depending on your setup, this parameter will contain the path to your svn repository,however you access it.

–stdlayout option tells git svn that my subversion repository is in the standard layout of trunk/branch/tags directories.

–authors file – this is a file I created by hand containing a list of all the people who committed project material in the past. It’d format is as follows:

-t tags -b branches -T trunk – these values are for completeness and I’m not sure they are necessary given the –stdlayout option, but if you’ve called your branches, tags and trunks anything different, this is how you find it.

Once the command has completed you should find a projects.git folder has been created and navigating into it you will see all your nasty subversion sub-folders which you should have set up as individual repositories in the first place…tut tut!

Step 2 – Breaking out Git into baby gits

The structure of the new git repository is something like this.

The .git folder (you cans see it using ls -a) contains a list of all the git related material such as project history, revisions and tags.
Check you can see your history by typing

The next task is to break the contents of the rather large ‘project’ folder into a git repository per project.
The tool for the job is a combination of the very useful git subtree command and a bit of custom shell scripting specific to this job.

The subtree functionality was written by Avery Pennarun and is hosted by the github site. You can download the git subtree command from the URL http://github.com/apenwarr/git-subtree. Installation instructions are on the same site so I won’t bore you with it here.

Once you have it installed you’re almost ready to roll. The following shell script has comments at the top to explain what it does. Save this to your favourite scripts folder and chmod it so it’s executable (chmod +x git_breakout.sh).

Then cd into the folder you created earlier (in my case /usr/local/share/gitrepository/projects). Run the script and watch the output. Depending on the size of your projects it will take a bit of time to complete; mine took about 20 minutes.

Once the script has completed, you should be able to see a new directory structure, something along the following lines.

Each folder is now its’ own git repository with it’s own internal version history, tags, branches etc.
cd into one of the folders and do a git log just to confirm you still have history.

The original projects.git can be removed (after backing it up for safety’s sake) if you’re satisfied everything is in place. We’re now ready to proceed with the next stage of the game, namely backing up our new git repositories to a remote share.

Step 3 – Backing up Git

I use a 2TB removable disk as a central data server for sharing files throughout my network. My thinking is that should the place ever burn down and I have the opportunity, there’s only one box I need to grab before rushing out the door. Of course, I burn frequent snapshots of this box to DVD.

I have created a ‘backups’ share on the disk which I map to my physical server using NFS.

The entry in my /etc/fstab file is something like this.

This means I can read/write to my backups folder as if it was a local folder on a local disk.

The job at hand is to be able to backup my git repositories using a cron task scheduled to run when everything is nice and quiet. Below is the script I use to run my git backups.

Once this script is run, you ‘ll have a copy of all your repositories in the backup folder. The next time it’s run should only take a fraction of the time as it won’t have to reestablish the git repository again.

So there you have it, migrating a subversion repository to git is really simple, the fun starts when you try to play with the results. I hope this has been helpful to others faced with the same job. Any improvements please let me know.

Designing Enterprise Software

I’ve just finished reading a little gem of a book. It’s called ‘The Martian Principles for Successful Enterprise Systems’ with a subtitle of ’20 Lessons Learned from NASA’s Mars Exploration Rover Mission’. The author is Ronald Mak.

Imagine designing an information retrieval, indexing and presentation system for the two Mars rover vehicles that were sent on a one-way reconnaissance mission to Mars for a three month mission. The feisty little vehicles kept going for two years and the information systems had to be designed to cope with this unexpected project over-run.

The book runs to 168 pages and is a ‘should-read’ for anybody involved in designing or buying large-scale enterprise software. From an architect’s perspective, you get a reinforced mental checklist of the aspects of your designs that make them work and ensure they keep working long after you’ve moved on. From a customer’s perspective, you gain an appreciation of the effort put into designing such systems. From a developer’s perspective, now you know why you spend so much time writing and executing unit tests.

The book has short and well directed chapters and is an easy read with coverage of both the technical side of software development and the soft or human side.

As a result of this read, I went back to enhance some application logging classes that I’ve used on a number of projects to provide more granular output and statistics on usage patterns.

Working with the .Net Compact Framework

The attached files contain binaries and source code for a .Net Compact Framework application we built to enable the input of petrol consumption details and the subsequent downloading of that information to a desktop application. It was written in C# under a Visual Studio 2005 project and makes use of a set of libraries provided by OpenNETCF Consulting.

To develop the application you will need to download the Smart Device Framework libraries from OPenNETCF and add a reference to them in your project. I have included a feature that allows you to dynamically upload all necessary components to your mobile device. This is a cool feature enabled using a .Net custom installer application.

Feel free to download and modify the application to suit your fancy.

Fuel Monitor 1.1.0 Source Files

Fuel Monitor 1.1.0 Binaries

Validating your PHP Input

In this world of bad guys and hackers and crackers, us developers have to be on our guard against all sorts of stuff that tries to break our websites. Validating input using JavaScript alone is not sufficient to gurad against the baddies as it can be worked around by building their own custom forms to interact with your site and/or disabling JavaScript.

What this means is that every piece of data you accept from a user should be validated against a set of rules for that datatype.

Some general validation rules

If it’s a text string, does it contain any less than or greater than brackets?

If it’s a date, are it’s day, monthy, year parts valid numbers?

If it’s a credit card number, is there an online number validation algorithm you can use to verify it?

Regular Expressions in PHP

In order to validate your user-input, you’ll generally find yourself resorting to regular expressions. In PHP there are two options.ereg and preg. ereg is older, less sophisticated and somewhat slower than it’s younger brother. My preference is to use the preg options even though they’re a little more complicated, the end results is betterer(sic).

An example of using preg to validate a date:

An online manual is available from the main php website at http://www.php.net/manual/en/ref.pcre.php

And some lessons explicitly on how to use the preg function is available here http://www.php.net/manual/en/function.preg-match.php

Using magic_quotes_gpc or addslashes()

I’ve worked on a bundle of web based applications over the years and time and time again I’ve seen the recurring problem of the slash. Yes, we’ve probably all seen it in one or more forums where the apostrophe some user entered, probably with the name O’Brein ends up as O\\Brein.

Why does this happen in sites running on PHP? The answer is a duplication of escapes. Yep, a Houdini Supreme.

Firstly a systems administrator has installed PHP and set the value for magic_quotes_gpc = on in the system’s php.ini (usually located in /etc/). This will automatically add slashes to all GET/POST/COOKIE data. This makes it safe before writing it to a database. Mr O’Brein becomes Mr O\\’Brein when magic_quotes_gpc is set to on.

Secondly, a programmer has come along and thinking they’re doing the right thing takes all user input and uses the addslashes() funtion to escape all quotes. This results in a doubling of the escapes so, Mr O’\\Brien now becomes MR O\\\\’Brein.

When this data is rendered, we see the automatic removal of only one set of escapes but the other set is left behind…yuck!!

When programmers see this they think…”I’ll just use the stripslashes() method, I mean, that’s what it’s there for”. As the light from the idea bulb fades, they realise they’re fixing a problem that should never have occurred in the first place. You need to go to the source of your data and clean it up, make sure you’re either using magic_quotes_gpc=on OR addslashes. My preference is to use addslashes all the time and turn magic_quotes_gpc off, this way the logic of your code explicitly sets user input to be what you want.