InstaDMG Issues, and Workflow Automation via Your Friendly Butler, Jenkins

January 17th, 2013 by Allister Banks

“It takes so long to run.”

“One change happens and I need to redo the whole thing”

“I copy-paste the newest catalogs I see posted on the web, the formatting breaks, and I continually have to go back and check to make sure it’s the newest one”

These are the issues commonly experienced with those who want to take advantage of InstaDMG, and to some, it may be enough to prevent them from being rid of their Golden Master ways. Of course there are a few options to address each of these, in turn, but you may have noticed a theme on blog posts I’ve penned recently, and that is:

BETTER LIVING THROUGH AUTOMATION!

(We’ll get to how automation takes over shortly.) First, to review, a customized InstaDMG build commonly consists of a few parts: the user account, a function to answer the setup assistant steps, and the bootstrap parts for your patch and/or configuration management system. To take advantage of the(hopefully) well-QA’d vanilla catalogs, you can nest it in your custom catalog via an include-file line, and you only update your custom software parts listed above in one place. (And preferably you keep those projects and catalog under version control as well.)

All the concerns paraphrased at the start of this post just happen to be discussed recently on The Graham Gilbert Dot Com. Go there now, and hear what he has to say about it. Check out his other posts, I can wait.

Graham Gilberts Blog
Back? Cool. Now you may think those are all the answers you need. You’re mostly right, you smarty you! SSDs are not so out-of-reach for normal folk, and they really do help to speed the I/O bound process up, so there’s less cost to create and repeat builds in general. But then there’s the other manual interaction and regular repetition parts – how can we limit it to as little as possible? Yes, the InstaDMG robot’s going to do the heavy lifting for us by speedily building an image, and using version control on our catalogs help us track change over time, but what if Integrating the changes from the vanilla catalogs was Continuous? (Answers within!)

Jenkins

Jenkins is your trusty butler, always at the ready to take care of tasks for you. His full name is JenkinsCI, for Continuous Integration. Traditionally used with software projects that require compilation, automated build systems like this one can help smooth over contention for resources when multiple collaborators are possibly checking in to the same project. While not obviously applicable for our use, it is purpose-built to check for updates that folks may make in a version control system…like our vanilla catalogs are kept in.

Here’s a couple of steps to consider if you want to get this process automated for your custom catalog:
- Install Jenkins on a build box that will become a sort of ‘image toaster’ for you. It’ll create a ‘jenkins’ user and a ‘home’ for itself in /Users/Shared by default. Everything from that point forward is configured in its web GUI, which will startup at localhost:8080 by default
* Requisite warning, protect it as you see fit, it’s unsecured by default
- Since the ‘jenkins’ user isn’t in the admin group and needs to run InstaDMG for you as sudo/root(they’re doing the common, but not 100% recommended way of user creation with 236 UID/primary group IDs) add it to your sudoers file via visudo. The lines would look like this:
jenkins ALL = NOPASSWD: /Volumes/SSD/ # if you are using a external volume and want to ensure permission
jenkins ALL =(ALL) NOPASSWD:ALL

sudoers

Here’s where it goes in the sudoers file

- You can then setup the svn checking functionality by going to Manage Plugins, searching the Available tab for Jenkins Subversion Plug-in, installing, and letting the Jenkins service restart

- Then you can create a new job, name it, select the “Build a free-style software project” option and clicking Ok
- You’ll be dropped in the Configure interface, where you’ll select Subversion from Source Code Management, and specify the repo URL(in this case, http://instadmg.googlecode.com/svn/trunk/)
- OPTIONAL: you probably want to specify a RAID array or SSD volume vs. the default Jenkins HOME, which you’d specify by going into the Advanced Project Options by clicking the Advanced button. Then check the “Use custom workspace” and give it a path that the ‘jenkins’ user has read/write access to, taking a quick trip to terminal to chown -R jenkins:staff /path/to/SSD/and/workspace as necessary

custom workspace

Leeeeeeeeeeeroyyyyyyyyyy….

- Apply/Save and click ‘build now’ to initiate a checkout
- Drop your applicable InstallESD.dmg into /InstallerFiles/InstallerDiscs (or BaseOS) and chown -R jenkins:staff /path/to/installesd.dmg just for good measure
- Go back into the configuration of the job and customize the Build section to have these contents in “Execute shell”:
sudo /usr/bin/python /Users/Shared/Jenkins/Home/jobs/instaDMG/workspace/AddOns/InstaUp2Date/instaUp2Date.py -p myCustom10.8_vanilla

build execute shell
(Different than pictured, the options assume your job is named instaDMG and has a custom 10.8 catalog inside or InstaDMG’s AddOns/InstaUp2Date/CatalogFiles folder, which contains an include-file line to nest 10.8_vanilla inside it)
- You’ll also want it to regularly check svn for new revisions, by checking “Poll SCM” (for source configuration management, commonly referring to revision control systems like svn or git) under “Build Triggers” and adding a schedule like 30 * * * * (which is the cron syntax for running every half hour.)

build trigger scheduler

If you want to dump the downloaded cache to ensure the catalogs contents are freshly acquired each time, and are using an external SSD to build on, you’d change the path of instaUp2Date.py to that new location, and use the –set-cache-folder=/tmp and –instadmg-scratch-folder=/Volumes/SSD/temps flags(assuming a ‘temps’ folder which jenkins has permissions to at the root of your ssd.) Further options include getting an email/notification on success/failure (or just a successful svn checkout), and/or moving the resultant image to your DeployStudio repo.

I imagine a utopia where updating your custom packages sets off an attempt to checkout the newest catalogs AND triggers a build, which upon completion triggers some mechanism to spin up a machine/VM of the image, and finally Sikuli/Eggplant/Cucumber runs it through its paces to Quality Assure it.
It’s coming! I, for one, welcome our automated butler overlords.

Tags: , , , , , , , , , ,

Comments are closed.