In order to address situations where a Google Apps admin needs access to a user’s mail data, Google provides an Email Audit API. It allows administrators to audit a user’s email and chats, and also download a user’s complete mailbox. While Google provides this API, third-party tools are required in order to make use of the functionality. While there are some add-ons in the Google Apps Marketplace that make email auditing available, the most direct method of gaining access to this is with a command-line tool called Google Apps Manager. GAM is a very powerful management tool for Google Apps, but here we will focus on just what’s required to use the Email Audit API.

Using GAM requires granting access, with a Google Apps admin account, to a specific system. An OAuth token for the domain is stored in the GAM folder. Also, if you’re going to download email exports, it’s necessary to generate a GPG key and upload that to Google Apps. In light of both of these factors, it’s best to designate a specific system as the GAM management system. GAM is a collection of Python modules, so whatever system you designate should be something that has a recent version of Python. We’ll assume that we’re using a fairly recent Mac.

What we’ll do is download GPG and generate a GPG key, and then download GAM and get it connected to Google Apps.

Generating a GPG key

The GPGTools installer is here: http://www.gpgtools.org/installer/index.html

After installation, open up Terminal, in the account that you’ll be using to manage Google Apps.

Run the command:

$ gpg –gen-key –expert

For type of key, choose “RSA and RSA (default)”. For key size, you can probably safely choose a smaller key. Bear in mind that all your mailbox exports will be encrypted with this key and then will need to be decrypted after download. This can take a non-trivial amount of time, especially for larger mailboxes, and a larger key will mean much longer encryption and decryption times. A 1024-bit key should be fine in most cases.

When asked for how long the key should be valid, choose 0 so that the key does not expire.

Next you’ll be prompted for your name, email address and a comment. This information is not, at the moment, used by Google for anything. However, in the interests of long-term usability, I would recommend using the email address and name of an actual admin for the Google Apps domain.

Finally, you’ll be asked for a passphrase. This passphrase will be required in order to decrypt the downloaded mailboxes. Do not forget it. You will be unable to decrypt the downloads without it.

When key creation is complete, you’ll see something like this:

pub 1024R/0660D980 2012-03-22
Key fingerprint = A642 0721 2D4A 9150 6ED1 DBD7 AFFF 992F 0660 D980
uid Apps Admin
sub 1024R/6D1C197B 2012-03-22

Make a note of the ID of the public key, which in this case is 0660D980. You’ll need the ID to upload the key to Google.

Installing GAM

Prior to installing GAM, you’ll want to open up your default browser and log into to your Google Apps domain as an administrator. It’s not technically necessary – you can log in as an admin when the GAM install needs access, but you’ll find it authenticates more reliably if log in in advance.

GAM can be found here: http://code.google.com/p/google-apps-manager/downloads/list

Download the python-src package, and put it somewhere in the home directory of the same user that generated the GPG key. The most reliable way to invoke GAM is using the python command to call the script:

$ python ~/Desktop/gam-2/gam.py

This assumes it was unzipped to the Desktop of the user account. Change the path where appropriate. In order to make this a bit easier, you can create an alias that will allow you to call it with just “gam”

$ alias gam=”python ~/Desktop/gam-2/gam.py”

From here on, we’ll assume you did this. Bear in mind that aliases created this way only last until the session ends (i.e. the Terminal window gets closed).

The first command you’ll need to run is:

$ gam info domain

You’ll be asked to enter your Google Apps Domain, and then you’ll be asked for a Client ID and secret. These are only necessary if you’ll be using Group Settings commands, which we won’t. Press enter to continue. You’ll now be presented with a list of scopes that this GAM install will be authorized for. You can just enter “16″ to continue with all selected, or you can just select Audit Monitors, Activity and Mailbox Exports for Email Audit functions. When you continue, you’ll see this:

You should now see a web page asking you to grant Google Apps Manager access. If you’re not logged in as an administrator, you can do that now, though you may experience some odd behavior. Once you grant access, return to the terminal Window and press Enter. At this point, GAM will retrieve information about your domain from Google Apps, and you’ll be returned to a shell prompt. GAM is installed and almost ready to use.

Uploading the GPG Key

There’s one final step to take before mailbox export requests are possible. The GPG key you generated earlier must be uploaded to Google. What you can do is have gpg export the key and pipe that directly to GAM. You’ll need the ID of the key so that you export the correct one to GAM. If you didn’t make a note of the ID earlier, you can see all the available keys with:

$ gpg –list-keys

pub 1024R/0660D980 2012-03-22
uid Apps Admin
sub 1024R/6D1C197B 2012-03-22

The ID you want is that of the public key. In this case, 0660D980. Now export an ASCII armored key and pipe it to GAM.

$ gpg –export –armor 0660D980 | gam audit uploadkey

Now you’re ready to request mailbox exports.

Dealing with mailbox exports

To request a mailbox export, use:

$ gam audit export includedeleted

This will submit a request for a mailbox export, including all drafts, chants, and trash. You can leave off “includedeleted” if you don’t want their trash. GAM will show you a request ID, which you can use to check the status of a request.

To check the status of one request, use:

$ gam audit export status

If you leave off either username or request ID, you’ll be shown the status of all requests, pending and completed. To download a request you can use:

$ gam audit export download

You must specify both the username and the request ID. Please note that GAM will download the files to the current working directory. The files will be named “export---.mbox.gpg. The numbers will start at 0. In order to decrypt the downloaded files, you’ll need to use GPG.

$ gpg –output –decrypt

This will decrypt one of the files. The predicatbility of the names makes it easy to programatically decrypt all the files. For instance if the username were bob, the ID were 53521381, and there were 8 files, you could use this command:

$ for i in {0..7}; do gpg –output export-bob-53521381-$i.mbox –decrypt export-bob-53521381-$i.mbox.gpg; done

When decryption is completed, you can take the resulting mbox files and import them into any mail client that supports mbox – Thunderbird is a good choice, though Mail.app should work as well – or you can just look at them in a text editor.

Further Reading

For more details about using GAM or the Email Audit API, please consult the official documentation.

Google Apps Manager Wiki: http://code.google.com/p/google-apps-manager/wiki/GettingStarted

Google’s Email Audit API reference: https://developers.google.com/google-apps/email-audit/