Using the Baserock Import tool
This tutorial is here to get you familiar with the tool, so you know where to run it, and what output to expect. There are more in-depth tutorials for unusual packages, but this is here so that you can be confident in the basics before dealing with anomalous cases.
As well as this tutorial, there is documentation in the 'README' files in the Baserock Import Git repository. Please read the main README first, to learn what the tool does and what it does not do.
The Baserock Import tool is available in Baserock 'devel' systems. See the using baserock page for how to set up a devel system.
NOTE: This document is very new, as is the import tool. Please report any lies, inconsistencies, and pain points (ideally with patches to fix them!) to the mailing list. For a swifter response, ask on #baserock on irc.freenode.net
Part 1: Preliminaries
- Installing:
First, check you have the import tool installed. Run:
$ baserock-import
If you get the error message, "baserock-import not found", you need to update your baserock devel system. See upgrading baserock for instructions.
If you have the tool, running 'baserock-import' with no arguments should list the tool's usage. Great! You can move on.
2: Updating:
Unless you've newly upgraded to a devel vm, you now need to update your version of the import tool. To update, in /src, git clone the repository with the newer version of the import tool:
$ cd /src
$ git clone git://git.baserock.org/baserock/baserock/import
This should give you a directory in /src called 'import'. Next:
$ cd /usr/bin
$ mv baserock-import baserock-import.dist
Now make a new baserock-import file:
$ vim baserock-import
and put the following in it:
#!/bin/sh
importpath=/src/import
PYTHONPATH="$importpath" "$importpath/baserock-import" "$@"
(if you cloned the 'import' repo somewhere else, you will need to edit the importpath accordingly.)
Now, do:
$ chmod +x baserock-import
Your import tool should now be updated.
3: Making the tool work:
You may need to install some dependencies to run the tool. At the moment, this is only the case for Nodejs (npm) packages, so see the npm tutorial for the specifics.
After you've done this, try importing!
Part 2: Doing an import
The basic command is 'baserock-import PACKAGE-TYPE PACKAGE [version]'. At the moment, the supported package-types are: npm, PyPI and rubygems. The types may require slightly different commands (eg: the npm tool requires some environment variables set on the commandline), so please check the relevant tutorials.
To use the tool, I find it easiest to go into the directory containing baserock/baserock, then run the import command with the definitions-dir flag set to ./baserock/baserock/definitions. I also use the '--use-local-sources' flag, to ensure repo urls are set up correctly, and I don't get errors at build time. 'use-master-if-no-tag' is useful for quick tests. (for explanations on these, see 'Flags', below)
Example command to import the rubygem 'sinatra' as a quick test:
baserock-import rubygems sinatra --use-local-sources --definitions-dir=./baserock/baserock/definitions --use-master-if-no-tag
This builds the desired stratum in 'baserock/baserock/definitions/strata'; the tool will also generate the directories 'checkouts', 'lorries', and 'lorry-working-dir' in whichever directory you run the tool.
When a build succeeds, morph gives a message saying a stratum was generated. If you get errors, the tool will not generate a stratum. If you can't find information on the error in the readme or on the wiki, feel free to ask on the mailing list or on #baserock on irc.freenode.net.
Once a stratum is generated, manually add it to the .morph file describing the system you want; again, use the rubygems import tutorial as a template for now.
Part 3: Building the System
First, to check the stratum is what you're expecting, change directory to definitions/strata. You should see a [package].morph file, and possibly a [package] directory, if the package had dependencies (the package directory lists these dependencies).
Now, change directory to definitions/systems. You need to add your newly generated stratum to the system you want to enhance. For example, to add stuff to a web system:
$ vim web-system-x86_64-generic.morph
you'll see a list of things like:
- name: thing
morph: strata/thingy
at the bottom of this list, above the section titled 'configuration extensions', add the stratum you just created, eg:
- name: sinatra
morph: strata/sinatra.morph
You don't need to include the [pkg] directory; the [pkg].morph file tells baserock to get the dependencies from there.
Save and exit.
Now build and deploy the system, and try it out!
eg: change directory back to definitions/, and run:
scripts/cycle.sh systems/web-system-x86_64-generic.morph clusters/upgrade-devel.morph
Part 4: Flags
You can add the following flags to your command:
--use-master-if-no-tag
If there's no version tag in the git repo, this tells the tool to use the master version. This is a useful flag for a quick test, though it's better to git tag the repos manually for a system that needs to be reliable, since you're effectively ignoring info about required dependency versions.
--use-local-sources
Generates appropriate file:/// urls in the stratum. This ensures the 'repo' field points to the right place, ie: the url of the git repo. It will catch you out if you're not careful, since you'll only find there's a problem at build time!
--definitions-dir=[desired directory]
As expected, lets you specify where to put your definitions. I tend to set my desired directory to ./baserock/baserock/definitions, and then run the import command in the directory above baserock/baserock. This means the strata end up where I want them, but the definitions folder isn't cluttered with other things that the tool generates.
--force-stratum-generation
Generates a stratum even if errors occur. Generally not recommended.
--lorries-dir=[desired directory]:
Lets you specify where to put lorries, unsurprisingly enough.
--log=[name-of-log]
Lets you specify a log. You may have guessed this one. 