Ithiriel Thoughts From a Sysadmin And Occasional Coder

The RPMs in my repository are built from a git repository. In the repository, each package is a branch. This lets me keep all of the RPMs together while keeping them separate. To build them, I have a branch named BUILD. I switch to that branch and run make.

The Makefile merges all of the other branches into the BUILD branch, builds the RPMs, copies them into a repository directory, and then rsyncs that directory to my webserver. The source for the Makefile is included at the end of this post.

This method is optimized around a single repository and only needing to enter the password for my GPG key once. The downsides are that the repository layout isn't great and all RPMs are rebuilt even if only one really needs to be. So there are probably better ways to do this. tito may be a way to do this.

Anyway, here's the Makefile:

build : clean
        # Merge branches
        for i in `git branch | grep -v BUILD` ; \
        do \
                git merge $$i ; \
        done
 
        # Build RPMs
        rpmbuild -ba --sign SPECS/*.spec
 
        # Create repositories
        mkdir -p repo/{i386,x86_64}
        cp RPMS/noarch/* repo/i386/
        cp RPMS/noarch/* repo/x86_64/
        if [ -d RPMS/i386 ] ; \
        then \
                cp RPMS/i386/* repo/i386/ ; \
        fi
        if [ -d RPMS/x86_64 ] ; \
        then \
                cp RPMS/x86_64/* repo/x86_64/ ; \
        fi
        createrepo repo/i386/
        createrepo repo/x86_64/
 
        # Push repositories to server.
        rsync -av -e ssh repo/i386 repo/x86_64 user@host:/path
 
        # Push git repos to github
        git push --all origin
 
clean :
        rm -rf repo RPMS/i386/ RPMS/x86_64/ RPMS/noarch/

Now that you have signed RPMs, you have to find a way to make these available for installation.1 If you want to make these available for public use, that means a yum repository. If you're going to use them for private systems only, a yum repository is likely to be the best way to make these available and to handle updates later.

Creating a repository is easy to do: Copy your RPMs to a web-accessible directory and run the createrepo command.

It's traditional to have a separate repository for each combination of OS and architecture. EPEL shows a good example of this.

After creating the repository, you have to tell yum about it. To do this, you need to create a file in /etc/yum.repos.d/ with the .repo extension2 that contains the information for it. For an example repository, you would create a file named example.repo with these contents:

[example]
name=Example Repository for Enterprise Linux 5 - $basearch
baseurl=http://repository.example.cpm/el5/$basearch
failovermethod=priority
enabled=1
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EXAMPLE

You can define multiple repositories in a repo file. Each repository definition needs its own section with a unique name. The name of the section is how yum will refer to the repository.

Note that this file says that the GPG key is on the local filesystem. You can use a URL instead like seen here.

After you add this repository file (and put the GPG key in place if you want to keep it on the local filesystem), you should be able to see the RPMs in your repository when you run yum install.

The problem with this approach is that you (or your users) then have to manually maintain the repository files. For example, what if you want to move your repository? If you have the luxury of using a configuration management system, e.g. puppet, this may not be a big deal. However, if you don't or you don't have the access to the systems, there's no easy way to do this.3

How does EPEL deal with this? To make distributing the repo and GPG key easy for the users and to allow for easily updating information later, they package this information in the epel-release RPM. You can do something similar to distribute information for your repository and allow for updating it later.

The easiest place to start will be to download the epel-release SRPM, install it into your RPM build environment, and then make changes to the spec file. For the example repository above, you would end up with this spec file:4

Name:           example-release
Version:        5
Release:        5
Summary:      Example repository for Enterprise Linux configuration
 
Group:          System Environment/Base
License:        GPL
URL:            http://www.example.org
 
Source0:        http://repository.example.org/RPM-GPG-KEY-EXAMPLE
Source1:        GPL
Source2:        example.repo
 
BuildRoot:      %{_tmppath}/%{name}-%{version}-%{release}-root-%(%{__id_u} -n)
 
BuildArch:     noarch
Requires:      redhat-release >=  %{version}
Conflicts:     fedora-release
 
%description
This package contains the Example repository for Enterprise Linux
GPG key as well as configuration for yum.
 
 
%prep
%setup -q  -c -T
install -pm 644 %{SOURCE0} .
install -pm 644 %{SOURCE1} .
 
%build
 
 
%install
rm -rf $RPM_BUILD_ROOT
 
#GPG Key
install -Dpm 644 %{SOURCE0} \
    $RPM_BUILD_ROOT%{_sysconfdir}/pki/rpm-gpg/RPM-GPG-KEY-OBERONPROJECT
 
# yum
install -dm 755 $RPM_BUILD_ROOT%{_sysconfdir}/yum.repos.d
install -pm 644 %{SOURCE2} \
    $RPM_BUILD_ROOT%{_sysconfdir}/yum.repos.d
 
%clean
rm -rf $RPM_BUILD_ROOT
 
%files
%defattr(-,root,root,-)
%doc GPL
%config(noreplace) /etc/yum.repos.d/*
/etc/pki/rpm-gpg/*
 
 
%changelog
* Wed Apr 06 2011 Chris Ess <caess@ithiriel.com> - 5-5
- Modified epel-release package for the example repository
 
 Tue Aug 10 2010 Seth Vidal <skvidal at fedoraproject.org> - 5-4
- conflict fedora-release so people don't indadvertently do something silly
 
* Fri Apr 25 2008 Matt Domsch <Matt_Domsch@dell.com> - 5-3
- use mirrorlists in epel-testing.repo
- use download.fedoraproject.org in (commented out) baseurls
 
* Mon Apr 02 2007 Michael Stahnke <mastahnke@gmail.com> - 5-2
- Missed a syntax correction in epel-testing.repo
 
* Mon Apr 02 2007 Michael Stahnke <mastahnke@gmail.com> - 5-1
- Hard coded version '5' in epel yum repo files.
 
* Mon Apr 02 2007 Michael Stahnke <mastahnke@gmail.com> - 5-0
- Initial Package for RHEL 5

(Please note that this SPEC file is covered under the GPL and I am not the original copyright holder. As a result, I don't think I can remove the changelog for epel-release since it's the only way I see to attribute previous work. I might (and, therefore, you might) be able to do without this by adding a comment that says that this is based on version 5-4 of the epel-release SRPM. I'm not a lawyer and, as such, am not clear on the legal requirements here.)

Once you have the spec file, build (and sign!) the RPM. You can now distribute the RPM to your systems (or your users). To allow for updating this later, add it to your repository as well. Don't forget to add the SRPM to your SRPM repository as well. (If you're making this repository public, you need to have an SRPM repository! It also doesn't hurt to keep one since if your local source trees go away and your version control system5 explodes in a magnificent burst of flame.)

Having this RPM makes provisioning the repository on new machines even easier when using kickstart. Just add a repo line for your repository and add the RPM to the %packages section and it'll get installed automatically.6 You can use this trick to install other RPMs in the repository too.

This covers just about everything I can tell you about building RPMs and making a yum repository. In the future, I'll cover how I'm currently updating the repository I've been setting up. If you have a lot of packages to build, it might make your life easier.

• 1. If you don't have signed RPMs yet, you might want to read my previous post.
• 2. I'm not certain that this is required but I've never looked into it.
• 3. If you're running a public repo, you probably don't want to change the URL if at all possible.
• 4. Well, not quite. You might be building for a different version of RHEL. If nothing else, the changelog entry is going to be different.
• 5. You are using version control for your RPM builds, right?
• 6. Kickstart doesn't check to see if RPMs are signed so you can get away with this.

So you've built a shiny new RPM, let's say rubygem-cucumber-0.10.0-1.noarch.rpm, and you want to install it on a system. You enter the command:1

sudo yum install -y ./rubygem-cucumber-0.10.0-1.noarch.rpm

and then get this:

(some output removed)
 
Dependencies Resolved
 
================================================================================
 Package          Arch   Version        Repository                         Size
================================================================================
Installing:
 rubygem-cucumber noarch 0.10.0-1       /rubygem-cucumber-0.10.0-1.noarch 3.3 M
Installing for dependencies:
 rubygem-builder  noarch 2.1.2-2.el5    epel                               81 k
 rubygem-diff-lcs noarch 1.1.2-3.el5    epel                              123 k
 rubygem-gherkin  x86_64 2.3.4-1        oberonproject                     1.2 M
 rubygem-json     x86_64 1.4.6-1        oberonproject                     469 k
 rubygem-term-ansicolor
                  noarch 1.0.5-1.el5    epel                               42 k
 
Transaction Summary
================================================================================
Install       6 Package(s)
Upgrade       0 Package(s)
 
Total size: 5.2 M
Downloading Packages:
 
 
Package rubygem-cucumber-0.10.0-1.noarch.rpm is not signed

Well, bummer. yum wants the RPM to be signed.2

To sign your RPM, you first need a GPG key. To create one, run gpg --gen-key and follow the instructions. Once it's created, you should be able to see it by running gpg --list-keys. (For the rest of this, I'm going to assume the key is named "Software Packager". Where you see this, replace it with the name for the key.)

In order for yum to allow using your key, you'll need to import it into the RPM database. First, export the key to a file:

gpg --export -a 'Software Packager' > RPM-GPG-KEY-packager

Now, import it into the RPM database:

sudo rpm --import RPM-GPG-KEY-packager

To tell rpmbuild to use this key, add the following lines to your .rpmmacros file:

%_signature gpg
%_gpg_name Software Packager

Since you have an RPM built, you can add a signature with rpm --addsign, like so:

rpm --addsign ./rubygem-cucumber-0.10.0-1.noarch.rpm

Now, when you run sudo yum install -y ./rubygem-cucumber-0.10.0-1.noarch.rpm, the RPM will install successfully.

If you want to sign RPMs automatically when you build them, which I suggest, add the --sign option to rpmbuild like so:

rpmbuild -ba --sign SPECS/rubygem-cucumber.spec

So now that you have signed RPMs, you surely want to put them in a local repository. I'll show you how to do that (or at least how I do it) in the near future.3

• 1. You are doing this as a normal user and using sudo for anything that requires root privileges, right?
• 2. Yes, you could just install it with sudo rpm -i ./rubygem-cucumber-0.10.0-1.noarch.rpm but then you have to manually install the dependencies as well. On a single machine, this may not be too bad, but this won't scale.

  You can also pass --nogpgcheck to yum install but this may be prohibited by your local security policies. For example, the NSA Guide to the Secure Configuration of Red Hat Enterprise Linux 5 recommends ensuring that all yum repositories check the GPG keys.)

• 3. If you can't wait, check out the createrepo command.

Sometimes you run into a shortcoming in your Linux distribution and you find that a package you want doesn't exist. Perhaps there are is no package for the software available (for example, there is no rubygem-cucumber-nagios package available for CentOS 5). Perhaps the package available is too old (for example, there is no official PHP 5.3.x package for CentOS 5). Perhaps you need functionality built into the package that isn't normally there (for example, patching Apache httpd for mpm-itk). In these cases, you either need to find a package someone else has made or roll your own.

Before you think about installing directly from source, stop. This is a bad habit to get into. It will not scale.

Red Hat Enterprise Linux, Fedora Linux, SuSE, and their derivatives use RPMs for packaging. In this, I hope to show you how to build RPMs from source RPMs. In the future, I hope to show how to add patches to source RPMs to build customized packages.

(You may be able to follow these instructions to build your own RPMs from source. To do this, you will need to skip the step of "installing" the source RPM and will instead need to write your own spec file. I don't have any personal experience with this so I can't help you.)

Before you start anything, make sure you're not doing this as root. If you build RPMs as root, you run a risk of files ending up in system paths rather than in the RPM build path.

Doing it as a normal user? Good. Set up your build environment:

(For more information, you may want to read the CentOS documentation on setting up the RPM build environment.)

On CentOS and other RHEL derivatives, you will need to install the rpm-build and redhat-rpm-config packages. On other distributions, you may need to install other packages.

Once you have your build directory set up, install your chosen source RPM:

This will install the source files in ~/rpmbuild/SOURCES and the spec file in ~/rpmbuild/SPECS.

To build the RPM, you can simply run:

This will put the binary RPM under in architecture-dependent directory under ~/rpmbuild/RPMS/. For example, the RPM built for rubygem-gem2rpm is architecture dependent and rubygem-gem2rpm-0.6.0-1.noarch.rpm will be placed under ~/rpmbuild/RPMS/noarch/.

From here, you could simply install the rpm with sudo rpm -i ~/rpmbuild/RPMS/noarch/rubygem-gem2rpm-0.6.0-1.noarch.rpm. I don't recommend this approach. Instead, I recommend signing the RPMs and adding them to a central yum repository. I will cover both of these in the future so stay tuned.