Building an RPM

While attempting to install the Snort IDS and the corresponding logging tool, Barnyard, onto a production x86_64 CentOS 5 server, I found myself in a situation where I would have to produce my own binary RPMs. On the Snort site, there is only an i386 binary RPM and a source RPM for Snort iself, while barnyard was only available as a source tar-ball. The alternatives would have been to compile the software locally and copy the resulting binary files into production, or to compile the applications on the hosts in question. Both of these options presented me with dilemmas:

Installing Development Tools in Production

I'm not happy installing development tools and libraries onto production machines, even temporarily. They can be used by malicious users to compile code, but even "trusted" users may end up abusing them by performing dev tasks or compiling and installing unauthorised software on the machine in question.

Maintenance of Software when Updates/Patches are released

Performing updates of the software, or indeed removing it, is much more difficult when installed from source. I would have to recompile the code on each host the software was running on.

So, instead of compiling the two packages on the servers in question, I opted to build my own binary RPMs that I could easily deploy on the production environment, or any other CentOS 5 instance.
In this article I'll be concentrating on how I built an RPM for Barnyard. There are two reasons for this:

1. There was only a tar-ball containing the source code available for Barnyard (as opposed to the source RPM for Snort itself). Focusing on the packaging of Barnyard means I'll be able to demonstrate the entire RPM building process.


2. Compiling Barnyard for a 64-bit environment requires that you apply a patch to the source code before compilation (see below for details). This provides an opportunity to document the ability of the rpmbuild tool to apply patches as part of the packaging process.

A lot of information regarding the RPM building process can be found online; I specifically used The Linux Documentation Project's howto guide, an article on the NetAdminTools site, the RPM site itself and an extremely useful PDF from the Guru Labs site. Even though I found these resources to be extremely useful, I felt I should document the process myself as there was information that I found to be ambiguous and, sometimes, out of date. If there's anything ambiguous about my article, please leave a note in the comments and I'll do my best to clarify!

Building RPMs is not particularly difficult if you are comfortable with compiling from source and working out the relevant dependencies. Because I hadn't installed Snort (or, in fact, any software from source) for a while, I adopted a two VM approach; one VM for testing the compilation and operation of Snort, the other for the RPM build process.

Initial Testing



I could have probably got away with simply using a single VM, but performing the initial compilation and testing on a separate VM meant that my build system was a clean instance of CentOS. As I have and will be working with software I have never used before, a dedicated "testing" VM ensures that I won't compromise a perfectly good RPM building VM, or my host operating system.

After successfully building and testing Barnyard on the test VM, I knew exactly what development packages I required in order to compile the application with the features that I desired:

• mysql-devel


• openssl-devel


• zlib-devel

With these packages installed, building Barnyard with the standard ./configure, make, make install process worked like a charm. However, while testing the application's ability to parse Snort binary logs and insert them into a snort database, the barnyard process kept on dying with the message: Invalid Packet Length.

When searching for a solution initially, all I could find were suggestions that my binary logs had become corrupt. This was unlikely, given I had just installed Snort and Barnyard onto a fresh VM, and further searches revealed that for 64 bit builds, a patch was required in order for correct operation.

After applying the patch to the source code and recompiling, barnyard worked perfectly; reading the binary logs and inserting any alerts into the Snort database. I was now ready to proceed to the actual RPM building stage.

Packaging the Software

I booted my build VM, copied over the source package and 64 bit patch and installed all the relevant development RPMs before taking the following steps to produce the package:

Copy Source and Patch to Appropriate Directory
The directory /usr/src/redhat/SOURCES is used to store the source tar.gz file and any patches you wish to compile. In my case, I copied the barnyard-0.2.0.tar.gz and barnyard.64bit.diff files to the directory.

Create a Spec file
The Spec file defines how an RPM is to be built and should reside in the /usr/src/redhat/SPECS directory. The name of the file should match the format: --.spec - so my spec file was named barnyard-0.2.0-1.spec (the build number "1" references the fact that this is the first attempt at building the package).
The file itself is composed of several sections that describe the operations to be carried out during the build process.
Below is the content of the spec file I created:


Summary: Barnyard output system
Name: barnyard
Version: 0.2.0
Release: 1
License: QPL
Group: IDS
Source: barnyard-0.2.0.tar.gz
Patch: barnyard.64bit.diff
URL: http://dl.snort.org/barnyard/barnyard-0.2.0.tar.gz
Buildroot: %{_tmppath}/%{name}-%{version}-root
Requires: mysql zlib openssl
BuildRequires: mysql-devel zlib-devel openssl-devel

%description
Barnyard is an output system for Snort. Snort creates a special binary output format called unified. Barnyard reads this file, and then resends the data to a database backend. Unlike the database output plug-in, Barnyard manages the sending of events to the database and stores them when the database temporarily cannot accept connections.

%prep
%setup -q
%patch -p1


%build
./configure --prefix=/usr --enable-mysql --with-mysql-includes=/usr/include/mysql --with-mysql-libraries=/usr/lib64/mysql
make RPM_OPT_FLAGS="$RPM_OPT_FLAGS"
%install
%makeinstall

%files
/usr/bin/barnyard

%changelog

* Tue Oct 20 2009 Peter Green
- Initial build.


Most of the entries in the file are self-explanatory (Summary, Name and %description, for example), however, I feel I should elaborate on some:

Group:

This defines the package group that this package should be a member of.

Source:

This entry is fairly self explanatory. However, it's worth mentioning because it's possible to define multiple source packages by using multiple entries and simply appending numbers to each; for example: Patch0:, Patch1:, etc.

Patch:

Another obvious entry; this defines any patches to be applied to the source. Again, like the "Source:" entry, multiple patches can be defined by similarly appending numbers to each.

Buildroot: %{_tmppath}/%{name}-%{version}-root

Defines the location where the compiled software will be installed to before being packaged as an RPM.

Requires:

This lists the software packages required for the software to run. Attempting to install the RPM without this prerequisites present on a system will result in dependency errors. However, package management software like yum can use this requirements to automatically install any dependencies. The actual process of building the RPM will calculate any specific libraries that are required for the software to execute, so it is possible to leave this out. However, dependencies will be defined as particular files, instead of package names.

BuildRequires:

Is similar to the Requires: header, but is used to define dependencies for compiling the software.

%prep

This defines a section; it is typically used to "prepare" the sources (i.e. extracting the source code and applying patches).

%setup -q

This is a macro; it's interpreted by the rpmbuild tool to mean extract the source archive into the /usr/src/redhat/BUILD directory. The -q parameter turns off verbose output for the macro.

%patch -p1

Another macro; this applies the patch defined previously. The -p1 parameter sets the -p parameter passed to the patch command itself.

%build

This defines the start of the stanza where the configuration and compilation of the software take place. Instead of using the explicit configure and make shell commands, it's possible to use %configure and %make macros. However, in this case, I needed to use specific parameters to ensure that barnyard would be built as required.

%install

This stanza is where the software compiled in the %build section is installed into the directory defined using BuildRoot:

%makeinstall

This macro is what accomplishes the installation of the software into the BuildRoot

%files

This stanza lists all the files created by the install process. As you can see, the Barnyard package simply consists of one file: the executable itself. Care should be taken when creating this section of the spec file; listing a directory (/usr/bin, for example) will mean that rpmbuild will package all the files in that directory and the package produced will appear to own the directory in question! Package ownership of directories can be defined using the %dir macro. For example: %dir /etc/barnyard
During the actual RPM building process, I notice that the rpmbuild utility warned me if it detected a file that wasn't included in the %files stanza.

%changelog

This section is for documenting the changes made for each iteration of the build. As this was my first build of Barnyard, there is only one entry.

Generating the RPM



With the rpmbuild command, you can test each stage of the packaging process by passing different arguments to the -b switch. During the process of building the barnyard rpm, I used the following:

rpmbuild -bp /usr/src/redhat/SPECS/barnyard-0.2.0-1.spec

Carries out the %prep stage defined in the spec file; in this case, the source tarball is extracted and the 64bit patch is applied. Using this variation of the command, it was possible for me to ensure that the patch would be applied successfully to the source code.

rpmbuild -bc /usr/src/redhat/SPECS/barnyard-0.2.0-1.spec

Performs the steps defined in the %prep and %build sections. Allowed me to check that the compilation process would work OK.

rpmbuild -bi /usr/src/redhat/SPECS/barnyard-0.2.0-1.spec

Executes the %prep, %build and %install sections of the prep file. Useful, because it allows you to easily see which files are produced by the make install process; simply check under your BuildRoot.

rpmbuild -ba /usr/src/redhat/SPECS/barnyard-0.2.0-1.spec

Produces both source and binary RPMs. There are switches that allow you to produce just the source (-bs) and binary (-bb) separately, if you prefer.

rpmbuild --clean /usr/src/redhat/SPECS/barnyard-0.2.0-1.spec

Cleans up the build tree; useful while testing the software will configure and compile correctly.

Installing the RPM



After I had produced a binary RPM for Barnyard, I copied it over to the production hosts and installed using the rpm utility. The software worked exactly as intended on both hosts and I now had an RPM for future use on other CentOS hosts.

The only one issue I had with the package produced is that it isn't a signed package. My next step will be to set up a GPG key on the build host to allow me to sign packages that I produce!