Vostro-no! Wifi Issues with Dell Vostro 1710

I have been known to extol the virtues of a Linux desktop to PC owners who don't require any software that specifically requires Windows to run. Seeing how the average person uses their machines for web-based tasks (searching, email, Facebook, etc.), all they really need is a decent browser. Co-incidentally, this is the same reason tablets are selling so well and traditional PC sales are declining; why buy/upgrade when your current system handles these basic tasks without breaking a sweat?

Anyway, back on topic: I have always found, that the average user cannot, under any circumstances, be subjected to major changes in their desktop computing experience, much to my chagrin. However, I have found one Linux distribution that provides as close to a pre-Windows 8 experience as possible: Linux Mint, specifically the Cinnamon desktop release. Being based on Ubuntu, it pretty much works with most hardware out of the box, however, recently I ran into an issue with an old Dell Vostro 1710 whereby the wireless didn't work without some tweaking.

After I found the wireless device hadn't been initialised, I scanned through the output of dmesg to see what had gone wrong and happened across the following messages:

[ 10.338771] b43-phy0: Broadcom 4312 WLAN found (core revision 15)
[ 10.679089] b43-phy0 ERROR: Firmware file "b43/ucode15.fw" not found
[ 10.679094] b43-phy0 ERROR: Firmware file "b43-open/ucode15.fw" not found
[ 10.679097] b43-phy0 ERROR: You must go to http://wireless.kernel.org/en/users/Drivers/b43#devicefirmware and download the correct firmware for this driver version. Please carefully read all instructions on this website.

I could have simply visited the URL in that message, but I wasn't happy with the thought of manually installing the firmware for the device; after all, this system needed to be manageable by someone not familiar with the command line. Quickly searching around online revealed a post on the Ubuntu forums that suggested the firmware-b43-installer package would take care of this for me. However, during the package installation, I was greeted by the following message:

An unsupported BCM4312 Low-Power (LP-PHY) device was found.
Use b43 LP-PHY firmware (firmware-b43-lpphy-installer package) instead.

Trying the firmware-b43-lpphy-installer package resulted in success:

(Reading database ... 138948 files and directories currently installed.)
Removing firmware-b43-installer ...
Selecting previously unselected package firmware-b43-lpphy-installer.
(Reading database ... 138945 files and directories currently installed.)
Unpacking firmware-b43-lpphy-installer (from .../firmware-b43-lpphy-installer_1%3a015-14_all.deb) ...
Setting up firmware-b43-lpphy-installer (1:015-14) ...
No chroot environment found. Starting normal installation
--2013-08-26 14:14:58-- http://downloads.openwrt.org/sources/broadcom-wl-4.178.10.4.tar.bz2
Resolving downloads.openwrt.org (downloads.openwrt.org)... 78.24.191.177
Connecting to downloads.openwrt.org (downloads.openwrt.org)|78.24.191.177|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 5986780 (5.7M) [application/octet-stream]
Saving to: `broadcom-wl-4.178.10.4.tar.bz2'

5,986,780 2.94M/s in 1.9s

2013-08-26 14:15:00 (2.94 MB/s) - `broadcom-wl-4.178.10.4.tar.bz2' saved [5986780/5986780]

Creating new firmware directory...
broadcom-wl-4.178.10.4/
broadcom-wl-4.178.10.4/config/
broadcom-wl-4.178.10.4/config/wlconfig_nomimo
broadcom-wl-4.178.10.4/config/wlconfig_lx_router_apsta_1chipG
broadcom-wl-4.178.10.4/config/wlconfig_lx_wl_sdstd
broadcom-wl-4.178.10.4/config/wlconfig_lx_shared
broadcom-wl-4.178.10.4/config/diffupdate.sh
broadcom-wl-4.178.10.4/config/wlconfig_lx_router_ap_1chipG
broadcom-wl-4.178.10.4/config/wlconfig_lx_router_ap
broadcom-wl-4.178.10.4/config/wlconfig_lx_router_ap_sdstd
broadcom-wl-4.178.10.4/config/wl.mk
broadcom-wl-4.178.10.4/config/wltunable_lx_router.h
broadcom-wl-4.178.10.4/config/wlconfig_lx_router_sta
broadcom-wl-4.178.10.4/config/wl_default
broadcom-wl-4.178.10.4/config/wltunable_lx_router_1chipG.h
broadcom-wl-4.178.10.4/config/wl_hnd
broadcom-wl-4.178.10.4/config/wlconfig_lx_router_sta_1chipG
broadcom-wl-4.178.10.4/config/wlconfig_lx_router_apsta
broadcom-wl-4.178.10.4/config/wlconfig_apdef
broadcom-wl-4.178.10.4/README
broadcom-wl-4.178.10.4/linux/
broadcom-wl-4.178.10.4/linux/wl_sta.o
broadcom-wl-4.178.10.4/linux/wl.o
broadcom-wl-4.178.10.4/linux/wl_ap.o
broadcom-wl-4.178.10.4/linux/wl_apsta.o
This file is recognised as:
filename : wl_apsta.o
version : 478.104
MD5 : bb8537e3204a1ea5903fe3e66b5e2763
Extracting b43/ucode5.fw
Extracting b43/pcm5.fw
Extracting b43/b0g0bsinitvals5.fw
Extracting b43/a0g0bsinitvals5.fw
Extracting b43/b0g0initvals5.fw
Extracting b43/a0g1initvals5.fw
Extracting b43/a0g0initvals5.fw
Extracting b43/a0g1bsinitvals5.fw
Extracting b43/ucode9.fw
Extracting b43/a0g1initvals9.fw
Extracting b43/a0g0bsinitvals9.fw
Extracting b43/b0g0bsinitvals9.fw
Extracting b43/b0g0initvals9.fw
Extracting b43/a0g1bsinitvals9.fw
Extracting b43/a0g0initvals9.fw
Extracting b43/ucode11.fw
Extracting b43/n0bsinitvals11.fw
Extracting b43/n0absinitvals11.fw
Extracting b43/n0initvals11.fw
Extracting b43/ucode13.fw
Extracting b43/b0g0initvals13.fw
Extracting b43/a0g1bsinitvals13.fw
Extracting b43/a0g1initvals13.fw
Extracting b43/lp0bsinitvals13.fw
Extracting b43/b0g0bsinitvals13.fw
Extracting b43/lp0initvals13.fw
Extracting b43/ucode14.fw
Extracting b43/lp0initvals14.fw
Extracting b43/lp0bsinitvals14.fw
Extracting b43/ucode15.fw
Extracting b43/lp0bsinitvals15.fw
Extracting b43/lp0initvals15.fw
Extracting b43/ucode16.fw
Extracting b43/n0bsinitvals16.fw
Extracting b43/sslpn0initvals16.fw
Extracting b43/n0initvals16.fw
Extracting b43/lp0initvals16.fw
Extracting b43/sslpn0bsinitvals16.fw
Extracting b43/lp0bsinitvals16.fw

After a reboot (I probably could have reloaded the module with a modprobe -r b43 && modprobe b43), I was able to use the network applet to view local wireless networks. For reference, the chipset in use was the BCM4312; below is the relevant line from the output of lspci:

06:00.0 Network controller: Broadcom Corporation BCM4312 802.11b/g LP-PHY (rev 01)

Tri-Boot Refresh - Mountain Lion, Windows 8 and Fedora 18

This post has been a long time coming. Over the course of several months I spent the odd morning here and there attempting to build a new tri-boot machine for myself. My previous machine was still running Snow Leopard, Windows 7 and Fedora 14 and in dire need of upgrading. However, I was certain that an upgrade of OS X would destroy my carefully constructed partition table, which is why I opted to build a new machine so as to be able to continue to do my job!

Now, you might be wondering why it took so long for me to implement such a system, especially as I had already been through the process once before. The delay was caused by my initial insistence on a pure EFI/GPT booting solution; I wanted all three operating systems to boot via EFI and not rely on the Mac's hybrid GPT/MSDOS partition table and legacy boot options.

The main problem that I continually encountered was successfully booting Windows 8 via EFI. I tried several methods of installing the Microsoft OS, all of which failed to produce a bootable system:

• Booting directly off the DVD (by holding down although and selecting the "EFI" option). This resulted in the screen blanking and a spontaneous reboot before the installer even fully loaded.
• Following a well documented process that involved partitioning the disk using a WinPE environment and installing Windows via the dism tool that's distributed as part of the Microsoft Automated Deployment Kit. While this bypassed the initial Windows Setup step routine, I still experienced a similar issue; the screen blanked, the system rebooted and booting off the Windows partition again resulted in the following message:
  
  The computer restarted unexpectedly or encountered and unexpected error. Windows installation cannot proceed. To install Windows, click "OK" to restart the computer, and the restart installation.
  
  Grabbing the minidump from the partition and parsing it through the WinDBG tool pointed the finger at the Intel integrated GPU driver igdkmd64.sys.
• After the above discovery, I tried tweaking hardware prior to booting the operating system by issuing commands in the EFI shell provided by rEFIt. Specifically, to try and force the Nvidia GPU to be active during the initial boot. This got me a little further through the install process, but still eventually resulted in a system crash caused by the igdkmd64.sys.
• As a final attempt, I followed a suggestion to strip out the integrated graphics drivers after deploying the OS image. However, this also proved unsuccessful, even when combining this method with the previous technique of modifying the hardware settings via EFI shell.

Even after all the above failures, I was still keen to get this triple boot implementation working, but I was finally convinced to let the idea drop when I read some notes from people who stated that audio was known not to work with Windows instances booting via EFI. Given that my primary reason for having a system that boots Windows was to play games, it would have been terrible to sacrifice audio for the sake of possible faster boot times!

The Installation Process

After I made the decision to resort to the hybrid GPT/msdos partition table method, installation was much the same as before, with a couple of extra steps to carry out during the Linux install. I'll do my best to detail the process here, in case there are any brave souls out there willing to give it a go! I shouldn't have to say this, but this is a destructive process, so be sure to back up anything important first and be aware that your mileage may very, depending on the Apple hardware you're using (my current machine is a MacBook Pro 6,2)

1. Boot from the OS X installation media and use Disk Utility to create the following partition scheme (these sizes can obviously be varied, depending on your priorities):
   1. Size: 80GB, label: "OS X", FS Type: HFS+, journaled, case sensitive (my personal preference, the default is case insensitive)
   2. Size: 250GB, label: "WINDOWS", FS Type: msdos
   3. Size: 1.07GB (the smallest sized partition the GUI driven Disk Utility tool is able to create), label "BOOT", FS type: msdos
   4. Size: 20GB, label: "ROOT", FS type: msdos
   5. Size: 229GB (the remaining space), label: "LVM", FS type: msdos
2. Exit Disk Utility and begin the installation, selecting the OS X partition when prompted.
3. A dialog box will appear warning that the File Vault (probably the Full Disk Encryption mode)and the Recovery Mode will not be available; probably due to the installer not wanting to trash the complex partition scheme. Confirm you're happy to live without these features to continue the installation routine.
4. Once OS X installed, install the rEFIt bootloader to facilitate booting the other operating systems and their installers. NB: it might take a reboot or two for the bootloader to appear.
5. Insert the Windows 8 DVD and reboot the machine.
6. Boot off Win 8 DVD by CD/DVD labelled "Windows". This boots the installer using the legacy/BIOS mode of the Mac.
7. Now, it may be possible at this point to follow the usual Windows installation routine; answering questions as prompted and clicking next. However, due to all the problems I had with trusting installation Wizards, I opted to hit Fn-shift-F10 to spawn a command prompt and perform the installation manually. Technically, just shift-F10 is the necessary key-combination to bring up a command prompt, but on a Mac the function keys are by default bound to other specific purposes (volume control, screen brightness, etc.), so using the Fn (or function) key is necessary to access the regular function keys.
   1. Run Microsoft's command line partitioning tool: diskpart
   2. To be sure of the installation drive, list the available disks: list disk
   3. Select the destination drive (usually disk 0): select disk 0
   4. List the available partitions to be sure the intended destination: list partition
   5. Select the destination partition (in this case, it should be number 3): select partition 3
   6. Format the partition as NTFS: format quick fs=ntfs label=Windows
   7. Exit the partition tool: exit
   8. Change directory to the root of the X: drive. This is necessary to access the version of the Dism tool that can apply the OS image: cd \
   9. Run the Dism tool to apply the Windows 8 image (in this case, the Professional edition):
      dism /apply-image /imagefile:d:\sources\install.wim /index:1 /applydir:c:\
   10. Change directory to the root of the new installation: c:
   11. Configure the Windows boot loader: bcdboot c:\Windows /l en-gb /s c:
   12. Exit/close the command prompt.
   13. Exit/close the Windows installer, which results in the Mac rebooting.
8. Select the icon of the Windows logo to boot into the OS and finalise the installation as per usual (configuring user accounts, etc.).
9. Eject the Windows DVD and insert the Fedora DVD.
10. Reboot.
11. Select the CD/DVD icon.
12. Once the installer has completed booting, switch to an alternative virtual console by pressing Fn+ctrl+alt+F3, as it is necessary to create the layout of your LVM volumes before continuing with the installer:
    1. Launch the LVM tool: lvm
    2. Designate the last partition on the disk as an LVM Physical Volume (PV): pvcreate /dev/sda6
    3. Create a volume group: vgcreate vg_fedora /dev/sda6
    4. Create a volume for the /tmp file system: lvcreate -L 1G -n tmp vg_fedora
    5. Create a volume for the /var file system: lvcreate -L 1G -n var vg_fedora
    6. Create a volume for the /home file system: lvcreate -L 1G -n home vg_fedora
    7. Exit the tool and return to the installer by pressing the keys Fn+alt+F7.
13. Proceed as normal through the installation, using the previously created 20GB partition for the root (/) volume and LVM volumes for their respective mount points. One very important note: when selecting the disk for installation, be sure to not install a bootloader, as per the instructions in the Fedora manual. During my initial attempts, this would always fail and prevent the installation from completing successfully; it needs to be carried out manually post-install.
14. Once the installer completes, allow it to reboot the system.
15. Select the CD/DVD icon.
16. At the Linux boot prompt, navigate to and select the rescue system option.
17. Let the rescue system boot, detect and mount the installed instance of Fedora.
18. Select the shell option to be dropped to the command line.
19. Change root to the mounted install: chroot /mnt/sysimage
20. Generate a GRUB configuration: grub-mkconfig -o /boot/grub/grub.cfg
21. Install GRUB to the HDDs boot sector: grub2-install --recheck /dev/sda
22. Reboot and select the penguin icon to boot into Fedora and complete the initial setup of the OS.

If you've managed to follow all these steps, hopefully you're lucky enough to possess a triple boot machine! Some noteworthy points about the the above steps and the setup as a whole are as follows:

• If you run into a situation where either Windows or Fedora will not boot, it may be necessary to run the rEFIt sync tool. When booting it's one of the options below the available operating systems and it repairs the hybrid partition table after the use of a partitioning tool that's not aware of it's existence.
• After installing Fedora, trying to boot Windows via rEFIt will result in the GRUB bootloader appearing! However, the Windows install is an option here, simply select it to boot the OS.
• I found out about how to manually install GRUB by following steps I found on the Arch Linux wiki.

Feel free to sound off in the comments if you have anything to add, or you've tried the process yourself and hit a snag, I'd be happy to try and help.

Fedora 18 Xorg Segmentation Fault - NVidia Driver Configuration

I just encountered an extremely odd issue with my Fedora 18 this morning:

• First, I noticed that the grey background of the log in screen had been replaced by the blue "falling stars" image that's the default desktop wallpaper for the OS.
• Secondly, when logging into my usual Cinnamon session, the menu bar had switched from the Windows-esque "start" menu located at the bottom of the screen to a Gnome 2.x style menu at the top.
• Finally, when I tried to navigate to Delicious in Google Chrome, the screen blanked and returned to the log in menu!

Checking the Xorg log file showed a segmentation fault had occurred:

[ 192.214] (EE)
[ 192.215] (EE) Backtrace:
[ 192.230] (EE) 0: /usr/bin/Xorg (OsLookupColor+0x139) [0x472509]
[ 192.230] (EE) 1: /lib64/libpthread.so.0 (__restore_rt+0x0) [0x3f6080efff]
[ 192.230] (EE)
[ 192.230] (EE) Segmentation fault at address 0x0
[ 192.230]
Fatal server error:
[ 192.230] Caught signal 11 (Segmentation fault). Server aborting
[ 192.230]
[ 192.230] (EE)
Please consult the Fedora Project support
at http://wiki.x.org
for help.
[ 192.230] (EE) Please also check the log file at "/var/log/Xorg.0.log" for additional information.
[ 192.230] (EE)
[ 192.236] (II) evdev: Power Button: Close
[ 192.236] (II) UnloadModule: "evdev"
[ 192.243] (II) evdev: Power Button: Close
[ 192.243] (II) UnloadModule: "evdev"
[ 192.257] (II) evdev: Sleep Button: Close
[ 192.257] (II) UnloadModule: "evdev"
[ 192.274] (II) evdev: Apple Inc. Apple Internal Keyboard / Trackpad: Close
[ 192.274] (II) UnloadModule: "evdev"
[ 192.318] (II) UnloadModule: "synaptics"
[ 192.332] (II) evdev: Built-in iSight: Close
[ 192.332] (II) UnloadModule: "evdev"
[ 192.345] (II) evdev: Logitech USB Receiver: Close
[ 192.345] (II) UnloadModule: "evdev"
[ 192.367] (II) evdev: Logitech USB Receiver: Close
[ 192.367] (II) UnloadModule: "evdev"
[ 192.877] Server terminated with error (1). Closing log file.


Thinking this would be some strange transient error, I logged in again and found that I was able to replicate the issue by simply visiting delicious.com again! Googling around on the subject pulled up a couple of forum posts that suggested adding / amending the Files section in the /etc/X11/xorg.conf file to the following:

Section "Files"
  ModulePath "/usr/lib64/nvidia/xorg"
  ModulePath "/usr/lib64/xorg/modules"
EndSection


Once I restarted the X Server, I was greeted by the original log in screen with the grey background. Logging into the system yielded another success: my Windows-style menu had returned! The final check was also a success: I was able to visit delicious.com without the X Server crashing.

Low Cost CCTV Solution

Over the past several months, I have implemented and refined a home-security system built around the open source application, ZoneMinder. It started with a fairly simple requirement: being able to investigate any alerts generated by our ADT monitored alarm system.

The Software

• Ubuntu - I opted for this distro because there was the potential for the system to double as a media centre. I honestly believe that Canonical have done a great job making a Linux distribution for everyone to use and I wanted to make sure any non-techies using the machine wouldn't be (completely) lost!
• ZoneMinder - after installing Ubuntu, installing ZoneMinder was a breeze; it's available via the standard package repositories.
• MySQL - required for ZoneMinder's database. Installing the application via apt meant that this was installed and the relevant DB creation was taken care of automatically.
• Apache - required for ZoneMinder's web interface. Again, this was automatically installed an configured as part of the apt installation routine.

The Hardware

• Samsung X60 - I bought this machine back in 2005 (my first laptop purchase) and for a couple of years it was my primary machine for work and gaming. However, it soon became obsolete and was added to a pile of spares. I set up the laptop underneath the TV in our lounge, using the docking station I had purchased with the machine. This meant I could connect the machine to the TV, which prompted the choice of Ubuntu as the OS (see above).
• Logitech Quickcam 4000 - this device was directed towards the centre of the living room. Not only did this provide ZoneMinder with a great view over the main thoroughfare of the house, but when not monitoring the room the camera could be used by Skype, turning the sofa into a comfy video-calling booth.
• Logitech QuickCam Express - I positioned this to face the rear of the house; covering the back door.

With the system installed and configured and both of the cameras in place, I had a view over most the lower floor of our home. However, this was only the start; I had previously considered using a smartphone as a wifi-enabled network webcam, and I thought it would be perfect if I could hook one up to the ZoneMinder server. Sure enough, after a bit of searching online, I came across a guide on Google+ for re-purposing an Android smartphone as a remotely accessible camera, by simply serving the camera feed over HTTP. The only caveat: the software (IP Webcam) required the device be running Android 2.2 (Froyo) or higher. The two devices I had earmarked for use were an old T-Mobile G1/HTC Dream and a HTC Hero, which were only officially upgraded to 1.6 (Donut) and 2.1 (Eclair) respectively. To get around this issue I simply installed the most stable version of Cyanogenmod for both devices, Cyanogenmod 6 (Android 2.2/Froyo) on the G1/Dream and Cyanogenmod 7 (Android 2.3/Gingerbread) on the Hero.

Using the remote monitor feature in ZoneMinder, I was quickly able to configure both devices as wifi cameras and I positioned them in windows to cover the front and rear exterior of the property. With all the cameras in place, it was possible to fine-tune ZoneMinder's configuration:

• Monitor Calibration - to ensure a negligible number of false alarms, I found I had to adjust the sensitivity of the monitors/cameras and set "preclusive" zones; a region whereby any detected motion negates any potential alarms from being fired. This was mainly to compensate for changes in lighting conditions (when the Sun is obscured by a cloud, for example).
• Secure Remote Access - using port forwarding on my home router, and configuring Apache to serve the ZoneMinder control panel over HTTPS, I was able to view the monitors when outside of the house.
• Emailing Alerts - having ZoneMinder generate email alerts should certain conditions be met (enough alarms being triggered, for example).

One final step I took to improve the remote viewing capabilities of my Android phone was to install the Lite version of IP Camera Viewer. This little app allows you to set up multiple camera feeds (not just those of ZoneAlarm) to view on your phone. You are able to view each feed independently, or as a matrix/grid. While it doesn't react to any of ZoneMinders alarms, it does provide a really quick and easy way to check up on home should an alert be received.

Cycling Desktop Wallpaper in Gnome

I have found a couple of good applications for configuring a Gnome desktop wallpaper/background to cycle among a specific set of images.

Gnome Wallpaper Slideshow
This lightweight utility is simply a python script that renders a GTK+ GUI allowing you to choose a directory containing images, as well as set picture and transition durations.


Once you confirm your choices, it writes an xml file out to ~/.gnome-wallpaper/gnome-wallpaper-slideshow.xml, which is used by the Gnome to render the desktop. It also outputs a plaintext file to ~/.gnome-wallpaper/gnome-wallpaper-slideshow.config that records the options selected in the GUI so it's able to reload them next time you run the script. As soon as you've exited the application, you'll notice your desktop change and begin cycling through the images.

The only issue I found was not being able to set a preference for how I wanted to fit my images to the desktop; i.e. whether they should be centred, tiled, stretched, etc. Instead, Gnome simply resizes the images to fit the desktop. This was a problem for me as some of the images I had chosen were not the same size as my desktop and looked odd when scaled.

Wallpapoz
I found this application to be a bit more full-featured: it's possible to define a different image for each desktop, as well as specify your preference for whether an image should be scaled to the size of the desktop, used as a tile, or displayed at it's original size. On top of that, the software is available from the Fedora yum repository, so installing it was as simple as typing yum install wallpapoz into a terminal as root.
Once installed, you can access the program through the Gnome Applications menu (Applications -> Accessories -> Wallpapoz).


You are able to add directories or individual files to cycle through, while the preferences window gives you access to the more advanced functions.


Out of the two applications, I initially preferred gnome-wallpaper-slideshow because it was so lightweight. However, I eventually opted to use Wallpapoz, because of the image scaling issues I mentioned earlier.

Triple-Boot MacBook Pro: OS X, Windows 7 and Fedora 13

I have finally been able to install (and, more importantly, boot) 3 distinct operating systems on my work laptop; a MacBook Pro 3,3. I have been using my Mac in a dual-boot configuration for several years now; booting between OS X and Fedora. Being able to boot into OS X has allowed me to apply any new firmware released as part of an Apple Software Update, as well as providing a quick-booting OS for basic web browsing (less so now with the advances made in the Fedora distribution and Linux in general).

Having used Windows 7 on my home PC for some time, I decided that this would be the version of Windows that I would endeavour to install on the Mac alongside the other two operating systems. I've found Windows 7 to be a great improvement over Vista and, dare I say it, actually an enjoyable desktop OS to use. Of course, Windows 7 would also allow me to fully partake in any gaming sessions that I happened upon while I had my laptop! However, the most compelling reason to get my Mac triple-booting was to learn more about GPT, EFI and the boot process in general on an EFI-based system.

Obviously, the first step I took was to ensure I had a recent backup of all the important data from my machine, as I would be completely trashing the existing partition table. Once I had done this, I began the installation of the first operating system.

OS X


I inserted the Snow Leopard installation DVD into the machine and booted from it. From the installer, I accessed the Disk Utility and created a new GPT partition scheme consisting of 4 partitions (well, technically 5, see below for further details):

1. OS X: 90GB, HFS+, Journaled and Case-Sensitive


2. WIN7: 120GB, MSDOS


3. LINUX: 3GB, MSDOS


4. LVM: Remaining Space, MSDOS

I then exited Disk Utility and continued with the installation of the operating system, directing the installer to use the "OS X" partition.

Windows 7


As the laptop uses an Intel Core 2 Duo CPU, I opted to install the 64bit version of Windows 7 Professional. When choosing where to install the operating system, the partitioning tool displayed a rather different view to the OS X Disk Utility; a small FAT partition (used by EFI) was positioned in front of the OS X partition, while some of the drive was marked as unallocated space. I carefully selected the 3rd partition, confirming I had the correct one by looking at it's label (WIN7) and size (approximately 120GB), and formatted it using the "Advanced" features of the partitioning tool. This is a necessary step, as the OS X Disk Utility formats the file system as FAT32 and Windows 7 requires NTFS. After this, I proceeded to select the partition I had just formatted and installed the OS.
During the installation process, the machine had to reboot, which meant I had to ensure that it booted from the Windows 7 partition in order to complete the install. On a Mac, this is accomplished by holding down the "alt" key on the keyboard when the system first boots. It causes a basic boot manager to eventually be displayed, where you can select which of the installed operating systems to boot.

Fedora 13


Finally, I booted off the Fedora 13 installation disc (again, opting for the 64bit release). When I was presented with the partitioning tool, I found I had to manually format the 4th partition as ext3 and the 5th partition as an LVM physical volume using the command line. This involved switching to another virtual console, which on a Mac requires you to hold down the "fn" and "alt" keys and press "F2". From here, it is possible to use the mkfs.ext3 and lvm command line tools to format /dev/sda4 partition, create an LVM physical volume on /dev/sda5, a volume group and some volumes.
Again, I had to be wary of choosing the wrong partitions because, as with the Windows 7 installation, the tools displayed a more complicated view of the partition table; the EFI partition and un-partitioned space were visible.
Once I had finished on the command line, I switched back to the graphical installer ("fn", "alt" and "F7") and instructed the partition manager to use /dev/sda4 as the root partition and the newly created LVM volumes for /usr, /tmp, /var, /home and swap. It's important to note that I didn't define a separate /boot partition; this is because I wanted to keep the partition scheme as simple as possible, see below for details.
When the installer reached the stage where you define where GRUB should be installed, I found that the suggested default option was what I needed; /dev/sda4. This is different to a more vanilla install of Fedora, where you would usually install GRUB to the Master Boot Record (MBR); here, there is no MBR to install to as the partition scheme is GPT.

Final Steps

After the Fedora installation completed, I rebooted into OS X (by simply not pressing anything when the machine initially boots) and installed rEFIt. This appears to be the only bootloader that would allow me to boot all three installed operating systems. I had to run the rEFIt partitioning tool before this was possible however; accessible when rEFIt loads at boot-time. This performs the necessary magic that syncs the hybrid GPT/MBR partition table, allowing each OS to boot correctly. I had to perform one final restart of the machine, which is another option in the rEFIt bootloader, and then my triple-boot Mac was complete; apart from all the subsequent OS updates and configuration, of course!

Troubles Encountered and Lessons Learnt

As mentioned previously, this install procedure took me several attempts to get right. I tried using boot camp originally, but soon found that it added unnecessary complexity to the partitioning process. I also found some guides that demonstrated methods for partitioning the disc using the Disk Utility in OS X, but this assumed an existing OS X installation was present on the system. However, as this second approach mentioned the use of Disk Utility, I assumed that using this tool to create the partition scheme for all the operating systems would work; and I was correct!

While on the subject of partitioning, I did end up discovering exactly how the hybrid GPT/MBR partition table works. OS X itself requires a GPT partition scheme; it simply refuses to install to any MBR-style disk. However, Windows requires an MBR partition scheme to boot from and will refuse to install into a partition that it believes resides on a GPT formatted disk.

The hybrid GPT/MBR scheme allows the first 4 partitions of the GPT scheme to be used as an MBR-style disk. However, it is not possible for one of these partitions to be created as an extended partition. On a pure MBR disk, you are restricted to defining 4 primary partitions. One of these can be used to hold further logical partitions, which allows for more complicated partition schemes (i.e. more than 4 partitions).

Because GPT does not allow for extended and logical partitions to be defined, a hybrid implementation limits the MBR scheme to 4 primary partitions. On top of that, because we're booting an EFI based system, there is a hidden FAT32 partition at the beginning of the disk where boot configuration is stored. This brings the total number of MBR partitions available for operating systems to 3; hence why my 2nd, 3rd and 4th partitions are used by OS X, Windows 7 and Fedora's root/boot file systems respectively. Theoretically, it should be possible to install a Linux distribution into higher numbered partitions, however the GRUB boot loader used by Fedora only understands MBR, not GPT. In order to maintain some file system organisation, I created the 5th partition as an LVM logical volume, bringing all the flexibility of resizeable volumes to the Fedora install.

The other major issue encountered was actually booting off the Windows 7 install DVD. I had sourced a copy of Windows 7 Professional 64bit from the Microsoft Partner program site; which resulted in a rather large ISO image being downloaded to my machine. After burning this to DVD and attempting to boot off the disc, I found I was unable to progress past what appeared to be a broken boot menu. There were 2 options, however, both were lacking any explanation as to what it was I would be booting into! On top of that, neither option appeared to want to boot, and I was left with the impression that the system had actually crashed.

I did some research and found that the problem because of the way the image was mastered. Apparently, some important component used in the boot process is not able to handle files that conform to the ISO 9660 standard; specifically, version numbers being appended to the file name.

This made it necessary to re-master the provided image and burn it with some rather specific settings; the most important seemed to be force the omission of the version numbers (see the previously linked blog post for further details). Once I did this, the laptop booted from the disc perfectly.

Overall, I'm very pleased with the fruits of my labour. The real test will be to see if I can replicate the results on another machine.

MySQL: Too Many Open Files

Recently, I experienced an issue with a MySQL instance running on a CentOS 5.4 host whereby applications were unable to contact the daemon. This was despite there being being no network issues and the operating system not reporting any excessive load. Restarting MySQL fixed the issue and allowed the applications to continue working. While investigating the root cause of the problem, I found that the /var/log/mysqld.log file contained the following messages:

100804 4:43:00 [ERROR] /usr/libexec/mysqld: Can't open file: './test_db/test_table.frm' (errno: 24)
100804 4:43:00 [ERROR] /usr/libexec/mysqld: Can't open file: './test_db/test_table.frm' (errno: 24)
100804 4:43:03 [ERROR] Error in accept: Too many open files

In order to prevent the problem occurring again, I increased the maximum number of file descriptors available to MySQL. First, I increased the maximum number of open file descriptors available to the mysql user account by adding the following two lines to the file /etc/security/limits.conf:

mysql soft nofile 8192
mysql hard nofile 8192

Next, I modified the configuration of the MySQL instance to explicitly set the maximum number of open files allowed by simply adding the following line to the /etc/my.cnf file:

open-files-limit=8192

This configuration directive has a default value of 0, which means MySQL will attempt to allocate enough file descriptors itself. However, as I knew exactly how many open files where available to the MySQL user (8192), I could safely use this as the value for open-files-limit.
A restart of MySQL applied the changes and resolved the issue.

Building an RPM

While attempting to install the Snort IDS and the corresponding logging tool, Barnyard, onto a production x86_64 CentOS 5 server, I found myself in a situation where I would have to produce my own binary RPMs. On the Snort site, there is only an i386 binary RPM and a source RPM for Snort iself, while barnyard was only available as a source tar-ball. The alternatives would have been to compile the software locally and copy the resulting binary files into production, or to compile the applications on the hosts in question. Both of these options presented me with dilemmas:

Installing Development Tools in Production

I'm not happy installing development tools and libraries onto production machines, even temporarily. They can be used by malicious users to compile code, but even "trusted" users may end up abusing them by performing dev tasks or compiling and installing unauthorised software on the machine in question.

Maintenance of Software when Updates/Patches are released

Performing updates of the software, or indeed removing it, is much more difficult when installed from source. I would have to recompile the code on each host the software was running on.

So, instead of compiling the two packages on the servers in question, I opted to build my own binary RPMs that I could easily deploy on the production environment, or any other CentOS 5 instance.
In this article I'll be concentrating on how I built an RPM for Barnyard. There are two reasons for this:

1. There was only a tar-ball containing the source code available for Barnyard (as opposed to the source RPM for Snort itself). Focusing on the packaging of Barnyard means I'll be able to demonstrate the entire RPM building process.


2. Compiling Barnyard for a 64-bit environment requires that you apply a patch to the source code before compilation (see below for details). This provides an opportunity to document the ability of the rpmbuild tool to apply patches as part of the packaging process.

A lot of information regarding the RPM building process can be found online; I specifically used The Linux Documentation Project's howto guide, an article on the NetAdminTools site, the RPM site itself and an extremely useful PDF from the Guru Labs site. Even though I found these resources to be extremely useful, I felt I should document the process myself as there was information that I found to be ambiguous and, sometimes, out of date. If there's anything ambiguous about my article, please leave a note in the comments and I'll do my best to clarify!

Building RPMs is not particularly difficult if you are comfortable with compiling from source and working out the relevant dependencies. Because I hadn't installed Snort (or, in fact, any software from source) for a while, I adopted a two VM approach; one VM for testing the compilation and operation of Snort, the other for the RPM build process.

Initial Testing



I could have probably got away with simply using a single VM, but performing the initial compilation and testing on a separate VM meant that my build system was a clean instance of CentOS. As I have and will be working with software I have never used before, a dedicated "testing" VM ensures that I won't compromise a perfectly good RPM building VM, or my host operating system.

After successfully building and testing Barnyard on the test VM, I knew exactly what development packages I required in order to compile the application with the features that I desired:

• mysql-devel


• openssl-devel


• zlib-devel

With these packages installed, building Barnyard with the standard ./configure, make, make install process worked like a charm. However, while testing the application's ability to parse Snort binary logs and insert them into a snort database, the barnyard process kept on dying with the message: Invalid Packet Length.

When searching for a solution initially, all I could find were suggestions that my binary logs had become corrupt. This was unlikely, given I had just installed Snort and Barnyard onto a fresh VM, and further searches revealed that for 64 bit builds, a patch was required in order for correct operation.

After applying the patch to the source code and recompiling, barnyard worked perfectly; reading the binary logs and inserting any alerts into the Snort database. I was now ready to proceed to the actual RPM building stage.

Packaging the Software

I booted my build VM, copied over the source package and 64 bit patch and installed all the relevant development RPMs before taking the following steps to produce the package:

Copy Source and Patch to Appropriate Directory
The directory /usr/src/redhat/SOURCES is used to store the source tar.gz file and any patches you wish to compile. In my case, I copied the barnyard-0.2.0.tar.gz and barnyard.64bit.diff files to the directory.

Create a Spec file
The Spec file defines how an RPM is to be built and should reside in the /usr/src/redhat/SPECS directory. The name of the file should match the format: --.spec - so my spec file was named barnyard-0.2.0-1.spec (the build number "1" references the fact that this is the first attempt at building the package).
The file itself is composed of several sections that describe the operations to be carried out during the build process.
Below is the content of the spec file I created:


Summary: Barnyard output system
Name: barnyard
Version: 0.2.0
Release: 1
License: QPL
Group: IDS
Source: barnyard-0.2.0.tar.gz
Patch: barnyard.64bit.diff
URL: http://dl.snort.org/barnyard/barnyard-0.2.0.tar.gz
Buildroot: %{_tmppath}/%{name}-%{version}-root
Requires: mysql zlib openssl
BuildRequires: mysql-devel zlib-devel openssl-devel

%description
Barnyard is an output system for Snort. Snort creates a special binary output format called unified. Barnyard reads this file, and then resends the data to a database backend. Unlike the database output plug-in, Barnyard manages the sending of events to the database and stores them when the database temporarily cannot accept connections.

%prep
%setup -q
%patch -p1


%build
./configure --prefix=/usr --enable-mysql --with-mysql-includes=/usr/include/mysql --with-mysql-libraries=/usr/lib64/mysql
make RPM_OPT_FLAGS="$RPM_OPT_FLAGS"
%install
%makeinstall

%files
/usr/bin/barnyard

%changelog

* Tue Oct 20 2009 Peter Green
- Initial build.


Most of the entries in the file are self-explanatory (Summary, Name and %description, for example), however, I feel I should elaborate on some:

Group:

This defines the package group that this package should be a member of.

Source:

This entry is fairly self explanatory. However, it's worth mentioning because it's possible to define multiple source packages by using multiple entries and simply appending numbers to each; for example: Patch0:, Patch1:, etc.

Patch:

Another obvious entry; this defines any patches to be applied to the source. Again, like the "Source:" entry, multiple patches can be defined by similarly appending numbers to each.

Buildroot: %{_tmppath}/%{name}-%{version}-root

Defines the location where the compiled software will be installed to before being packaged as an RPM.

Requires:

This lists the software packages required for the software to run. Attempting to install the RPM without this prerequisites present on a system will result in dependency errors. However, package management software like yum can use this requirements to automatically install any dependencies. The actual process of building the RPM will calculate any specific libraries that are required for the software to execute, so it is possible to leave this out. However, dependencies will be defined as particular files, instead of package names.

BuildRequires:

Is similar to the Requires: header, but is used to define dependencies for compiling the software.

%prep

This defines a section; it is typically used to "prepare" the sources (i.e. extracting the source code and applying patches).

%setup -q

This is a macro; it's interpreted by the rpmbuild tool to mean extract the source archive into the /usr/src/redhat/BUILD directory. The -q parameter turns off verbose output for the macro.

%patch -p1

Another macro; this applies the patch defined previously. The -p1 parameter sets the -p parameter passed to the patch command itself.

%build

This defines the start of the stanza where the configuration and compilation of the software take place. Instead of using the explicit configure and make shell commands, it's possible to use %configure and %make macros. However, in this case, I needed to use specific parameters to ensure that barnyard would be built as required.

%install

This stanza is where the software compiled in the %build section is installed into the directory defined using BuildRoot:

%makeinstall

This macro is what accomplishes the installation of the software into the BuildRoot

%files

This stanza lists all the files created by the install process. As you can see, the Barnyard package simply consists of one file: the executable itself. Care should be taken when creating this section of the spec file; listing a directory (/usr/bin, for example) will mean that rpmbuild will package all the files in that directory and the package produced will appear to own the directory in question! Package ownership of directories can be defined using the %dir macro. For example: %dir /etc/barnyard
During the actual RPM building process, I notice that the rpmbuild utility warned me if it detected a file that wasn't included in the %files stanza.

%changelog

This section is for documenting the changes made for each iteration of the build. As this was my first build of Barnyard, there is only one entry.

Generating the RPM



With the rpmbuild command, you can test each stage of the packaging process by passing different arguments to the -b switch. During the process of building the barnyard rpm, I used the following:

rpmbuild -bp /usr/src/redhat/SPECS/barnyard-0.2.0-1.spec

Carries out the %prep stage defined in the spec file; in this case, the source tarball is extracted and the 64bit patch is applied. Using this variation of the command, it was possible for me to ensure that the patch would be applied successfully to the source code.

rpmbuild -bc /usr/src/redhat/SPECS/barnyard-0.2.0-1.spec

Performs the steps defined in the %prep and %build sections. Allowed me to check that the compilation process would work OK.

rpmbuild -bi /usr/src/redhat/SPECS/barnyard-0.2.0-1.spec

Executes the %prep, %build and %install sections of the prep file. Useful, because it allows you to easily see which files are produced by the make install process; simply check under your BuildRoot.

rpmbuild -ba /usr/src/redhat/SPECS/barnyard-0.2.0-1.spec

Produces both source and binary RPMs. There are switches that allow you to produce just the source (-bs) and binary (-bb) separately, if you prefer.

rpmbuild --clean /usr/src/redhat/SPECS/barnyard-0.2.0-1.spec

Cleans up the build tree; useful while testing the software will configure and compile correctly.

Installing the RPM



After I had produced a binary RPM for Barnyard, I copied it over to the production hosts and installed using the rpm utility. The software worked exactly as intended on both hosts and I now had an RPM for future use on other CentOS hosts.

The only one issue I had with the package produced is that it isn't a signed package. My next step will be to set up a GPG key on the build host to allow me to sign packages that I produce!

mlocate DB Size Problem

I have just had to resolve an issue with the mlocate search tool on a backup server.
The machine is a small (10GB) VM with an iSCSI volume presented to it for data that is backed up. It turns out the mlocate DB had grown to an excessive size (>1GB!) because it had been including this file system in it's nightly DB refresh.
This meant that the /var file system ended up running out of space, which was causing all kinds of problems:

• The nightly mlocate DB refresh task could not complete.


• Emails could not be sent as the spool directory for sendmail could not be written to.


• Log files could not be written to.


• Performing a system update with yum was not possible.

My solution was to:

1. Remove all files from the /var/lib/mlocate directory.


2. Open the mlocate configuration file, /etc/updatedb.conf and edit the PRUNEPATHS directive; adding the backup file system, /srv to the list of paths that are excluded from being indexed.