Bitten by a Viper

In this case, the viper in question is an old industrial control computer, based on an Intel PXA255 processor, in PC104 form factor, running a minimal linux. It was originally manufactured by Arcom Ltd, who have in the intervening years been taken over by Eurotech S.p.A. and comes with a nice complement of default communications options.

Mine is a standard “full-fat” Viper board, with no additional expansion cards, but mounted in an industrial enclosure with an integral UPS battery-backup. Sadly it’s been lying around gathering dust in my study for about the last 5 years, and as part of a recent attempt to tidy up some of my clutter I decided I either needed to make use of it, or get rid of it.

A little searching unearthed the power supply, and then, according to the little red LED on the back panel, we had life.

The first problem was how to talk to it. This is a true industrial device – no screen or keyboard. The options are either to telnet or ssh in over the network connection, or use an ASCII terminal over a serial connection. And therein lay the first problem; it didn’t connect to my network as I was expecting, leaving only the serial option open to me. But as I’m almost exclusively laptop-based these days, finding a machine with a serial port, and then a serial cable to link it to the Viper, took quite some effort.

But having done so, and set up minicom with the correct serial port parameters (helpfully documented in the Viper manuals) I was able to login to the Viper, at which point a little debugging of the network stack revealed the issue – at some point I’d defined the Viper with a static IP address, on a different subnet to my current home network.

A few quick edits to return it to DHCP operation, and a little configuration work on my home servers dnsmasq configuration returned the Viper to my home network, at a readily accessible IP address, allowing me to ssh in again.

At this point, having proved that the system is working again, I now need to decide what to do with it; and in this case, I’m going to use it as part of my plan to implement a set of DIY ambient orbs, as I mentioned back in this post. The Viper happens to consume only 9w while network connected, and has a built-in I²C interface, which can connect to BlinkM LED’s which I can mount in my B&Q Cubo housings. This will allow me to control the colour of the orbs from a program running on the Viper.

Better than that though, I can then install an MQTT client on the Viper, allowing my Viper program to subscribe to a topic on my home servers MQTT micro-broker. This will allow me to have the Viper change the colours of the orbs in response to messages published to the broker on my home server. At that point I can trivially integrate any sort of external input to the orbs just by administering my home server, & without needing to make further updates to the Viper.

So all I need to do now is to get the AEL development kit (cross-compiler etc) set up so I can write some code for the Viper, and then buy some BlinkM LEDs. Which sounds simple, but unfortunately the AEL development kit hasn’t been updated for some 4 years now, and isn’t keen on installing on my latest and greatest Ubuntu 10.10 development machine. I’ve put some calls out to colleagues who have also had experience with these Arcom Vipers, but so far no joy. Next step will be to see if Eurotech can provide a little help – after all, I suspect the original Arcom engineers will mostly still be working there. I guess the worst case will be to set up a *really* old Linux system in a virtual machine, and install the AEL development kit on that instead.

Network upgrade

As most of you are aware, I’ve had a wireless network at home for something like the last 8 or 9 years. Originally an 802.11b network, powered by an original “flying saucer” Apple Airport, it’s been upgraded a couple of times, finally ending up as an 802.11g network powered by a Linksys WRT54GL router, acting as an access point. And it’s worked really well.

But a few years ago I bit the bullet, and installed my own structured cabling system, based around Cat5e, such that I had at least a couple of ethernet ports in each major room in the house. Because as we all know, wires are always faster and more reliable than wireless. This necessitated building a rack in the loft, and installing a 24 port 10/100 switch, and some patch panels to allow me to connect up all the modular faceplates that were scattered around the house. And again, it’s worked really well.

But this last year I wanted to extend my network to the (detached) garage, and discovered that my wireless didn’t have the range. And I also noticed that there were some other “black spots” around my house that had relatively poor coverage. And all 3 of my kids are now equipped with laptops, and streaming music and movies around the house via the 10/100 network. And while it was generally coping well, occasionally things would get a little over-whelmed.

So this month, I upgraded.

I replaced the old switch with a shiny new 24 port 10/100/1000 switch from TP-Link, and the original WRT54GL (acting as my access point) got upgraded to an 802.11n 300Mbps dedicated access point, again from TP-Link. The result has been pretty impressive. All congestion on the wired network has now vanished. The new access point definitely has significantly better range (thanks to its MIMO set up) now easily reaching my garage, has eliminated all but one of the wireless black-spots around the house, and improved the wireless speed pretty much everywhere.

And since my WRT54GL has been freed up by this exercise, I’ve been able to relocate it close to the wireless black-spot, add a second cell to my wireless network, and completely eliminate that last remaining black-spot too.

Of course, now I have no excuse for not completing my wireless sensor hub project…

Building a wireless sensor hub

Those of you not thinking “A what?” are probably already thinking “Why?”; the best answer I have is mostly just because I can. Of course, there was a little more to it than that. I had several little problems or projects in mind, and this had the potential to solve several of them at a stroke.

So, what were these problems? Well, I wanted to:

• extend my home network into the garage
• be warned if the power failed in the garage
• monitor the temperatures both outside, in the garage, and inside the garage freezer
• have the option to install a 1-wire weather station on the garage roof

What I needed was a small wireless computer system, with a couple of ethernet interfaces, capable of bridging my home wireless network back onto wired ethernet in the garage. It also needed to run some simple scripts to report it’s status over the wireless network, and be capable of managing both a 1-wire network, and any other sensors that I might need in the future. Clearly it would be a bonus if it were to be cheap.

And it turns out that thanks to the ubiquitous nature of Linux, and the inquisitive nature of the worlds hacker community, it’s perfectly feasible to do all that with a single, cheap off-the-shelf box; a Linksys WRT54GL wireless router. Which I just happen to have lying around spare. Of course, it also needs a special third-party version of the firmware in combination with a little bit of reconfiguration to act as a wireless bridge. And it also needs some “minor” hardware alterations to enable some hidden serial ports, allowing a simple (but incorrectly drawn!) serial to 1-wire adapter to be added inside the routers case. So it’s not exactly trivial, but the price is definitely right, and it makes for an interesting project for a rainy Sunday. Or two.

Earlier this week I ordered all the parts to enable the serial ports, and to construct the serial to 1-wire adapter. Once they arrive I’ll combine the two circuits onto a single small piece of stripboard, which I can then install within the router case, exposing the 1-wire interface through a panel-mounted socket. In the meantime I can at least get on with the other aspects of the conversion.

So today I replaced the firmware, updated the configuration so the system acts as a wireless bridge, and installed software on both the router and my home server to allow my home server to interrogate the 1-wire sensors attached to the router, over the wireless network. Until I modify the router hardware and attach some sensors I can run the software on the router in a test-mode, which simulates a range of sensors, allowing me to test the rest of the solution.

Taking this in more detail:

1. I chose to use OpenWRT as my replacement firmware. This is fairly similar to a standard Linux distribution, but has the advantage of a simple web-based GUI (called Luci) for management. Better however, is a replacement web-based GUI called X-Wrt, who package up their GUI and the base OpenWRT firmware together, in ready-to-flash images. For my WRT54GL, I needed the image at http://downloads.x-wrt.org/xwrt/kamikaze/8.09.2/brcm-2.4/default/openwrt-wrt54g-squashfs.bin.
   This can be used with the standard Linksys firmware “upgrade” to reflash the router. If you have already reflashed the router with a third-party firmware then you’ll typically find that you must use a .trx file. Fortunately you can convert .bin files to .trx files, as .bin files are simply .trx files with a header on the front. Usually the header is 32 bytes long; you can tell by examining the .bin file, and looking for the start of the .trx data, which starts with “HDR0″:
   
richard@t60p:~/Downloads$ hexdump -C openwrt-wrt54g-squashfs.bin
00000000 57 35 34 47 00 00 00 00 09 0c 1d 04 47 01 55 32 |W54G........G.U2|
00000010 4e 44 00 00 1f 00 00 00 00 00 00 00 00 00 00 00 |ND..............|
00000020 48 44 52 30 00 10 21 00 f3 aa 87 16 00 00 01 00 |HDR0..!.........|
00000030 1c 00 00 00 04 09 00 00 00 c0 07 00 1f 8b 08 00 |................|
00000040 00 00 00 00 02 03 a5 57 4d 6c 5b 59 19 3d be ef |.......WMl[Y.=..|

   You can then do the conversion with the command:
   
dd if=openwrt-wrt54g-squashfs.bin of=openwrt-wrt54-squashfs.trx bs=32 skip=1

2. Having loaded the firmware, the router will reboot. You must then telnet into the router using one of the LAN ports. The router will have an IP address of 192.168.1.1. At this stage there is no password, so set one using the passwd command, which will allow you to use SSH and the web-based GUI.
   You now need to disable the telnet service. You can do this most easily from the web-based GUI; surf to http://192.168.1.1/index.html. Disable the telnet service, and then stop it. Your telnet connection will be dropped. Log back into the router using SSH instead; ssh root@192.168.1.1, entering your new password.
3. You now need to configure the networking on the router. This is done using the files /etc/config/network and /etc/config/wireless. Back them both up, and then change them as follows:
   
# /etc/config/network
#### VLAN configuration
config switch eth0
option vlan0 "0 1 2 3 5*"
option vlan1 "4 5"
#### Loopback configuration
config interface loopback
option ifname "lo"
option proto static
option ipaddr 127.0.0.1
option netmask 255.0.0.0
#### LAN configuration
config interface lan
option type bridge
option ifname "eth0.0"
option proto dhcp
# -or-
# option proto static
# option ipaddr an-ip-on-your-subnet
# option netmask your-subnet-netmask
# option gateway your-gateway-ip-address
# option dns your-dns-server-ip-address
#### WAN configuration
config interface wan
option ifname "eth0.1"
option proto dhcp

   and:
   
# /etc/config/wireless
config wifi-device wl0
option type broadcom
option channel auto
config wifi-iface
option device wl0
option network lan
option mode sta
option ssid yourssid
# psk = WPA Personal, psk2 = WPA2 Personal
# wpa = WPA Enterprise, wpa2 = WPA2 Enterprise
option encryption psk
option key "your key here"

4. Now disable and stop the dnsmasq and firewall services using the web-based GUI. Failing to do this will prevent the bridge from passing your DHCP requests through to your home network correctly.
5. Reboot the router, and it should connect to your wireless network, requesting a dynamic IP address for its bridge interface. In my case, I configured my main DNS/DHCP server to always give my WRT54GL the same “dynamic” IP address whenever it connects, making it easier to access. You can achieve the same effect by defining the bridge interface with an appropriate static IP address if you prefer.
6. I then SSH’d back into the router, and updated the package management system by running the command opkg update. I was then able to install OWServer on the router by simply entering the command opkg install owserver. As I have no 1-wire hardware for OWServer to examine, for the time being I use it in simulation mode, using the command owserver –fake=05,10,21 –foreground, which allows me to test my home servers connectivity to the OWServer software.
7. On my home server, I downloaded http://sourceforge.net/projects/owfs/files/owfs/2.7p29/owfs-2.7p29.tar.gz/download, which I saved locally. I then untarred this, and compiled (as a simple test) the OWShell programs. Do a configure in the root of the untarred directory, and then change down into owfs-2.7p29/module/owshell, and run the commands:
   
make
sudo make install

   I can now do owdir -s router-host:4304 and be presented with a response from the OWServer running on the router. From here its a short step to getting the Perl and PHP language bindings created, allowing me easily script up access to the 1-wire network hosted on the WRT54GL. But first I need to get some real sensors attached to the router…

Digitising DVD’s with Linux to stream to PS3

Having posted about converting the format of video files to suit streaming to a PS3, I got an email asking me how I actually convert a DVD into a digital file, and how my network is set up to stream videos to the PS3.

So, a little more explanation. I have a small, very low-power server that runs 24×7 in my house, and is connected to my 100mbps ethernet network. On that server I have quite a lot of disk storage, and run Mediatomb, which is a DNLA-compliant UPnP media server. It serves my collection of video and audio files to any device on my network that wants to access them.

In the case of the video files, that device is my PS3, which is also connected to my ethernet network, and also has an HDMI connection to my LCD TV. With this configuration I can start the PS3, which automatically detects the Mediatomb media server and puts an icon on its GUI interface. I can then select that icon, get a list of the available video files, and by clicking on one, have it played on my TV for me.

The only configuration involved in this solution is of Mediatomb, which involves a couple of documented changes to its configuration file, and specifying where my media files are via its web interface. All very simple.

To create the media files from a DVD, I do the following:

1. Extract the content of the DVD onto my hard drive using a program called vobcopy.
2. Use ffmpeg to convert that content into a more compact form, better suited to streaming over a network.
3. Copy the resulting file to my server where Mediatomb can access it.

Now lets look at the first two steps in more detail:

DVD’s actually store their content as a series of VOB‘s; these contain the actual video that you see when you play back a DVD. In general there are separate VOB’s for the main movie, any adverts, trailers, bonus features, etc etc. And just to make life a little more complex, some DVDs store the main movie in more than one VOB, though fortunately vobcopy can hide that from us.

To make things a little more difficult, the movie industry have then encrypted the DVD to make it harder to do what we are about to do. To get around this, you must have installed libdvdcss2, which is a DVD decryption library that is available from the Medibuntu repository.

To find which VOB to extract from the DVD, simply run “vobcopy –info”. This will produce some output like:

richard@t60p:~$ vobcopy –info
Vobcopy 1.1.0 – GPL Copyright (c) 2001 – 2007 robos@muon.de
[Hint] All lines starting with “libdvdread:” are not from vobcopy but from the libdvdread-library

[Info] Path to dvd: /dev/sr0
libdvdread: Using libdvdcss version 1.2.10 for DVD access
[Info] Name of the dvd: STARWARS2UK_D1_2_PAL
[Info] There are 21 titles on this DVD.
[Info] There are 103 chapters on the dvd.
[Info] Most chapters has title 1 with 51 chapters.
[Info] All titles:
[Info] Title 1 has 51 chapters.
[Info] Title 2 has 1 chapter.
[Info] Title 3 has 2 chapters.
[Info] Title 4 has 1 chapter.
[Info] Title 5 has 1 chapter.
[Info] Title 6 has 1 chapter.
[Info] Title 7 has 1 chapter.
[Info] Title 8 has 1 chapter.
[Info] Title 9 has 1 chapter.
[Info] Title 10 has 2 chapters.
[Info] Title 11 has 10 chapters.
[Info] Title 12 has 14 chapters.
[Info] Title 13 has 1 chapter.
[Info] Title 14 has 1 chapter.
[Info] Title 15 has 2 chapters.
[Info] Title 16 has 1 chapter.
[Info] Title 17 has 4 chapters.
[Info] Title 18 has 2 chapters.
[Info] Title 19 has 2 chapters.
[Info] Title 20 has 1 chapter.
[Info] Title 21 has 3 chapters.

[Info] There are 21 angles on this dvd.
[Info] All titles:
[Info] Title 1 has 1 angle.
[Info] Title 2 has 1 angle.
[Info] Title 3 has 1 angle.
[Info] Title 4 has 1 angle.
[Info] Title 5 has 1 angle.
[Info] Title 6 has 1 angle.
[Info] Title 7 has 1 angle.
[Info] Title 8 has 1 angle.
[Info] Title 9 has 1 angle.
[Info] Title 10 has 1 angle.
[Info] Title 11 has 1 angle.
[Info] Title 12 has 1 angle.
[Info] Title 13 has 1 angle.
[Info] Title 14 has 1 angle.
[Info] Title 15 has 1 angle.
[Info] Title 16 has 1 angle.
[Info] Title 17 has 1 angle.
[Info] Title 18 has 1 angle.
[Info] Title 19 has 1 angle.
[Info] Title 20 has 1 angle.
[Info] Title 21 has 1 angle.
[Info] Using Title: 1
[Info] Title has 51 chapters and 1 angles
[Info] Using Chapter: 1
[Info] Using Angle: 1

[Info] DVD-name: STARWARS2UK_D1_2_PAL
[Info] Disk free: 74104.160156 MB
[Info] Vobs size: 5733.919922 MB

It’s highly likely that the VOB with the most chapters is the main movie; in this case title 1. We can check that by running the command “mplayer dvd://[vob_number]“. If we see the main movie playing then we can extract that VOB to our hard disk by running the command “vobcopy –title-number [vob_number]“. vobcopy will then proceed to decrypt and copy that VOB to your hard disk (as STARWARS2UK_D1_2_PAL3-1.vob in this case).

Now we can convert that (very large) file into something smaller and more easy to stream. This uses exactly the same command as the last post; ffmpeg. This time however, we need to make sensible guesses for the bitrates that we want to use for the video and audio streams. Personally I go with 2000k for the video, and 192k for the audio. It’s good enough in terms of quality, and produces a file ~1/3rd the size of the original VOB, which is much more amenable to being streamed over a 100mbps ethernet network. If you’re hoping to do this over wireless, then you’ll probably need to compress even more and sacrifice quality … wireless just doesn’t have the bandwidth to do good quality video streaming.

So, the command to convert that VOB to a .mp4 file is:

ffmpeg -i STARWARS2UK_D1_2_PAL3-1.vob -vcodec libx264 -b 2000k -acodec libfaac -ab 192k STARWARS2UK_D1_2_PAL3-1.mp4

That command will take at least as long to execute as the movie would have taken to play. But once completed, the resulting file can then be copied to my server and be available for instant access in the future.

Hampshire Linux User Group meeting follow-up

The non-working Vodafone SIM problem has been resolved. It turns out that Vodafone don’t simply have contract mobile broadband and PAYG mobile broadband, they also have “Top Up and Go” mobile broadband. Which seems to be some sort of PAYG mobile broadband with some port blocking, implemented to stop you using VoIP or P2P services.

Anyway, that’s the SIM that I have.

Knowing that, a simple google on the settings of the various offerings shows that the APN, userid and password are different from the normal PAYG offering. Changing my connection properties in Network Manager to use the correct settings resulted in an immediate connection to the internet @ 3.6Mbps, though admittedly I could see the Vodafone mast from where I was connecting.

My next problem is to work out how to top up my credit as I use the service. Vodafone appear to have set this up so everything is done through the software that they supply to make their dongle work on Windows. Which of course, I don’t (and can’t) use, as I’m running under Linux. I was really hoping that I’d be able to check my credit & top it up via the Vodafone website, but it’s looking like it’s pretty difficult to register the SIM on their website unless you’re running their desktop software, which requires Windows. And if you’re not running Windows there is very limited support, which is rather frustrating.

I’ll keep working on it for a while longer, but it’s starting to look like I might have to install Windows and their dashboard software on a spare partition just to get the SIM registered on their website, which is not exactly ideal.

Update: I finally managed to work out how to register the SIM on the Vodafone website so that I can check my credit balance, and top up the credit on the SIM using my credit card. The website is quite large and complex, and there seems to be more focus on advertising and upselling of products to their customers than on the usability of the core function of managing existing products, which didn’t help much. Still, I got there in the end.

As a Linux user, wanting to register a “TopUp and Go” SIM without resorting to Windows, this is the process I suggest you follow:

• Register for “My Account” on the Vodafone website. This simply gives you an account on their website, but doesn’t associate any products with your account.
• Now put your data SIM into an ordinary (Vodafone) mobile phone.
• Enter *#100# and press send. The network will now tell you what mobile phone number is associated with your data SIM.
• Login to your account and register a “Pay As You Use” phone. The website will ask you to enter the phone number of the phone. Enter the phone number you obtained in the previous step.
• The website will now send you an SMS message, containing a secret code. You can read this on your mobile phone.
• Enter the secret code back into the website (proving you own the SIM you are registering), and the website will ask you for the remaining information it needs to associate the SIM with your account. It automatically works out that you have a “TopUp and Go” data SIM rather than a standard PAYG mobile phone SIM.
• You can now see your credit limit by logging into your account.
• You can also associate a credit card (or your bank details) with your “My Account”, and use it to top up your SIM as required.
• Remember to put your data SIM back in the USB dongle, and your voice SIM back in your mobile phone.

Shredded spam

It’s been a few days since I posted about hardening my email server, and I thought I’d just mention that since then I have received exactly one spam email. Which is pretty good, given that they seemed to be mounting up at about a hundred a day before that.

Update: I just checked the logs, and as of 12th May, the number of spam emails that I’ve rejected per day since implementing this on the 3rd May:

Date	Total blocked	Spamhaus blocked
May 3rd	56	22
May 4th	49	23
May 5th	70	28
May 6th	88	34
May 7th	78	33
May 8th	63	29
May 9th	98	32
May 10th	50	32
May 11th	76	34

So less than I thought, but definitely more than I (or my family) want to receive and then deal with manually! Incidentally, the numbers I blocked by using the Spamhaus block lists proves to be about 50% of the total, making them a pretty useful resource.

However, I was somewhat surprised that that one had managed to get past my defences when all the others had failed, so I had a closer look at the email and the headers that describe it’s path through the internet, and on inspection it’s clearly one-man-and-a-dog based somewhere in Essex trying to phish details of Abbey National building society accounts, using what appears to be a normal email client. It’s definitely not your normal bulk spam emailing anyway.

I’m vaguely curious about how he got hold of my email address, but not enough to do anything much about it, so I’ve simply added him to a blacklist on the server, and he is now history too. As the A-Team used to say, “I love it when a plan comes together!”

“Spam, lovely Spam, wonderful Spam.”

SPAM is a tinned meat product, that judging from its longevity, must be enjoyed by some people. Spam on the other hand is universally hated, and apparently now accounts for over 95% of all email that is sent, or in excess of 200 billion emails a day. 80% of those are sent by one of only 200 or so spammers around the world. As a result of this the IT industry wastes an inordinate amount of time, money and resources filtering out the vast majority of this rubbish before it ever reaches the end-users of most email systems. The former became associated with the latter by way of a 1970′s sketch from Monty Pythons Flying Circus.

Why do I mention this? Well, for historical reasons, I run my own mailserver here at home. This has advantages and disadvantages. The main disadvantage is that setting one up is quite complex, and getting it exactly right is essential, otherwise you can easily set up what is known as an open relay, which allows anyone to send email using your email server – and the people who would quickly take advantage of this are spammers. A faster way to get your ISP connection terminated is probably hard to find.

The advantage for me was that it was the only way I could provide economic email for my family while I was using the IBM-supplied broadband package. Now that I have switched to my own ISP the need is not so pressing, but we’d all need to change our personal email addresses, and update the many people who use them. So it’s just easier for me to keep running the system, especially as it’s been a completely trouble-free experience for the last 3 or 4 years.

Except when I went to collect my personal email this morning, rather than the 2 or 3 emails I’d normally expect I found well over a hundred emails, all of which were spam, advertising the types of products that I’d mostly rather not even know about, let alone take advantage of. I have to take quite enough drugs already thanks – no need for any more, especially from unknown, unverifiable sources.

Why the sudden increase? Well, to be honest, it’s partly my own fault. I set up a secure email server that can’t be used to relay email, but I didn’t make any effort to make it particularly resilient to someone who wanted to harvest my email addresses and then target them with spam. And we’ve been flying under the radar ever since. Except it looks like someone has lowered the radar; they’ve obviously scanned my IP address, found an email server, tried to use it as a open relay (and failed) and then rather than moving on, have worked out the email addresses that my server supports, and put them onto a large number of mailing lists. Gee, thanks a lot.

So my job this afternoon is to significantly upgrade the configuration on my email server to ensure that I filter out all the spam that is starting to arrive. After all, its bad enough that I should see some of this rubbish, let alone have it being sent to my wife and daughters too.

UPDATE:

Well, as seems to often be the way at the moment, I didn’t mange to get started on this when I expected. But when I did get around to it (mid-evening instead) I was surprised just how much work has been put into my mailserver (postfix) to make it trivial to harden. The server is incredibly modular, with open interfaces that make it easy to hook up to external “helper” applications. By installing a couple of extra packages using the Ubuntu package management system I was then able to make about 20 lines of changes to my postfix configuration files that should stop almost all the spam that I’ve been receiving. The only slight concern is that I may have hardened the server to the point where I occasionally discard some good email too – something to watch out for.

But so far so good; I still pass the open relay checkers on the web, and appear to be successfully sending and receiving normal emails. Just no sign of any more spam. Which I am currently interpreting as being a good thing.

Tuesday spent “tinkering”

Yesterday I decided it was time to work out exactly what is going on with my home server. This was one of those little jobs that I’d decided wouldn’t require much physical effort, and that could be picked up and put down again as required when I felt tired. In short, ideal for my current situation.

The main problem is that I have three large SATA drives in the server, that I want to run as a RAID 5 array, but the motherboard I am using (in common with almost all other mini-itx motherboards) only supports two SATA drives. Originally I tried to work around this by using a PATA to SATA converter, but found that it resulted in some odd errors and what appeared to be data loss. So I bought a PCI card with a pair of SATA ports, based around the SiL3124 chipset, to provide me with support for a total of four SATA drives. However, any drive connected to that PCI card started showing errors whenever the I/O rate went above some arbitrary level, making it completely unsuitable for inclusion in a RAID array.

So yesterday I replaced the SiL3124 PCI card with another based on the VIA VT6421, which provides 3 x SATA ports and an IDE port, and since I had the server in pieces, took the opportunity to add an IDE to Compact Flash adapter, along with an 8GB Compact Flash card.

My intention was to be able to install the operating system onto the Compact Flash card, and then use the 3 x SATA drives purely as a RAID 5 array for data storage. By installing the operating system onto the memory card (with no moving parts that wear out) I would minimise the chance of any failures, making the system more reliable. The RAID 5 array would provide redundancy for my data storage, allowing a drive to fail without impacting my data.

However, in the tradition of plans laid by mice and men, when I got everything updated, the drive connected to the new PCI card still exhibits strange I/O errors. And swapping the drives around still leads to errors only from the drive connected to the PCI card. Which leaves me with the frustrating conclusion that it’s probably something to do with the implementation of the PCI bus on the motherboard – which is not what I was hoping for.

This somewhat Frankensteinian server has worked well for the last couple of years, but I’m starting to think that given how much of my home now depends on it working smoothly, it may be sensible to actually buy a new server designed specifically for the task, rather than wasting precious time chasing down obscure hardware issues in my DIY creation. At the moment, this one looks just about perfect (apart from the price … gulp …) would allow me to reuse my existing drives, and can probably even be made to boot from my compact flash card still. Decisions, decisions!

Update: I upgraded to Jaunty (9.04) recently, and the problem with PCI-based SATA cards has magically fixed itself. So it looks like this really was a problem with the way the Linux kernel and it’s SATA drivers were interacting with the hardware, rather than the hardware itself. Now I just need to find the time to take the server down, and grow the RAID1 array into a RAID5 array. It’s back on the todo list, albeit as a low-priority item.

Fast, Faster … broken

Back in this post I talked about my move to Be Unlimiteds ADSL2+ service, and how I was using my own router, a Linksys AG241 with some custom firmware, to obtain Annex M support (a facility where you can sacrifice some downstream bandwidth to obtain more upstream bandwidth).

For the last 3 weeks or so it’s all been working just fine, but when my wife and daughters came to see me the other day, they reported that “the internet wasn’t working”.

When they got home I talked my eldest daughter through some simple diagnostics, which showed that everything was working fine, apart from the AG241 router, which despite showing all the expected flickering lights on the front panel, was no longer live on the network.

A quick power-cycle has resolved the problem, and everything is now working properly again, but this leaves a question mark in my mind over the long term reliability of the custom firmware. Was this just a one off, perhaps caused by some sort of power brown-out, or was this the result of the AG241 running out of memory because of a memory leak somewhere? I guess time will tell, but in the meantime, I’m starting to look for a really good ADSL2+ router with extensive customisation features, and rock solid reliability.

Because I really don’t want to have to power-cycle my router every couple of weeks

Wireless broadband internet – fixed

An exchange of emails with the friend who has lent me his 3G USB modem revealed that there is no PIN number on the SIM. However, apparently these things can be a little awkward, and error messages can be obscure, especially when there is no 3G or GPRS signal.

Reassured that I’m not about to lock myself out of the network by getting the SIM PIN number wrong too many times in a row, I did a little debugging. The modem may be recognised as a USB device, but it most definitely is not recognised as a modem. A quick check on my kernel configuration parameters reveals that I’ve not included a couple of crucial drivers.

15 minutes recompiling and reinstalling my kernel, insert the USB stick, and Network Manager cuts in, and establishes a 3G connection to the Internet. It really was as simple as that.

So expect updates from my hospital bed in due course