Hampshire Linux User Group meeting follow-up

The non-working Vodafone SIM problem has been resolved. It turns out that Vodafone don’t simply have contract mobile broadband and PAYG mobile broadband, they also have “Top Up and Go” mobile broadband. Which seems to be some sort of PAYG mobile broadband with some port blocking, implemented to stop you using VoIP or P2P services.

Anyway, that’s the SIM that I have.

Knowing that, a simple google on the settings of the various offerings shows that the APN, userid and password are different from the normal PAYG offering. Changing my connection properties in Network Manager to use the correct settings resulted in an immediate connection to the internet @ 3.6Mbps, though admittedly I could see the Vodafone mast from where I was connecting.

My next problem is to work out how to top up my credit as I use the service. Vodafone appear to have set this up so everything is done through the software that they supply to make their dongle work on Windows. Which of course, I don’t (and can’t) use, as I’m running under Linux. I was really hoping that I’d be able to check my credit & top it up via the Vodafone website, but it’s looking like it’s pretty difficult to register the SIM on their website unless you’re running their desktop software, which requires Windows. And if you’re not running Windows there is very limited support, which is rather frustrating.

I’ll keep working on it for a while longer, but it’s starting to look like I might have to install Windows and their dashboard software on a spare partition just to get the SIM registered on their website, which is not exactly ideal.

Update: I finally managed to work out how to register the SIM on the Vodafone website so that I can check my credit balance, and top up the credit on the SIM using my credit card. The website is quite large and complex, and there seems to be more focus on advertising and upselling of products to their customers than on the usability of the core function of managing existing products, which didn’t help much. Still, I got there in the end.

As a Linux user, wanting to register a “TopUp and Go” SIM without resorting to Windows, this is the process I suggest you follow:

  • Register for “My Account” on the Vodafone website. This simply gives you an account on their website, but doesn’t associate any products with your account.
  • Now put your data SIM into an ordinary (Vodafone) mobile phone.
  • Enter *#100# and press send. The network will now tell you what mobile phone number is associated with your data SIM.
  • Login to your account and register a “Pay As You Use” phone. The website will ask you to enter the phone number of the phone. Enter the phone number you obtained in the previous step.
  • The website will now send you an SMS message, containing a secret code. You can read this on your mobile phone.
  • Enter the secret code back into the website (proving you own the SIM you are registering), and the website will ask you for the remaining information it needs to associate the SIM with your account. It automatically works out that you have a “TopUp and Go” data SIM rather than a standard PAYG mobile phone SIM.
  • You can now see your credit limit by logging into your account.
  • You can also associate a credit card (or your bank details) with your “My Account”, and use it to top up your SIM as required.
  • Remember to put your data SIM back in the USB dongle, and your voice SIM back in your mobile phone.

Shredded spam

It’s been a few days since I posted about hardening my email server, and I thought I’d just mention that since then I have received exactly one spam email. Which is pretty good, given that they seemed to be mounting up at about a hundred a day before that.

Update: I just checked the logs, and as of 12th May, the number of spam emails that I’ve rejected per day since implementing this on the 3rd May:

Date Total blocked Spamhaus blocked
May 3rd 56 22
May 4th 49 23
May 5th 70 28
May 6th 88 34
May 7th 78 33
May 8th 63 29
May 9th 98 32
May 10th 50 32
May 11th 76 34

So less than I thought, but definitely more than I (or my family) want to receive and then deal with manually! Incidentally, the numbers I blocked by using the Spamhaus block lists proves to be about 50% of the total, making them a pretty useful resource.

However, I was somewhat surprised that that one had managed to get past my defences when all the others had failed, so I had a closer look at the email and the headers that describe it’s path through the internet, and on inspection it’s clearly one-man-and-a-dog based somewhere in Essex trying to phish details of Abbey National building society accounts, using what appears to be a normal email client. It’s definitely not your normal bulk spam emailing anyway.

I’m vaguely curious about how he got hold of my email address, but not enough to do anything much about it, so I’ve simply added him to a blacklist on the server, and he is now history too. As the A-Team used to say, “I love it when a plan comes together!”

“Spam, lovely Spam, wonderful Spam.”

SPAM is a tinned meat product, that judging from its longevity, must be enjoyed by some people. Spam on the other hand is universally hated, and apparently now accounts for over 95% of all email that is sent, or in excess of 200 billion emails a day. 80% of those are sent by one of only 200 or so spammers around the world. As a result of this the IT industry wastes an inordinate amount of time, money and resources filtering out the vast majority of this rubbish before it ever reaches the end-users of most email systems. The former became associated with the latter by way of a 1970’s sketch from Monty Pythons Flying Circus.

Why do I mention this? Well, for historical reasons, I run my own mailserver here at home. This has advantages and disadvantages. The main disadvantage is that setting one up is quite complex, and getting it exactly right is essential, otherwise you can easily set up what is known as an open relay, which allows anyone to send email using your email server – and the people who would quickly take advantage of this are spammers. A faster way to get your ISP connection terminated is probably hard to find.

The advantage for me was that it was the only way I could provide economic email for my family while I was using the IBM-supplied broadband package. Now that I have switched to my own ISP the need is not so pressing, but we’d all need to change our personal email addresses, and update the many people who use them. So it’s just easier for me to keep running the system, especially as it’s been a completely trouble-free experience for the last 3 or 4 years.

Except when I went to collect my personal email this morning, rather than the 2 or 3 emails I’d normally expect I found well over a hundred emails, all of which were spam, advertising the types of products that I’d mostly rather not even know about, let alone take advantage of. I have to take quite enough drugs already thanks – no need for any more, especially from unknown, unverifiable sources.

Why the sudden increase? Well, to be honest, it’s partly my own fault. I set up a secure email server that can’t be used to relay email, but I didn’t make any effort to make it particularly resilient to someone who wanted to harvest my email addresses and then target them with spam. And we’ve been flying under the radar ever since. Except it looks like someone has lowered the radar; they’ve obviously scanned my IP address, found an email server, tried to use it as a open relay (and failed) and then rather than moving on, have worked out the email addresses that my server supports, and put them onto a large number of mailing lists. Gee, thanks a lot.

So my job this afternoon is to significantly upgrade the configuration on my email server to ensure that I filter out all the spam that is starting to arrive. After all, its bad enough that I should see some of this rubbish, let alone have it being sent to my wife and daughters too.


Well, as seems to often be the way at the moment, I didn’t mange to get started on this when I expected. But when I did get around to it (mid-evening instead) I was surprised just how much work has been put into my mailserver (postfix) to make it trivial to harden. The server is incredibly modular, with open interfaces that make it easy to hook up to external “helper” applications. By installing a couple of extra packages using the Ubuntu package management system I was then able to make about 20 lines of changes to my postfix configuration files that should stop almost all the spam that I’ve been receiving. The only slight concern is that I may have hardened the server to the point where I occasionally discard some good email too – something to watch out for.

But so far so good; I still pass the open relay checkers on the web, and appear to be successfully sending and receiving normal emails. Just no sign of any more spam. Which I am currently interpreting as being a good thing.

Tuesday spent “tinkering”

Yesterday I decided it was time to work out exactly what is going on with my home server. This was one of those little jobs that I’d decided wouldn’t require much physical effort, and that could be picked up and put down again as required when I felt tired. In short, ideal for my current situation.

The main problem is that I have three large SATA drives in the server, that I want to run as a RAID 5 array, but the motherboard I am using (in common with almost all other mini-itx motherboards) only supports two SATA drives. Originally I tried to work around this by using a PATA to SATA converter, but found that it resulted in some odd errors and what appeared to be data loss. So I bought a PCI card with a pair of SATA ports, based around the SiL3124 chipset, to provide me with support for a total of four SATA drives. However, any drive connected to that PCI card started showing errors whenever the I/O rate went above some arbitrary level, making it completely unsuitable for inclusion in a RAID array.

So yesterday I replaced the SiL3124 PCI card with another based on the VIA VT6421, which provides 3 x SATA ports and an IDE port, and since I had the server in pieces, took the opportunity to add an IDE to Compact Flash adapter, along with an 8GB Compact Flash card.

My intention was to be able to install the operating system onto the Compact Flash card, and then use the 3 x SATA drives purely as a RAID 5 array for data storage. By installing the operating system onto the memory card (with no moving parts that wear out) I would minimise the chance of any failures, making the system more reliable. The RAID 5 array would provide redundancy for my data storage, allowing a drive to fail without impacting my data.

However, in the tradition of plans laid by mice and men, when I got everything updated, the drive connected to the new PCI card still exhibits strange I/O errors. And swapping the drives around still leads to errors only from the drive connected to the PCI card. Which leaves me with the frustrating conclusion that it’s probably something to do with the implementation of the PCI bus on the motherboard – which is not what I was hoping for.

This somewhat Frankensteinian server has worked well for the last couple of years, but I’m starting to think that given how much of my home now depends on it working smoothly, it may be sensible to actually buy a new server designed specifically for the task, rather than wasting precious time chasing down obscure hardware issues in my DIY creation. At the moment, this one looks just about perfect (apart from the price … gulp …) would allow me to reuse my existing drives, and can probably even be made to boot from my compact flash card still. Decisions, decisions!

Update: I upgraded to Jaunty (9.04) recently, and the problem with PCI-based SATA cards has magically fixed itself. So it looks like this really was a problem with the way the Linux kernel and it’s SATA drivers were interacting with the hardware, rather than the hardware itself. Now I just need to find the time to take the server down, and grow the RAID1 array into a RAID5 array. It’s back on the todo list, albeit as a low-priority item.

Fast, Faster … broken

Back in this post I talked about my move to Be Unlimiteds ADSL2+ service, and how I was using my own router, a Linksys AG241 with some custom firmware, to obtain Annex M support (a facility where you can sacrifice some downstream bandwidth to obtain more upstream bandwidth).

For the last 3 weeks or so it’s all been working just fine, but when my wife and daughters came to see me the other day, they reported that “the internet wasn’t working”.

When they got home I talked my eldest daughter through some simple diagnostics, which showed that everything was working fine, apart from the AG241 router, which despite showing all the expected flickering lights on the front panel, was no longer live on the network.

A quick power-cycle has resolved the problem, and everything is now working properly again, but this leaves a question mark in my mind over the long term reliability of the custom firmware. Was this just a one off, perhaps caused by some sort of power brown-out, or was this the result of the AG241 running out of memory because of a memory leak somewhere? I guess time will tell, but in the meantime, I’m starting to look for a really good ADSL2+ router with extensive customisation features, and rock solid reliability.

Because I really don’t want to have to power-cycle my router every couple of weeks 😦

Wireless broadband internet – fixed

An exchange of emails with the friend who has lent me his 3G USB modem revealed that there is no PIN number on the SIM. However, apparently these things can be a little awkward, and error messages can be obscure, especially when there is no 3G or GPRS signal.

Reassured that I’m not about to lock myself out of the network by getting the SIM PIN number wrong too many times in a row, I did a little debugging. The modem may be recognised as a USB device, but it most definitely is not recognised as a modem. A quick check on my kernel configuration parameters reveals that I’ve not included a couple of crucial drivers.

15 minutes recompiling and reinstalling my kernel, insert the USB stick, and Network Manager cuts in, and establishes a 3G connection to the Internet. It really was as simple as that.

So expect updates from my hospital bed in due course 🙂

Wireless broadband internet

As I’ve already mentioned, I’m hoping to be able to take my Aspire One into hospital with me. Assuming that I’m well enough, then from my perspective, it will be the ideal distraction from boredom. I’ve got 20GB of music on it, and I’ve just been given another 10GB of plays and drama etc that have been recorded from the radio. I’ve even got a digital TV/radio tuner, which ought to let me watch a bit of live TV if I want.

However, what would be really cool would be to be able to continue updating this blog from my hospital bed. And its now looking like that might be possible, as I’ve just been loaned a Vodafone USB 3G modem for the time that I’m going to be in hospital – thanks Andy!

Unfortunately, it doesn’t work with the custom kernel that I’m currently running to help improve the performance of the One. So it looks like I will need to spend some time tweaking the configuration of my custom kernel again. In addition, when I switched back to the “normal” kernel, the modem still doesn’t work, failing this time at the request for the SIM pin.

So, I guess I need to resolve the kernel compilation, and find out either what the pin code is for the SIM card, or what I’m doing wrong… 🙂

Fast, Faster … oooops. Fastest, eventually.

My IBM broadband cut off first thing this morning, just as I was about to grab my email.

My existing Linksys WAG200G ADSL2+ wireless gateway has been a surprisingly nice router, but isn’t capable of supporting Annex M, which I want to use to double my uplink speed on the Be service. So out it came, and in went a Linksys AG241, which has been reflashed with some custom firmware to allow it to support Annex M. It is actually more configurable than the WAG200G, with a few more routing functions, at the expense of there being no wireless. As my wireless is implemented via separate WAP’s anyway, this is not an issue.

I configured the settings according to the depressingly sparse information that came from Be Unlimited with their “BeBox” router. Needless to say, it didn’t work. After much fiddling, and still no success, I unpacked the BeBox, disconnected the rest of my network, and hooked it up. It immediately made a connection to the Internet, which for a moment or two made me think that perhaps I should just go with the flow, and stick with it.

But then I looked at the configuration menus, to work out how to set it up to work with the rest of my system. Or rather I should say the lack of menus. Selective port forwarding? Nope. QoS? Not that I could see. Custom firewall rules? No chance.

Anyway, with the Internet back up it was easy enough to find the mistake I’d made in setting up the AG241 (I’d picked the wrong DSL modulation setting), so I swapped the BeBox back out, put the Linksys back in, changed the DSL modulation, and now my home network is back up and working smoothly again. And the BeBox is back in its box.

Finally, by setting my SNR Margins to 3dB I’m now getting sync speeds of 17,176Kbps down, 2,648kbps up, and running a speed test gave me throughputs of 14,670kbps down, and 2,234kbps up. Which is about 7 times faster than the IBM service. We’ll see how reliable it is with that (relatively) low SNR Margin … if it holds the connection overnight then I’ll stick with it, but if it turns out to be unreliable I can switch it back up to 6dB, and lose only about 10-20% of my throughput. Which is a small sacrifice to have reliability.

Internet TV

A colleague pointed me at get_iplayer the other day, and I spent a couple of hours this weekend playing with it. Without a doubt it is the easiest way I have come across for getting hold of content from the various UK “play it again” services for TV and Radio.

BBC TV programme downloads are in H264 video and AAC audio, and although they are not high definition – more like good VHS quality – the convenience is spectacular. I sucked down a couple of programmes from the BBC; an episode of Being Human, and an episode of Doctor Who, which took about 15 minutes each (on my current 2Mbit connection). Both were downloaded as Quicktime .mov files because the feed that get_iplayer uses is designed for the iPhone.

I then converted them to .mp4 containers (using the command ffmpeg -i input.mov -vcodec copy -acodec copy output.mp4) and stored them on my NAS. From there, they were streamed (using the Mediatomb DNLA server) to my PS3 over my wireless network, which was able to play them back onto my flatscreen TV. And it worked beautifully.

Except for the stuttering. And the occasional unexpected pause. Turns out that despite my PS3 seeing 85% WiFi signal strength, it can’t consistently transfer more than about 0.5Mbit/s over my home wireless network. Which is not enough for video streaming. A quick check with a long Cat5e cable shows no signs of stuttering or pausing, so this is clearly a problem with the wireless drivers in the PS3. So it looks like I will need to hurry along my plan to drop some more ethernet lines to the back of the TV area of the lounge. Except of course, that isn’t going to happen for a few months now 🙂

However, notwithstanding the networking issues, the potential here is great. I can write some webpages on my NAS that I can access from my PS3, on the flatscreen TV in the lounge. That will run get_iplayer and return a list of available programmes, which I can select to download and have stored on my NAS, where they will then be converted to .mp4 format which I can stream to the PS3, so I can watch them. When my fast connection comes online (hopefully tomorrow!) that should take no more than 2 or 3 minutes for an hour-long TV programme.

Internet TV … almost on demand.

Fast, Faster, Fastest…

For the last couple of years I’ve been on a 2Mbit corporate ADSL deal. The advantage is that it has been funded for me (aka “it’s free”), and it’s fast enough for most of the things I need to do while at home, but the drawback is that it has no mail service, which means my family can’t easily have any email addresses. To solve this I simply set up my own email server on my home server. And then discovered that the entire IP address range for this service has been blacklisted by Spamhaus, which meant many people simply rejected email from my server on principle. In short, the IBM package was fine for me to use for IBM business use, but a total pain for the rest of my family.

This came to a head just before Christmas, and so I’m now in the process of migrating off the IBM deal and onto my own broadband package with Be Unlimited. They offer a rather nice “up to 24Mbit” ADSL2+ service for £20 a month, including a static IP address, no blacklisted IP address ranges, and proper reverse DNS support so I can associate domain names with the service. Best of all, my management have agreed to make a contribution towards my business use of it.

And today I got home from work to find that Be Unlimited had delivered my new “BeBox”. This is a rebrandged Thompson Speedtouch TG585 v7 router, with some customised firmware on it. Sadly it doesn’t have a terribly good reputation for reliability, probably as a result of that customised firmware, so it’s not something I actually wanted. Bizarrely Be Unlimited seem to think that this router is worth £100, and insist that at the end of the contract you either send it back to them, or pay them the £100. Which is even more odd, given that you can buy one of the wretched devices here for only £36 retail. Since I already have a couple of nice ADSL2+ routers, the last thing I needed was another, let alone an unreliable one, but try as I might I couldn’t persuade Be not to send me yet another. For the life of me, I can’t understand why they are so insistent about this, but at least they have a Freepost address to return it to, so I can package it up and send it back to them at no cost just as soon as I’ve proved to myself that their line is up and running properly.

The estimates for my connection speed are somewhere around 17Mbits, and the line should be migrated and activated on Monday. I’ll actually sacrifice a little of that downlink speed for an increased uplink speed, but with luck I’d hope to see somewhere around 16Mbit down and 2.5Mbit up. To be honest, I’m not quite sure what I’m going to use all that bandwidth for yet. Feel free to make some suggestions!