Playing with callerid again

Back in April last year I started playing with an old external serial-attached modem to read the callerid of incoming calls. My intention was to intercept calls from direct marketeers. The concept was good, but I ran into problems with the modem; it took up a lot of space, kept overheating, and lacked any voice facilities, limiting what I could do with it. In addition (probably because of the modem constantly overheating) the software I was running kept crashing.

So in the end, I gave up on the idea.

But recently we seem to have had a spate of annoying calls from direct marketeers based in India, selling products for UK companies that are cynically avoiding the UK’s regulations around direct marketing opt-outs. The straw that broke the camels back was the call that came through at 6am on a Saturday morning.

The problem here is that the phone companies don’t care about this. They make money from all these calls, so its not in their interest to block them. They’ll sell me a service to block “withheld” numbers, but not numbers that show as “unavailable”. Unfortunately, these days the majority of the problem calls come from India, and show as “unavailable” because the Indian call centers are using VoIP systems to place their calls to the UK, and they deliberately ensure that their callerid is “unavailable”.

So I’m back on the idea of making my own solution to this problem. So first off, I purchased a USR 5637 USB fax modem that is compatible with the UK callerid protocols. Even better, this is a voice modem too, so it can interact with a caller, sending audio as well as data down the phone line, and recognise touchtones. It’s also small, self-powered, cool-running and very reliable.

Next I spent some time looking to see what other people have done in this space, and eventually found this webpage, that describes a simple Bash script that intercepts calls that are not on a whitelist, and plays modem tones to them before hanging up. Recognised callers are allowed to ring the phone as normal, progressing to an answerphone if necessary. It’s not exactly the functionality that I want, but the simplicity is beguiling, and it’s trivial to extend it to do something closer to what I do want. And anyway, anything more sophisticated is going to require something like Asterisk, and switching my whole phone system over to VoIP, which is not going to be very family-friendly.

So for now, I’m gathering lists of all incoming calls to establish a basic whitelist, before moving on to do some really basic screening of calls.

Calling home, free of charge

I’ve just been through a period of travel hell; individually each of the the three back-to-back trips are interesting, useful, and in some ways, quite good fun. But they’re back to back. So over a 17 day period, I’ve actually had only 3 days in the UK, and two of those I was still working. Of course, mention business travel to anyone that doesn’t do it, and it brings to mind visions of exotic locations and lavish expense accounts. Whereas the reality tends to be cramped economy class travel, very long working days, and lonely hotel rooms a long way from friends and family.

More importantly it means that just doing the normal day to day family things, like chatting to your kids about their day in school can rapidly become an extortionately expensive luxury that you feel ought to be kept to the brief minimum. Especially when the company that you’re travelling for thinks that it shouldn’t pay for those phone calls – which particularly irks me.

And that got me thinking – I actually have all the facilities I need to enable me to call home to my family for nothing. My company expects me to need an Internet connection in whatever hotel I stay in, and fully funds it. I carry a laptop and an Android Smartphone. In combination with the rather sophisticated phone that I have at home, I can talk to my family for as long as I want for no additional costs, using a technology called VoIP, based on an open standard called SIP.

My home phone is a Siemens Gigaset S685 IP DECT system, with 5 handsets. It’s what the marketing world likes to term “pro-sumer”, by which they really mean it’s got lots of features that most consumers will never use, but it’s not really got enough features for true commercial use. They also mean it’s a bit expensive for a consumer product.

But in this case, we’re talking about a DECT phone that connects to both my home phone line and my home broadband, and can have up to 7 logical phone lines connected to it – the physical “POTS” line, and 6 VoIP connections over the internet. The base unit can support separate conversations on up to 3 lines in parallel, with as many handsets connected to each of those lines as required. Each handset can act as a speaker-phone, or have either a headset or bluetooth earpiece attached to. It can even do call routing, where it looks at the number you dial, and decides which line to place your call on. In short, it’s absolutely packed with features.

The key to the free calls home is of course, the VoIP lines, because as of version 2.3, Android can also make VoIP calls. The trick is simply getting the two to talk to one another, while they are in different countries.

So first you need to find a SIP provider for your home phone and your Smartphone. The best way that I’ve found to do this is to set up a SIP-based virtual PBX. You create a free account with them (your PBX) and then add users to your PBX. Each user is given their own SIP credentials so they can logon, and you can associate an extension number with each of those users, allowing them to easily exchange calls – which is exactly what I need, as (unlike my Android smartphone) the handsets on my old Gigaset cannot call SIP URI’s directly.

The first provider I came across that allows this is OnSip, in the USA. My experience with them has been good, but there are many others out there too. Unfortunately it’s not perfect – for me, there are a couple of quirks with their service. Firstly, they don’t seem to expect anyone outside the USA to be using their service, so I cannot enter my address correctly. Which I suspect is technically a breach of their T&C’s. And secondly, it means that all the call tones you’ll hear when using this (ringing, engaged, unobtainable etc) will be American rather than British. I can live with that, but if you choose to go down this route too, DO NOT ENABLE THE E911 OPTION. You could theoretically call out US emergency services to whatever pseudo-fictitious address you have registered with them, which would not be good.

To make it work:

1. Register with your chosen free SIP PBX provider. I’ll assume OnSip, who will ask you to register a userid, then validate it by email, before letting you set up your first PBX.
2. 

   Registering for OnSip PBX

   When you come to define that PBX, you’ll see a screen similar to this one, asking for initial configuration information. You can call the domain name (which is essentially the Internet name of the PBX) anything you like. Incidentally, changing that country field seems to do nothing. It doesn’t even change the format of the information they collect for your address or (real) phone number.
3. 

   Creating a new user

   Having now got a SIP PBX available, we can add some users to it. Each user is roughly equivalent to a telephone on a normal PBX, but in this case the user accesses the PBX by way of some SIP credentials. The users can be called via those credentials, or (if the caller is another user of this PBX) the extension number that is associated with that user. This happens irrespective of the device that the user is using those credentials on, or its location.
4. 

   List of virtual PBX users

   After entering a user for the Gigaset (my house phone) and one for my smartphone, I have a PBX with two users associated with it. I’ve obscured some critical pieces of information to maintain some privacy, but fundamentally the OnSip system gives me a set of SIP credentials for each “user” of the system (bottom right on the image), and associates an extension with them too.
5. Next we need to get the Gigaset to register with the SIP PBX. To do this, logon to the Gigaset web interface, select the “Settings” tab, and then navigate to Telephony, Connections.
   

   
   
   Gigaset Settings

   Now define a new provider by clicking on one of the “Edit” buttons as seen in the first of these screenshots. This will bring up the basic settings page seen in the second screenshot. Into this screen enter the data found on the OnSip user configuration screen under the Phone Configuration heading. Copy the OnSip username field into the Gigaset username field, the OnSip Auth Username field into the Gigaset Authentication Username field, the OnSip SIP Password into the Gigaset Authentication Password field, and then click the “Show Advanced Settings” button, which will extend the form with the additional fields seen in the third screenshot. Now copy the OnSip Proxy/Domain field into the four Gigaset fields: Domain, Proxy Server Address, Registrar Server, and Outbound Proxy. When you save the settings the Gigaset will attempt to register with the OnSip PBX. Sometimes this takes a few seconds. You may need to refresh the browser to see the status change to Registered.
6. Now we need to make the Android Smartphone register to the OnSip PBX too. To do this, bring up the Android Settings by pressing “Menu” from the home screen, and tap “Settings”.
   

   
   
   Android SIP setup

   Navigate to the Call Settings, and tap it to reveal the screen in the first screenshot.
   Tap on “Use Internet Calling” and set it to “Ask for each call”. Then tap on Accounts to bring up the Internet Calling (SIP) Accounts screen where we can enter SIP accounts associated with the phone. See the second screenshot.
   Now add a new account for the OnSip PBX by tapping “New Account”; this will bring up a screen like the one shown in the third screenshot, into which you need to enter your credentials.
   The content of each of the fields (from the OnSip phone configuration information) should be obvious by now. When you save the account you will want to check that the registration information is correct. The easiest way to do this is to enable the “Receive incoming calls” option (back on the first screenshot again), which will force Android to register all it’s accounts with their associated providers. If you get an error then either the provider is down (possible) or the settings are incorrect (more likely). Leaving this option enabled forces Android to keep those connections active, which runs all the radios, and eats the battery, but allows incoming VoIP calls to your Smartphone (say from the Gigaset). In my experience it’s too power-hungry to use in this mode, other than very sparingly. Fortunately you can make outgoing calls from the Smartphone without enabling it.
7. 

   Android Internet Calling enabled Contact

   Finally you need to define a contact in the Smartphone contacts to use with Internet Calling. As all my contacts are normally managed by Lotus Traveler for me, which has no concept of Internet Calling, I defined a new local-only contact that is not synched with any of my Accounts (ie, Google or Lotus Traveler) and used that. Enter the name of the contact as normal, then scroll all the way to the bottom of the contact, where you will find a “More” section. Expand that, and continue to scroll to the bottom, where you will find a field for “Internet call”; into that simply enter either the OnSip SIP URI of your Gigaset, or it’s OnSip extension number.

Note that this really only works when connected to a reasonably good quality WiFi network. Poor quality networks seem to give quite variable results. Sometimes they still work, other times one end or the other may experience audio problems and/or dropped calls. It seems to work just fine through both the IBM and my home firewalls, even at the same time. I’ve not checked the actual audio codecs in use, but sound quality is subjectively better than a normal cellular call. Neither Android or the Gigaset seem to do silemce suppression (ie injection of white-noise when no-one is speaking) so the pauses in a conversation are totally silent, which can be slightly disconcerting.

Normally if you want to dial the Smartphone from the Gigaset you would need to indicate to the phone that it should send the call over the appropriate VoIP provider. This quickly becomes a pain, but it’s easy to set up a simple dial plan on the Gigaset so calls that start “700” (remember I made my extensions be 7000 and 7001?) go out over the OnSip VoIP connection automatically, which makes the solution really easy to use (ie family-friendly) from the Gigaset.

Finally, there is a really interesting technology called iNum available. Sadly it’s not (as far as I can tell) implemented by any of the major telecoms ISPs in the UK, that when combined with SRV records in special DNS records, would allow some really cool automatic rerouting of calls over different networks to different endpoints, depending on context. In theory the network could understand whether I was at home or out with my mobile, and route the call appropriately. It could also do smart things like look at the inbound number and the time, and route business calls to voicemail out of office hours, but still let personal calls through as normal.

If only it were implemented.

Tesco Internet Phone service – RIP

Despite some annoying initial problems, I’ve been successfully using Tesco’s Internet Phone service since May of last year. This is a SIP-based service, costing £2.89 a month, that provides me with an additional phone number (in my case it appears to be in Southampton) and up to 3000 minutes of “free” calls to UK geographic (01, 02 and 03) phone numbers. Given that my main desire was for an inexpensive second line, it’s a pretty good deal; around half the monthly cost of any of their competitors, and devoid of the plethora of startup and termination charges that (say) Vonage impose.

In combination with my VoIP capable Siemens DECT system, it’s been great.

Unfortunately, as of the 27th April, the service will be closed down. Which is most frustrating. So now the hunt starts for a new VoIP supplier that offers a broadly comparable service.

Siemens S685IP update

Having been running these phones for a few months now, I made a fresh discovery the other day. The handsets have bluetooth support built into them. This appears to have two main benefits:

• Firstly, it could allow you to synchronise the handsets phone directory using a computer to handset bluetooth connection. This is of little interest to me, as I discovered (in this post) that its possible to synchronise each handsets directory using the base station’s web interface, and an appropriately formatted text file of VCARD entries.
• And secondly, and of much more interest to me, you can associate a bluetooth earpiece to the handset. Now, of itself this doesn’t sound like terribly exciting technology. After all, you could do this with a mobile phone years ago. But being able to do it with my home phone handset is rather novel, and given the number of times I find myself on fairly long conference calls where using a mobile phone is expensive, and I need to type on a keyboard, or want to go make a cup of tea, or just stretch my legs, this facility is proving to be a real boon.

It means that on those interminably long conference calls, rather than sitting with the handset held to my ear, or being attached to the handset by a corded earpiece, I can simply use my mobile phone’s bluetooth earpiece instead. Combined with the beltclip on the Siemens handset, this gives me complete hands-free freedom on my home phone system. I can walk around within about a 10m range of the handset itself, or clip the handset to my belt and wander up and down the street while talking on my bluetooth earpiece, and keeping my hands completely free for other tasks. It’s marvellous.

So score one for the Siemens handset designers. I guess it cost a bit more to squeeze the function in (these are not cheap handsets) but the usability is superb.

VoIP update

I’ve been meaning to provide an update on the Tesco Internet Phone (“TIP”) saga for a while, but never quite got around to it. The last you heard from me was here, where I was waiting for a voucher code. So, with apologies for the tardiness, here’s the current state of play:

TIP sent me the voucher code through the mail; it took about a week to arrive, but it did actually arrive. Entering the code into their website caused the promised upgrade to take place, allowing me to upgrade my Pay As You Go account to a pay monthly all inclusive UK calls plan. This works quite well, though the quality of the call is generally no better than a normal POTS line, and sometimes lower, which is rather disappointing. But then it is only costing £3 a month, so I don’t feel I can complain too much 🙂

I now have my Siemens Gigaset phones set up with both the POTS line and the TIP line, and they are working well. Incoming calls on the POTS line ring 4 of the handsets, while an incoming call on the VoIP line rings just the other handset. Unanswered POTS calls go to the POTS voicemail, while unanswered VoIP calls go to the in-built answering machine. This setup means the kids can have a handset dedicated to them, with the rest of the handsets remaining “ours”. Outbound calls currently default to the VoIP line (ie, free) but overriding this and using the POTS line (eg, when calling a non-geographic number, or if the VoIP line is already in use) is as simple as holding down the call button rather than just briefly pressing it.

Calls to numbers that would incur charges (non-geographic) are currently barred on the VoIP line, but there is a simple call routing system built into the phones which looks like it could be sufficient to automatically route such calls to the POTS line, which is currently unrestricted. That’s something I need to spend an hour or two playing with.

Update: 10 minutes this morning have shown that the dialing plan function built into the Gigaset is very basic, but enough to allow me to effectively route outgoing calls between the POTS and VoIP lines. So, calls to 07* and 08* numbers now go via the POTS line, as do local calls (because the VoIP service needs the full STD code, and none of us remember to enter it!) and the access codes for the POTS answerphone. 09* calls (premium rate stuff) are now blocked too.

This isn’t perfect as it will make it less simple to use both outgoing lines simultaneously, but at least it makes the phones work the way everyone expects them to, which stops the frustrated shouts of “The phones aren’t working again!”. Once I get a better dial plan working (ie Asterisk) then I can route everything by cost & line availability, adding in STD codes for local calls as necessary.

Update 2: It occurred to me that if we had all our frequently called local numbers entered in the handsets phone directories, with the STD areas codes pre-filled in, then almost all our calls would go via the free VoIP line. It also turns out that you can provision the handsets attached to the Gigaset base-station over the network using a file of VCARDs. The system is very picky about the format of the files, but once that was cracked (it requires file with MSDOS line terminations, not Unix ones) it was a simple but tedious job to create the 140-odd VCARD entries, and about 5 minutes to provision all the handsets.

And this simple solution now all works pretty much perfectly.

Meanwhile my plans to integrate Asterisk are still ongoing. I have my Asterisk 1.6 installation up and running, and defining another VoIP service on the phones to connect to the Asterisk server is trivial. I just need to spend more time on understanding, configuring and documenting a basic Asterisk system. As the moment I feel I’ve over-complicated the issue by adding the Asterisk GUI to the system, as this seems to do a lot of automatic configuration of the system, complicating the system considerably and making it much harder to understand. Removing this, and getting back to basic configuration files seems like the right way forward here. The good news is that the way the Gigaset system implements VoIP connections means that I can connect specific handsets to my Asterisk server, and whilst they are easy for me to access, they will not interfere with the (concurrent) normal use of the handsets. Which is probably a good thing for family harmony!

So, in the short term it’s working well, but longer term Asterisk can clearly add a whole new level of intelligence to the system, if only I can get my head around it all.

Tesco Internet Phone Service … sort of

And at last, after 5 days of not being available (!) the registration part of the Tesco Internet Phone Service (I’m going to call them “TIPS” for short) finally came back from the dead and started working again. So I dutifully registered myself, picked a phone number in the local area, and selected their “All day UK” inclusive package for £2.89 a month.

They then sent me an email with confirmation details, and an activation URL, which then triggered the credit-card authentication processing, and as if by magic, I am the proud owner of a new phone line.

Well, sort of.

Despite selecting the “All day UK” package, and them taking all my credit-card details, it turns out that they’ve provisioned me on a “Pay as you go” account.

Grrrr.

So I called up the helpline (somewhere in India) and talked to a nice chap, who tells me that to turn my “Pay as you go” account into the requested “All day UK” account, I just need to enter the code from the upgrade voucher that was included in the box with my Tesco phone.

AAAAARRRRGGGGGHHHHHH!

I point out that I don’t have a Tesco phone, and that in fact, I’d already told them I was planning to use their service with non-Tesco hardware when they asked me that as part of the registration process. Can the chap in India just flip the bit in their database to enable my account? No, but he tells me I can buy a £10 “Pay as you go” top-up in-store, which will also have an upgrade voucher in it.

I’m starting to get a bit tetchy at this point, and make clear to them that I don’t want to pay £10 to enable the account that I thought I’d already registered for. So the nice man offers to send me an upgrade voucher through the mail. It will take about a week to get to me, if that’s OK?

I ask if he can’t just read me the damned code so I can enter it into the website while I have him on the phone? No. All he can do is have one sent to me.

Words fail me. Do they want my business or not for &deities. sake? Anyway, since I have some time on my hands, I’ve just added a letter to Terry Leahy to my todo list… I can’t believe an aggressive, modern retailer would stand for a farce like this.

In other respects, the service looks like it could be quite good, if I can ever get them to properly provision it for me. The inbound number works well, the call quality seems perfectly fine, and there is built in voicemail too. You just have to get past the bizarre sign-up process.

DECT phones arrive

Back in this post (was it really that long ago?) I laid out my plan to create an all-singing, all-dancing phone system for the home, with the first step being to get some Siemens Gigaset S685IP DECT phones. Having made the decision, and told you all about the plan, I’d then had a serious bout of prevarication over the price of the things. At £260 they are definitely not what you could call the “budget” end of the market.

However, earlier in the week I noticed that IBM had decided to file another of my patent disclosures, and a previous patent filing has finally been issued by the US patent office. The awards that IBM gives me to recognise those events will more than cover the cost of the phones, so I flexed the credit card, and ordered them from LiGo, who are based in Scotland. The ordering process was trivial, with lots of updates on the order progress, with the phones finally turning up yesterday in the form of a base unit and pre-registered handset plus four additional (unregistered) handsets.

As someone who has never had DECT phones before, I have to say that they are nothing like the DECT phones that I remember looking at (admittedly many years ago). They bear more resemblance to (chunky) mobile phones, in both appearance and capabilities. They also no longer require custom battery packs, which I seem to remember always being a problem with early DECT phones. These just use a pair of (supplied) AAA NiCad rechargeables.

Impressions of the handsets are that they are a nice size, weight and “feel”, with more features than most people (including me!) will ever need. And the internal (paging) call clarity is simply amazing, probably as a result of the wide-band digital codecs that they are able to use. It will be interesting to see what the call quality is like over the PSTN and VoIP networks as I start to get everything hooked together.

The base station on the other hand, is an extremely flimsy affair with a combination of ports on the side and rear, which makes it rather difficult to neatly cable up – especially to the house LAN. However, since it’s being tucked out of sight in our case, that probably doesn’t matter too much.

Being NiCad powered, I charged all the batteries up overnight last night and now have four of the handsets running internal speakerphone calls to fully discharge the batteries again, at which point we can start using them normally. In theory this should minimise any memory effects in the batteries and give a longer service life, though I’m dubious of the actual value in practice. Meanwhile the fifth handset is acting as our main phone, and will have to be discharged separately once the other handsets are conditioned. According to the manuals, talk time is about 10 hours or so, so we should be ready to start playing more seriously with the phones this evening.

So I really need to do some research this morning into which VoIP provider I’m going to select – probably either Tesco or Vonage – and get an account or two arranged.

Update: The Tesco offering is now £2.89 a month for “unlimited” calls of up to 70 minutes duration to geographical UK numbers, ie, those starting 01, 02 or 03. Beyond 70 minutes, there is a 2p per minute charge. There is also a monthly fair use limit of 3000 minutes (about 33 hours) on those “unlimited” calls. Calls to mobiles are about 10p per minute, and non-geographic numbers are subject to complex rules to determine the costs. International calls vary significantly too … from 2p per minute upwards depending on destination. The inbound number can be chosen from a long (but not complete) list of area codes. For me, the nearest area code to my house is Southampton, which is close enough to count as local to anyone likely to want to call me.

Overall it looks like it will meet my needs, so I’ll probably go with it, though at the moment the Tesco website is failing to allow new registrations, so I can’t sign up until they get that fixed (they’re aware and “urgently working on it”)

Home PBX

Back in this post I mentioned that I was going to teach myself Asterisk, as part of a project to build a home PBX system. I thought it would be useful to outline what I’m trying to achieve, and why, before I start posting progress reports that might make little sense otherwise.

So, at the moment I have a single PSTN line into the house, which in addition to providing my phone service, also carries my ADSL service. Inside the house I have a series of extensions to various rooms, on which we currently have two old-fashioned wired telephones. Voicemail is provided at the exchange by our service provider. This works OK as far as it goes, but with five of us in the house, contention over the use of the one phone line is becoming an issue. In addition, despite being on the UK telephone preference service “opt-out” list, we still seem to get a lot of annoying cold-callers, mostly dialing in from abroad where UK legislation cannot affect them.

So, in an ideal world, my plan would be to initially add 2 more lines to the house using VoIP technology to alleviate the contention. Both would have geographic numbers associated with our local PSTN exchange, to allow me to nominally declare one inbound VoIP line for the children to share, and one for me to use primarily for when I’m working at home. However, outbound calls would make use of any of the available lines based on availability and a least cost routing algorithm.

I’d also replace the two wired telephones with several DECT handsets. My intent would be that each handset would operate independently of the others, and the system would support multiple parallel active calls. In addition, I would want to be able to conference together multiple handsets and/or multiple lines as needed.

Voicemail would be provided internally by the system, with individual accounts for each user, with messages forwarded to the user by email if required.

The system would also need to do time-based routing of incoming calls, so that after a specific time at night, all calls would go straight to voicemail, unless they come from a known and authorised number.

International callers, and callers with no CLI would always get trapped by the system and passed to an IVR system that would allow the caller to identify themselves. If they identify themselves as an authorised number then they would be passed on to the phones/voicemail, otherwise the call would be automatically terminated.

To create such a system I figure I need something like the following:

• My existing PSTN line
• 2 x geographic numbered VoIP lines
• Asterisk to handle the signalling, with a fairly complex configuration
• 1 x Linksys SPA3102 linking the PSTN line to Asterisk, and providing a fallback wired PSTN phone on one of its FXS ports
• A Polycom Kirk Wireless Server 300 to provide the interface between Asterisk and the DECT handsets
• A PoE injector to power the KWS300
• As many DECT handsets as eventually needed

However, the outlay on that is a little on the high side, and the learning curve very steep indeed before I’d get to a working phone system. So I propose to do this in two phases, and start with:

• 1 x PSTN line
• 2 x geographic numbered VoIP lines
• 1 x Quint package Siemens Gigaset S685IP

The Siemens Gigaset can handle some of my requirements out of the box, allowing me to get a basic working system up and running. It can also be interfaced to Asterisk, allowing me to start experimenting with Asterisk, before adding the SPA3102 and optionally upgrading the S685IP base station with the Kirk Wireless Server, reusing the handsets from the S685IP package as needed.

Well, it’s the current plan anyway 🙂