Into The Unknown

... No, I'm not talking about zombie movies shown at the now-defunct chain of UK cinemas. Smartypants.

Although I'm not sure how it'll turn out, it's still fun to follow the progress of a film while it's in production - and in this case, I'm just glad to see that someone's making the effort to try something new. What am I on about? The world's first User-Generated-Content Zombie Movie, that's what. Oh, and BBC Three, ever keen to appeal to the 16-30 tech-aware demographic, is running a series around the making-of of this entirely.

The person behind it all? An enterprising soul by the name of Bryony, and she's documenting her progress right up to the film's release date on Hallowe'en on both the BBC Three site and her own blog. Not as big budget as the thought-provoking A Swarm Of Angels project, but sometimes the low-budget, more unpredictable projects yield the most exciting results.

Hurray for the interwebs!

While I move house, here's two videos you may find interesting:

Vint Cerf on video distribution
Internet pioneer looks at video's future
Vint Cerf, vice president and chief Internet evangelist for Google, speaks at the Personal Democracy Forum in New York Tuesday about his vision of a solution for video distribution via downloads. He says online video will be distributed in broadcast mode.
Length: 08:11

Daily Debrief: How to break the P2P logjam
Video: Hogging web traffic with P2P transfers
CNET News.com's Charles Cooper speaks with Webware Editor in Chief Rafe Needleman about what one of the founding fathers of the Internet is doing to resolve the growing problem of peer-to-peer transfers.
Length: 04:23

Right, time to uproot the PCs. They're always last to leave the house!

Ok, these thoughts have been simmering on my brain's back burner for a while now. In preparation for my open letter to the music industry, I thought I'd warm up with a little brainstorming of my own, laid out in public for all to see.

We're hardly short of recent developments in the digital music arena: RealNetworks is launching an MP3 online music store under the Rhapsody brand, teaming up with the music discovery service iLike and US mobile provider Verizon - most likely with an aim to compete against AmazonMP3 and iTunes Plus. Warner Music has finally relinquished and agreed to distribute its music catalogue on Nokia's "Comes With Music" digital platform - less than a year after withholding its content from the same platform due to 'concerns over illegal downloading' (...your guess is as good as mine on this one). The $5-a-month RED music subscription service launched to much fanfare, with the promise of raising money to combat AIDS... And the BPI proudly announced that label revenue, outside of traditional direct music sales, is up 14% this year (t0 £121.6m), which makes a change from the cries of doom and gloom we usually hear.

(Oh, and Prince is suing people again for ludicrous reasons, which just amuses me.)

You might think that this is all well and good, and that things are slowly looking up - but I continue to be disappointed by the lack of progress the industry is making in the online sector. In fact, I think the current situation we have is actually worsening the situation for everyone, and it frustrates me. As a result of this, in my free moments, I've been pondering a solution which I think is quite workable, would benefit all involved in the industry (from labels to artists) and wouldn't be that hard to implement - and I'll explain. In due course.

But first, an overview of the new RealNetworks MP3 service... Their DRM-free platform is a step in the right direction, but I still find myself left wanting. From the DownloadSquad article covering the launch;

The Rhapsody MP3 store has music from all four major labels, with over 5 million tracks available for download. Most songs are priced at 99 cents, and most albums cost $9.99. That's about the same price that Apple charges for DRM-free AAC audio files, but a bit more than Amazon MP3 charges for many songs and albums. All songs will be encoded at 256kbps, and will be playable on any device that can handle MP3 audio. RealNetworks isn't killing off its DRM-restricted music service, but rather, plans to have the two services peacefully coexist. You can pay $12.99 a month to stream unlimited music to your computer, or you can pay per download to save songs that you can play forever.

RealNetworks is also rolling out a service that will let Verizon Wireless customers download music for their handsets. For $15 a month, users will be able to download an unlimited number of songs (with DRM) on a Windows PC and sync those songs with their cellphone. Currently seven handsets are supported, with several more coming soon.

The Rhapsody MP3 store is offering a $10 credit to the first 100,000 customers who purchase an album by July 4th. You need to sign up for an account and fork over your credit card information to qualify. But hey, free music, right?

All four majors... To the rest of us, that's Warner, Universal, SonyBMG and EMI. 'Right, 256kbps MP3 - that's great quality,' you might think, but you'd be wrong. The quality of online offerings is slowly increasing, but at a much slower rate of change than both the storage industry's developments in raw storage capacity and the computing industry's developments in raw computing power. Once again, an industry artificially stymied by the lack of willingness of its participants to move with the times. That said, whilst a 256kbps bitrate isn't ideal, I agree that this higher quality audio is better for both the customer and the industry as a whole (which, aside from a couple of notable exceptions, still seems somewhat reticent to up the quality of its digital music offerings - although there are no downsides to doing so already). But (and it's a big but) anyone who buys an MP3, WMA or iTunes AAC file when the CD is available is an idiot for helping to perpetuate this flawed business model. Why do I yammer on so much about buying CDs? Well, it boils down to quality and ability. Why does the quality matter so much? Because we as consumers need to be future-proofed. Until an online music store offers their music in a truly CD-quality format, the industry is doomed to repeat itself ad nauseum, and we'll never make any real progress in moving our music consumption from the expensive (for everyone) physical world to the higher-profit, digital world.

Why do people still buy CDs? Well, they're one of the only truly future-proofed formats left, as they can be encoded to whatever format is the format du jour. Want to put some of your classic 2004 rock music onto your 2-terabyte, thumbnail-sized digital music player, using the best format around at the time? Sure thing, just rip the tracks from the CD, encode them, and plop them onto your device. Sorted. You have the ability to do so at any time, and the quality of the original is sufficient to do so too. You wouldn't dream of reencoding an MP3 or WMA file - it would sound even worse than it did the first time.


I will not be satisfied with the retail digital music industry until they stop exclusively dealing in lossy audio formats. The general buying public have the herd mentality, unfortunately - they sometimes can't see past their nose, and this has been happening for a few years now. Put simply: Rhapsody is a con. iTunes is a con. AmazonMP3 is a con. We7 is a con. Any and all online digital music stores that don't offer PCM audio or lossless download options are, when push comes to shove, blithely conning their customers. Linn Records is breaking the mould in the classical arena - in some cases offering albums in 24/96 FLAC 'studio master' quality, higher quality than the retail CDs(!) - and they should be both encouraged and heartily congratulated for going against the grain. Similarly, electronica retailer Beatport has offered PCM audio downloads at an acceptable price point for some time now, as have a few niche labels' own web sites - unfortunately nowhere near enough sales to even make a dent on current consumer trends in the digital music retail market.

The Big Four labels, and the major online retailers they sell through, might counter that they are mainly responding to demand for the most popular formats like MP4-AAC, MP3 and WMA with the current online store offerings - but who is going to take that next step and dare to be radically different by offering a service BEFORE the market asks for it? If that step isn't taken, the market will never realise that it's both feasible and reasonable to expect it. We have a classic chicken and egg situation, except knowledgeable people and the labels themselves already know the answer - the labels just seem to want to ignore it for a while. Sticking one's head in the sand was never a bright idea, especially in this day and age.


By the way, if you're wondering what lossless audio is - logically, it's the exact opposite of lossy audio... And MP3, AAC and even Ogg are all lossy formats. (For some info, read this and for an excellent primer - with pictures! - read this ExtremeTech article.) The reason MP3s are so small is because when they are encoded, not all of the original audio information is retained - the codec uses clever techniques to 'recreate' the missing/lost portions of the audio spectrum, and each codec has its own algorithms which result in either increased efficiency under certain circumstances or improved filesizes. Lossless audio formats are formats like FLAC, ALAC and Monkey's Audio, which are still digital music files (just like MP3s in this respect) but which contain ALL of the original audio information from the source, compressed in such a way that none of the information is lost forever. So, if you create a FLAC file of a track you've ripped from a CD, you effectively have an identical, 1:1 copy of that audio in said FLAC file. This would not be so had you made an MP3 instead. If you burn that track to an audio CD from the FLAC file, the track on the CD you've just burnt it will sound absolutely identical to the original because it effectively IS the original. The beauty of digital copying at its simplest (and the root cause of the music industry's past decade of squirming, too).

MP3s, no matter how well they've been encoded and decoded, are never quite the same as the original - they might sound slightly different, the audio spectrum will definitely be different from the original and you will, to put it bluntly, have lost some of the sound. I know this is an oversimplification, but think of an MP3 - even a high quality one - as a fresh tape cassette recording from a CD. You'll never have the track in the same quality you would have if you had the CD in your hands.



So, why am I going on about quality? Well, because I think we still pay far too much for what we get, and the market has a bit of a cartel effect going - there is no way to shop around, like there is for physical CDs. The music labels are pushing mandatory minimum percentages onto the distributors as a final act of desperation, meaning the retailers have to enforce an artificially high price for their music in order to even make a half-decent profit for themselves. As a result of this, barring the few online music stores that work on either a subscription basis (eMusic) or a popularity basis (like the indie-centric Amie.st) there is no disparity in prices between one online music store and another.

Look at the price point for buying an MP3 from one of the panoply of legal download sites. Between 79 and 99 cents? 79p for an iTunes file (99p for iTunes Plus) if you're in the UK? For what... A 'nearly-there' copy of a track? It might sound ok through your crap iPod headphones, but what happens when you want to play it on your good soundsystem at home? What happens if you get one of those nice networked streaming audio devices like a Squeezebox or a Sonos? You want as high quality as possible to get the most from your music, but you're stuck with your awful quality MP3s.

What happens in a couple of years when my precious music collection, which I've paid for of course, becomes complete obsolete - just because it's all been encoded at a low bitrate in a format which is more than a decade old? If I had the CDs, I could just reencode them in a better format. An increasing amount of people, myself included, are buying albums, ripping them and encoding them to lossless audio formats such as FLAC, and then storing away the original purchases to make sure that they don't get scratched / broken / lost. Sites like MP3Tunes offer you secure, private online lockers in which you can archive very high quality audio for streaming to wherever you are at that time, be it work, home or elsewhere. (Incidentally, EMI have once again hilariously misinterpreted MP3Tunes' core concept, and are trying to sue the company out of existence under the misguided assumption that it is intended to aid piracy, instead of promote responsible safeguarding of your own music. Send EMI hatemail until they see sense.)

Companies are marketing devices which store your music in archive quality and make it instantly accessible over your local network, or even over your Internet connection to any device which has web access - this is just the tip of the iceberg. Even my old iRiver H140 MP3 player can play FLACs with a bit of custom firmware - I just load them on via its USB cable as I would do an MP3 or an Ogg file, and I can listen to them straight from its internal hard drive. I can also play them on my PC just like I'd do with MP3s.


Still with me? Good, let's talk shop. Let's work in US dollars for the moment so we can do some comparisons. If we look at the sweet spot for album pricing from a US high street retailer, we can see that the most popular range is between $10 and $14. (this is according to Best Buy's site, which lists the amount of items in each category). Coldplay's new album is on sale for $13.99. (From Amazon.com, it's $9.99, but I'll use the RRP here even though in reality few online retailers actually use it as a guide price). The album has 10 tracks on it, and at $0.99 each, the 256kbps download from Rhapsody clocks in at $9.90. For albums with more than 12 tracks on, buying the physical album can actually be cheaper than buying the same album as a digital download. (11x$0.99 = $11.09). With an increasing amount of stores even offering free delivery on all purchases, many are realising that the instant gratification itch can wait to be scratched for a day or so. But yet, we are still left wanting something... Something a little more flexible, a little more value for money, a way to get the true CD quality without going to the hassle of ordering the CD (or travelling to the store to pick it up). 'Surely it must be possible,' and you'd be right.


The problem with today's digital music stores

If you buy all your music from iTunes - or indeed any online music store locked into a proprietary format, with or without DRM, you're tieing yourself into a locked-down system which mandates that you store your music in a particular format. Unluckily, with some (Apple's iTunes) you're even tied to one manufacturer's device to listen to your music unless you pay a premium on top of the already-expensive regular prices - and the quality isn't even that good for your money. MP3 stores aren't much better, although it's definitely an improvement in terms of being able to move your music to whichever device you happen to own at the time, and you can create (albeit lower quality) CDs to play in your car. The latter also goes for iTunes-purchased music, but it's not an excuse for using a proprietary format (or a lossy one, for that matter).

Amazon MP3, iTunes and a few other providers have absolutely no technical excuse for only offering lossy files - when labels deliver their catalogue to these providers, they have to supply it in lossless format. iTunes Producer rips CDs to PCM audio and uploads it directly to Apple; Amazon MP3 accept CDs in the post, PCM audio or FLAC audio with the appropriate metadata accompanying the files, as do a lot of other providers. Why? Well, so they can reencode the tracks to higher quality... or different formats... should the need arise. If you leave it up to them, you are at their mercy. If you demand the highest quality audio in the first place, you can do as you like when you like. Whatever happened to consumer choice?



So, here's my solution for the digital music retail industry.

Offer all music sold via this new online service in one format, and one format only: FLAC. FLAC is open-source, meaning it's free for everybody to use, and it's cross-platform. The FLAC software and FLAC files are useable on Mac, PC, Unix, Linux and a host of other platforms. No 'vendor tie-in'. The format works on agreed standards too, so both PCs and portable devices can incorporate the standard safe in the knowledge that it won't change in six months' time - so you can get in-car stereos with CDR support and FLAC playback ability! Several albums' worth of music in true CD quality on one data CD which will play in my car and on my PC at home? Excellent.


So, you sell all your music in FLAC format. You also sell the audio for the same price as MP3s are currently sold for: 79-99 cents. You offer whole-album prices. Offer the whole album for a fixed price, which always works out slightly less buying the album's tracks individually and is around the 9-11 dollar mark. However, you still let your customers go a la carte and pay per track. There are a couple of advantages to this: some people might want only a couple of tracks from an album, so that option isn't removed from them. Some people might already have the CD, but they may be trying to encode the album to MP3 for their MP3 player and tracks 2 and 6 might be on a part of the CD which has been too badly scratched (from years of being thrown about in a CD wallet in the car!) - so now the customer can just buy and download FLAC copies of tracks 2 and 6, make their MP3s and make another CDR of the original album for their archive. You get more revenue from an extra, otherwise impossible sale, and the customer has their music in the format they want. Perfect.


You cannot charge a big premium for this service, and the labels will have to come to terms with this. There is, of course no DRM on this service, but DRM is dead, and has been for a long time - it was the great placebo of online digital music retail for too long. Labels might be scared by this - truly CD-quality audio, no DRM... I can already hear the cries of "truly CD quality music available with no protection? All it needs is one person to buy it and share it with everybody else, and we've lost all our revenue!" What do I say to this? Of course they will you idiot - but then, music fans have always done this. Sites like Project Playlist, along with all the existing music blogs, are a magical way to discover new music for yourself (and unfortunately, like any groundbreaking service which dares to be different, the industry can often be found doing its utmost to sue them out of existence).

You only fool yourself if you believe this pattern of usage to be unusual. Almost everyone has either lent or copied a CD of theirs for to a friend, particularly if they are introducing said friend to an artist for the first time. The sharing of purchased music amongst circles of friends is intrinsic to the success of an artist. MP3s made this a whole lot easier, but this practice didn't suddenly start in the 90s.

So, FLACs, no DRM, same price point as current MP3/WMA online offerings.


What is the icing on the cake? Well, you devise a platform which not only provides the digital audio for the core price, you also provide the artwork - and you can charge a little more for this (say, $1/£1) but ideally, it's included in the price of the music so everyone gets it for free. Why? Well, this is the icing on the cake. Let me explain my thought process here.

If I want to buy a new CD, I usually have two options:

1. Go to a shop, buy the CD and go home.
2. Go to a shop's web site, buy the CD and wait for it to be delivered.

Both of these are not particularly efficient ways of getting what I wanted - they involve travel, waiting and cost to both the retailer and myself (travel costs, all the usual costs of running a retail shop and everything else). I might want my audio in as high a quality as I can achieve, but I'm not 100% bothered about the quality of the packaging - it's not like it affects the sound of the music, it's just a bit of pretty artwork. So, you market the service as "bring your own jewel case. We supply the rest."

Here is the killer app: you create your own, pre-scored blank template pages, onto which artwork can be printed and directly placed inside an empty jewel case. You then offer the high quality artwork in a PDF-based file, formatted to the correct size so that it fits exactly inside a jewel case, and you provide it with the audio files at the time of purchase. You write a simple-to-use program for PC, Mac and Linux which streamlines the whole process of printing off the artwork - either the customer can buy some of this custom, blank-template paper, or use their own A4 sheets (and a pair of scissors) - and they print out their own high quality, full colour artwork on their own printer. A few minutes later, and they have the artwork at exactly the right dimensions if they want it. They pop it into their own jewel case, burn the FLAC audio to either FLAC (data) or PCM (audio) CD, and they have an exact copy of the music - just like an old-skool, silk-screened retail CD, with the exception that they've burnt this CD themselves.

Why pursue this method of distribution? The reasons are threefold:

• All of the traditional distribution costs of physical CDs are instantly eliminated; there is no need for minimum pressing runs of a thousand units or so, so less warehouse space is needed for unsold stock. Everybody's carbon footprints reduce - no distribution of stock from manufacturer to distributor to customer is required, no massive energy usage required to manufacture the products in the first place, and even the customer doesn't need to drive to the shop to buy the CD. Given the way the price of petrol is increasing, this could seal the deal on its own.
  
• Tracking of sales, royalties and other monies is far, far simpler. For every purchase you have a detailed audit trail, and a standardised method of logging all this important sales data can be used to ensure that the appropriate information is passed to the appropriate bodies for further meteing out to artists and labels. Mechanicals, a legacy royalty from the days of vinyl pressing (literally, a 'mechanical royalty', paid for each unit you pressed of a record) become far more relevant than they have ever been for current digital sales. A mechanical on an audio file which is exactly equivalent in quality to the original CD audio is quite acceptable, whereas a mechanical fee for an MP3 is almost laughable, yet the industry still enforces it because they can. (a tangible, per-unit fee on an intangible item... you make your own mind up about that)
  
• Distribution is streamlined. The album can go on sale as soon as the master copies of the audio and artwork are uploaded to the central storage point for sale to customers. The traditional delays - manufacturing turnaround times, etc - are gone. Just like that. You might have to wait for your artwork to be finished, but you can always offer that as a later, free download if you want to sell as soon as possible. Oh, and another pleasant side effect of this is that physical items will become much more of a 'special' item, as opposed to a commodity - see the resurgence of the 7" single as a 'limited edition' value-added purchase? The pressed CD could quickly fall into the same category - value added, with the appropriate (moderate) price premium. Of course, something else would have to be offered (exclusive content or goodies inside the jewel case combined with limited edition packaging), so the traditional CD wouldn't die out just yet.

The sensitive topic of job losses is unavoidable at this point. Yes, as a result of these kinds of sweeping changes, some people in both the music and peripheral industries would possibly lose their jobs, notably at companies who supply the traditional music industry - pressing plants, for example, and the printing companies who supply these plants with the finished artwork. But then again, vinyl production has been trickling along for more than a decade since the original rise of the CD, and some genres of music are still based almost entirely around the 12" vinyl as the primary method of distribution. (I happily pay £5 a time for a two-track 12".) Of course, physical production costs might increase again as a result of this move away from pre-manufactured physicals, but this would also be expected. While job cuts are an unfortunate side-effect of a changing industry, this wouldn't be unforeseen, so it wouldn't come as that much of a shock. Change is painful sometimes, but it's for the best.

Nonetheless, this concept is great for the environment, fantastic for the customer and even better for the artists! Treat the customer like a customer, instead of a mere consumer who is lucky to even have access to any kind of digital music, and you foster an attitude between music fans and labels which is far more healthy for the music industry. Give customers what they want and trust them to be adult about it - it's a return to pre-digital mindsets. You buy the music, you get it in a format with which you can do what you like, and you're treated with the maturity and respect a customer deserves. And, in response, you treat the artist with more respect, and you don't set about making copies of the album for every Tom, Dick and Harry who happen to ask for a copy of it. All that I've laid out is with a core aim of revaluing music (as opposed to devaluing), and I believe it could really be very successful.

Why this kind of service has not launched yet is beyond me, and I find it incredibly frustrating. As a knowledgeable fan of both technology and music, this is entirely possible - in fact, it's quite doable with today's technology - and you might as well corner the market niche before other people realise that there is money to be made in catering for people who want that top quality. Unfortunately, I don't think the music industry is ready for this yet - they will be need to be in truly dire straits before they will agree that this kind of service is what's needed to satisfy consumers, and by that time the golden opportunity will have passed. Case in point: AllOfMP3. Instead of squabbling over royalties, legalities and suchlike, they should have taken a step back and analysed the platform. Technologically, it was very different and quite brilliant - the higher the quality file you wanted to download, the more you paid. For 128kbps MP3s, you paid a few pence, and for FLAC, you paid a few times more. Even so, it was still a very good price. You could choose from many different preset and custom formats, Ogg, FLAC, PCM, VBR MP3, CBR MP3, WMA... Whatever you wanted, it was all there. Music industry - why are you tieing yourself down to just a couple of predefined formats when it's clear consumers want more choice?

If I had some Venture Capital funding and could bring a few likeminded people on board to help bring this kind of project to the public, I think I would probably launch this service right now. The market is ripe for this kind of quantum leap, but unfortunately the incumbent labels and online retailers only want to milk every last penny of revenue from the inadequate platforms they currently operate.

Give it 18 months or so, and I bet that a service similar to this will go into development or even launch. Consumers are increasingly aware of the quality of music, and the actual cost versus perceived cost to labels and artists for distributing their music online. They are also increasingly becoming connoiseurs of quality - so to offer this ultimate tier of quality for a reasonable price and instant download is the only feasible way forward. Now then, all that's left to be done is make the labels wake up and smell the coffee. - shouldn't be too hard, right?

... Right?

Hello world once again! A little earlier I finally flipped the switch on Into The Unknown's new design for 2008. Based on a (heavily) modified version of K2, it's a simple but effective layout which I think works well, while remaining not too dissimilar from the previous 'holding' design. There are some minor cosmetic fixes to perform, but for the most part - it's done. For the moment, anyway!

If you've seen the new header image and you're wondering, 'what's that star?' then this may interest you: the amazing background image you can see in the site's header is nothing more than a tiny (really!) detail from a much larger image of the Orion Nebula, at the centre of our galaxy. This is the first of a series of images which will be rotated round, something I think will bring a unique touch to peoples' initial impressions - and our galaxy really is a beautiful, wondrous thing to just look at. The Orion Nebula really sums up everything I'm trying to convey with this site, and my efforts; the great unknown, unimaginable concepts, vast, strange new vistas and environments... Plus of course, the name of this site does draw the altogether more obvious comparisons, which I'm quite happy about.

I'm all for crediting the original source (logically, it would be NASA/ESA/The Hubble Space Telescope Project) but I can't remember where I obtained it from! All I can remember is that to the best of my knowledge, the image was placed into the public domain for use by anyone, anywhere, so we're fine on those grounds. I've also uploaded the original source image for your perusal (it makes an excellent desktop wallpaper).

You can find more images by starting out on Google with a few choice keywords, or you can trip over to HubbleSite.org and begin your quest there. Aaron Gifford has a couple of excellent, widescreen-formatted images from the Hubble and Spitzer telescopes that you can use for your own desktop if you're thusly inclined.

For the past couple of months, I've been wrestling with a Voice-Over-IP system I've had to set up. Not only set up, but learn how to set up! From knowing almost nothing about VoIP technology, including Session Initiation Protocol (SIP, the protocol most VoIP phones use), to knowing a great deal more about how to set up a multi-user VoIP system, a few resources have helped me greatly.

First of all, the Voxilla forum is a great source of information and help from fellow users if you're working with any Linksys/Cisco or Sipura equipment. Like any popular forum, a quick search to check if your problem has already been discussed (or even solved) is recommended before diving in with your first post, but chances are even if you can't solve the problem in one go, the other people might help you narrow down the causes and make your life that bit easier.

Another forum should also be on your bookmarks: the Petri Forums, from Daniel Petri's tech discussion web site. This site's useful for more general networking problems and discussion, but quite often a problem with a VoIP network is due to a tricky network configuration (so by solving one problem, you remove the causes of the other!)

If you want to learn more about SIP and the often undocumented 'features' of the protocol and the technology, there are many guides and help documents on the web. However, I'm quite a visual person, and prefer a well laid out graph to reams and reams of text. The Tech-Invite site, "a Portal devoted to SIP and surrounding technologies" has loads of information, but what I'd like to point out is a series of 19 Really Useful Documents under the heading of "SIP Service Examples", with raelly well-written explanations accompanying large, easy to understand, coloured flow diagrams.

I was having problems with our phones establishing a call (they could see the incoming call, but for some reason they weren't reporting back to the remote SIP server that they were accepting the incoming call, so the actual call would never establish). The Call Pickup diagram greatly assisted me in understanding what was going wrong, and how to set about fixing it.

The InformIT web site also has a (text) article called "Session Initiation Protocol > Description of SIP" which explains - in a good amount of detail - both individual aspects of the SIP protocol, what various system status messages actually mean, and it even picks apart a selection of sample SIP messages, explaining what each part of it equates to in real world terms. If you want to expand your understanding of SIP, I suggest you browse through everything I've linked to during some spare time (with a few cups of tea and some biscuits!) and really clue up on SIP. I've been pulling my hair out at work for the past couple of months trying to establish a solid, problem-free VoIP setup and network, and I believe I've finally managed it.


Also, if you're thinking of setting up a VoIP network using Sipura or Linksys equipment, there are some network adapters you can buy which simplify the process - but which introduce problems of their own. For simplicity's sake, I suggest you run the phones directly attached to the network, with their appropriate SIP account details programmed directly into the phones, and with either DHCP IP addresses or static IPs configured via their (very 'comprehensive') web interface. Get rid of any adapters you might be considering (like SPA2000s or SPA9000s), because they introduce another layer of complexity.

Also, you wouldn't BELIEVE how many devices cause problems with SIP networks running more than one phone. For example, the Linksys WAG325N, one of their flagship pre-N wireless routers, sold specifically as having full VoIP compatibility, horribly fails when we hook up our kit through it. It just doesn't like the SIP traffic. To get a working network, here's how I configured our (separate) network:

Modem: ZyXEL P-660R-D1 ethernet modem (running in half bridge mode (Half Bridge Mode FAQ from the ZyXEL site), which is something only the D series modems can do, connecting to an ADSL Max Premium ISP for the higher upstream bandwidth)

Router: Linksys WRT54GL (or a WRT54G Version 4 will also do nicely), running the DD-WRT third-party firmware. It is crucial that you buy the correct, compatible version of the Linksys router to run DD-WRT on, many recent models are inadequately specced to run the firmware. Buy a WRT54GL if you can, avoid the WRT54GS or WRT54S or ANY of the Linksys routers which don't look like the classic blue and black routers with the two stub antennae at the back. I've linked to the Misco page for the router I purchased for work, and it works a treat (I also have one at home).

The Linksys router is set up to obtain a public IP via DHCP, and an appropriate subnet is configured (I used 10.1.1.1-254 for the local area network). The router plugs into the ethernet socket on the ZyXEL modem, and the modem (in half bridge mode) passes through the public IP address to the router - and bam! No second layer of NAT required. This greatly simplifies setting up a working VoIP network, as you don't need to set up static routes on any of the devices or 'kludge' your network setup. The router behaves like it's got the modem integrated. (The modem itself is still accessible via 192.168.1.1).

The reason you can't just put the modem into full bridge mode is that most ISPs in the UK only use PPPoA, which requires authentication whenever you connect. DD-WRT cannot deal with PPPoA connections natively, only direct PPPoE connections (which is why it works fine for cable modem connections), but with a router which can work in half bridge mode, this is trivial. In my setup, the ZyXEL modem handles the PPPoA authentication and silently passes all traffic through to the Linksys router to deal with - and the Linksys router handles NAT traversal, uPnP mapping and all the complex stuff. Indeed, when I plugged the phones in after configuring the modem and router appropriately, everything worked first time!

The reason I recommend DD-WRT for the router is that the firmware is vastly improved over the stock Linksys firmware; it turns the router from a capable to an excellent device, for no cost. Aside from that, it also provides you with a much greater deal of flexibility, compatibility and no-nonsense configuration and analysis (aside from the usual SSH, telnet and web interface for configuration, it will show you realtime bandwidth graphs via SVG with a compatible browser).


Some other tips I picked up... On your SIP phone, make sure your RTP Packet Size is 0.020 or less. (Most articles I've read don't suggest going lower than 0.010.) Linksys/Sipura phones ship with an RTP Packet Size of 0.030ms as their default, which can cause problems with the audio (stuttering / inaudible passages, distortion or even audio dropouts). Lowering the packet size lowers the latency, shortens the samplerate between packets and results in more reliable transmission of the audio from you to the other call (at the cost of marginally higher bandwidth usage). I noticed a marginal decrease in the latency between me speaking and the other party hearing my voice when I lowered the packet size, bringing it in line with a regular PSTN -> mobile phone call - much better! Before, audio was taking anything up to half a second to get to the other end, which really becomes noticeable in telephone conversations.

You can leave your subscription time to 3600 seconds for most SIP services, but ensure that NAT mapping and NAT keep-alive is turned on, and the NAT keep-alive interval is very short (I have our phones set to between 15 and 20 seconds, whereas the default on the Sipura devices is much longer). If you have multiple devices connecting through the same broadband connection, this helps remind the router to keep the phones' individual NAT maps alive, so that incoming calls (and outgoing calls) function correctly.

Six options on the Sipura phones I found useful to enable:


Handle VIA received
Handle VIA rport
Insert VIA received
Insert VIA rport
Substitute VIA Addr
Send Resp To Src Port

Set all of these to Yes (some of them default to No). These will aid the phones in handling with packets rewritten by NAT or proxy servers (which most networks will be using, unless you have a block of IPs with one dedicated to each one in a pass-through configuration, but that would be highly unusual). For more info on what each option means, see the VoIP-Info wiki article for Sipura devices, and this discussion thread on the Voxilla forum.


When testing and troubleshooting your phones, try making an outbound call to a regular number or mobile number - if it cuts off after 30 seconds (or a minute), you likely have a NAT problem. Check your settings and consult your provider for more info.

Be sure to check both incoming and outgoing calls work as expected both immediately after the phone successfully registers with the server - and after a while (30/40 minutes). If outgoing and incoming calls last more than 3 or 4 minutes, your network is most likely configured correctly, so if you have audio dropout or other problems (say, after half an hour in a call), something else is afoot.

In terms of codec choice, most phones will offer G.711 (I would recommend you choose G.711a instead of G.711u if you're outside of the United States, based on this discussion and this discussion), but as they are the highest quality (uncompressed) they use the most bandwidth. For more info on all the possible codecs used for VoIP platforms, see the VoIP-Info wiki article. If you can't use G.711a (or it's not feasible due to bandwidth constraints, then G.729 or G.726 - in that order - are the next best, and most commonly supported choices. If you're planning to use a fax device on a VoIP line, ensure that the service provider supports the T.38 protocol - and again, configuring the appropriate RTP Packet Size is crucial (setting a very low amount, like 0.010ms, will help with problematic devices that may receive a partially garbled transmission - or not connect at all). Some trial and error will be involved in finding the sweet spot - a discussion about 'Sipura tricks' for optimising your SIP call quality can be found on the DSLReports forum, which will also affect T.38 performance. NB: if you're sending data via T.38 to equipment which doesn't support T.38, results will be unpredictable (and you might as well not bother), so if fax over IP is important to you, ensure you check with your potential provider before you sign up, as it's not a standard offering.

Regarding RTP packet size, I've heard lowering it to reduce loss or uneven data transmission can also help with things like Sky Digiboxes and other devices which expect a regular PSTN line to 'phone home', although again, I've not checked this out, so YMMV. Check Google for more related reading.

Also, although I've detailed why a lower RTP packet size value is recommended, if you're in a situation where you find you have to use the (infrequently used) G.723.1 codec, your RTP packet size must be 0.30ms for the Sipura devices. This is something which I think is due to the constraints of the G.723.1 codec; its samplerate and packet size requiring the larger RTP packet size.

A final tip if you want to troubleshoot codecs - if you want to see what codec your Sipura phone is using to dial out or receive a call, make (or receive) a call, then go to its web interface while the call is in progress - and it's listed under the Line x Call y Status heading, a little way below the Phone Status and Extension statuses. The web interface can be accessed by going to its IP in a web browser (so for example, if your phone had an IP address of 10.1.1.201, just type that into a browser). To find your phone's IP address from the handset itself, press the button with the paper icon with the folded corner, just below the voicemail button (with the envelope icon) - this will go to the Setup screen. You can either page up or down with the cursor D-pad, or you can use the handset numbers to type in a number next to each menu entry to go straight to it. In this case, the Network setup is option 9 - tap 9, then you can see your current IP as entry 2 in the list. If you've plugged the phone in fresh out of the box, it will be configured for a DHCP-assigned IP, so this is the easiest way to get cracking. If you're feeling brave, you can even use the phone's interface to configure the IP address settings (but it's a little tricky to get right the first time, so I recommend you use the web interface, it's much handier). Once I have static IPs assigned for each handset, I always use the web interface.


To make your life a little easier when working out which codecs work and which don't, I always recommend first navigating to the advanced admin panel and enabling all codecs for use, even if you don't want to use them. There's some small links at the top of every web admin page, first of all you will see Admin Login - click that, then when the admin page reloads, click Advanced. This will become second nature after a while, and you can always bookmark the pages to go there direct if you like. You can't always modify all the phones' settings if the phones are locked to a provider, but if you buy them from Misco or BroadbandBuyer, they'll be sold as unlocked. Don't forget to flash to the latest firmware for each phone as soon as you get it, as this can sometimes unlock more extensions or fix problems in the older firmwares.


If you want any more help with configuring VoIP devices (particularly handsets or network configs similar to the one I've detailed here), you can get in touch with me via my company. Quite often I can direct people to existing information and support on the web which I've collated, but if you have a tricky problem I'll gladly provide my assistance on a commercial basis. This isn't a plug, but some people might just find it easier (or more cost-effective, if they're wasting time trying to sort it out themselves) to pay someone to get the job done! VoIP and SIP is a world of hurt if you're diving in for the first time, I'll readily admit that - some devices are far easier to set up, and play far more nicely with the wide variety of equipment and devices found in the office or home environment. The Sipura devices are far more picky, but they're not impossible to set up. I hope this article helps other people who feel they've hit a brick wall and can't figure out what their problems might be - and my final bit of advice is: don't give up! You'll feel very satisfied when your SIP system is fully operational, and as you'll most likely save a bit of cash too, it's all worth it in the end.

As I like to remind myself when I lose heart; being on the bleeding edge is often painful, but always rewarding!

After a brief hiatus, Into The Unknown is back - and it's had its dose of vitriol upped! Coming in the next few days: a look on the current state of the music industry, because yet again there's been more developments that affect both the industry and the consumer alike. Also, browser 'point version' milestones and comparisons, and a little dip into my lucky bag of topics as we cover what's coming to the boil (and what's bubbling over) in the wonderful world of the web, media and social industries.

The site itself is also going to get a lick of paint in the coming weeks, so that'll be fun too.

Stay tuned :)
Christopher

I'm not a huge fan of labels and generic concepts to describe a whole new way of thinking and doing things, but it can't be argued that the people who constitute Generation Y are more often than not distinctly different from those who make up Generation X.

Move aside parents, your kids are coming...

Gen Y is taking over. The generation of young adults that's composed of the children of Boomers, Generation Jones, and even some Gen X'ers, is the biggest generation since the Baby Boomers and three times the size of Gen X. As the Boomers fade into retirement and Gen Y takes root in the workplace, we're going to see some big changes ahead, not just at work, but on the web as a whole.

-- Why Gen Y Is Going to Change the Web

I'm a bit odd in that I associate more strongly with GenY than GenX but I wouldn't class myself as an out-and-out digital native - maybe a naturalised digital native? (My vinyl and CD collections would vehemently argue against that, though).

Either way, put aside five minutes to read the full article on ReadWriteWeb, it's actually quite interesting. Your thought for the day :)

... But the current industry major players can't figure out how in the hell they're supposed to wake this sleeping beast. Hands up who else is waiting for some real innovation? Hands up who doesn't think We7 will last in its current incarnation for more than 18 months? Yeah, me too.

In lieu of a more long-winded post (which may yet happen), I thought it wise to note the latest info on paid digital downloads from the OCC:

London - Digital music sales in the U.K. were up across the board in the first quarter of 2007, with digital album sales rising 69.3% from the same period a year ago, and digital singles up 42.3%, according to data compiled by the Official U.K. Charts Company.

Source: DMW

So, what's the industry still complaining about this time around? Oh wait, the $100,000 legal bills they're having to pay from some of their frivolous lawsuits won't be helping. That, and all the other people who don't have iTunes gift vouchers are starting to realise (I think) that the current crop of legal P2P services just aren't fitting their requirements (and are STILL clunkier and less satisfying to use than the original Napster), and so continue to take to the pirate seas en masse.) Yarr!

The sage commentary shall return when I have a free moment to sit down and write more cogently - for now, that's about as intelligent as I get. (Why am I always horrendously busy in fits and starts, with long periods of downtime inbetween? Very frustrating. If anybody can suggest some effective time management strategies, I'm all ears.)

Until next time...

Little by little, the music industry is tearing itself apart. Be it uncertainty, ignorance or unwillingness to accept the inevitable, we may never know (but I have a good idea). Not even a week ago, the RIAA announced that CD shipments are down 17.5% but digital is now 23% of total revenue. Metallica, always late to jump on the bandwagon, are also apparently mooting a Radiohead-style DIY-price-it-yourself release - after Radiohead have announced that their release was most likely a one-off (one thing it didn't do was make them money!)

And, in the same breath We7 finally gets a fuller-scale launch, with the licencing of SonyBMG's content to bolster its indie label content (with more from the other labels on the way apparently). Wake me up when the paradigm changes, because I gave we7 a spin (with some out-of-copyright material) and to be honest I was disgusted by the way the whole concept of forced 10 second preroll advertising on the start of EVERY track devalues the music.

The industry is always harping on about how digital downloads and rampant unlicenced P2P distribution is devalueing and commoditising music to the extent where people don't care about it - but when you have major launches of new web sites like We7 (famously backed by Peter Gabriel!) whose core revenue model revolves around commoditising music by placing adverts before the start of each track, what do they bloody expect? It ruins the listening experience because the technology hasn't yet caught up with the concept, and it's a stupid idea to try and roll an idea like We7 out now when they can only offer a "bodged" end product. I saw an email containing details of revenues to labels for putting their catalogue onto We7 - it wasn't even guaranteed revenue but PROJECTED revenue! And even then, it was an absolute pittance.


I have my own ideas for how to save the future of the music industry, and while they are a little "out there", I have a few good ideas cooking gently on the back burner and they do in places overlap with the shared goals of Michael Robertson. (If you used MP3.com, and particularly my.MP3.com, that was his project.) Unfortunately, Michael's currently being sued AGAIN by EMI for his latest innovation, the Oboe locker which ties in with his (legal) web site MP3Tunes.com. Another classic case of litigation in place of innovation - especially poignant given last month's ruling in the US by a federal judge against the music industry, which shatters one of the key arguments in many of the RIAA's legal cases against individuals accused of illegal filesharing. You can come to your own opinions about that (I'm sure of mine!)


We live in interesting times... Turmoil is still rife; the industry itself predicts that 2012 will be the year when digital revenue overtakes physicals - but I severely doubt the sales channels will even vaguely resemble the form they currently take. Sales platforms like iTunes are nothing more than a flash in the pan (and won't all those iTunes fans be crying into their proprietary players when they buy a different branded DAP and find they can't put their M4P AAC files onto it?!)

The timing on this couldn't be much better - I'm in the process of drafting up my open letter to the music industry, but I thought I'd mention this story in the meantime - I think it pretty much sums up everything bad about DRM...

[Microsoft] has arbitrarily decided that all the music you bought from it is now worthless because it can't be bothered to live up to its end of the bargain. It goes something like this, MS came up with a DRM infection and attached it to its music downloads. When the Zune came out, it decided it wanted a bigger piece of the pie, so it cut out all its partners.

The new toy, the Zune has a completely different DRM infection making any of the things you bought from Microsoft saddled by the old way incompatible with the new way. All the partners that made devices compatible with the old way, and all of the people who bought infected music, well, MS has your money, so you're dumped.

This was about a year and a half ago. Today, MS is saying that come August 31, a week or four shy of the two-year anniversary of the night of long knives, you will not be able to re-authorise your music. That means that the music is yours to keep. On one PC. As long as it doesn't crash. Or you don't update your OS. Or something doesn't just decide to stop the music playing. If you do any of these things, you just lost your music permanently.

Basically, MS is stealing from you. It has your money, but you can't have access the services you bought any more. So they are stopping, and legally, you are screwed. Microsoft has got your money though.

If you were a subscriber to MSN Music, you will have recently found yourself out in the cold when Microsoft decided to arbitrarily turn off their DRM authentication servers, leaving all your music you'd legally paid for in (effectively) stasis. Walmart did this with their video rental service last year, Microsoft this year - I'm glad that it's a discernable move away from DRM, but still... Support your legacy customers, or are you condoning your customers to break the law and strip the DRM from their tracks so they can achieve the same level of functionality as they enjoyed prior to this?

The full article, in all its gory detail, can be found on The Inquirer.