Into The Unknown

I've been taking a load of photos for one of my other sites recently, and adding some of them to Flickr for the Birmingham groups - and one of my biggest hassles has been geotagging each photo.

I have a Bluetooth GPS receiver which I use with my PDA for its satnav software, which outputs its data in the standard NMEA format - so why can't I just fire up a Bluetooth interface on my camera, sync it with the GPS receiver, and automatically have it add the GPS coordinates to the EXIF data for each image? I know there's some VERY high end DSLRs that offer this facility, either through integrated GPS or a dock/holster-style fitting for a compatible external receiver... But this shouldn't be something which is purely the reserve of well-off pro-ams or industry professionals. The technology is freely available and already quite cheap, so in the age of the Semantic Web and an ever-increasing insistence on metadata-for-everything, why is this whole process such an arduous one complete?

Web 3 will never take off if it's as long-winded as this to input data into the Matrix in the first place - never mind extract it at a later date...

What with all the fancy technologies and acronyms whirling around the web (and your head) at the moment, it's easy to get lost in the simpler things that help the Web work. One of those things is RSS, or Really Simple Syndication.

The whole concept of RSS really opened up the ability to take data from multiple sources, repurpose it and repackage it for an entirely different end result - and developers love it. Today, RSS is definitely one of the web's core features. It's the ability to take feeds from sites - and there's agreed formats (several of which are the de-facto standards) in which this data is published for the benefit of all to aggregate, remix, mashup... Who couldn't like that kind of thing? Projects like Yahoo Pipes just make the concept even more accessible for end users, meaning that what was once the preserve of hard-core coders is now accessible by everyone. And isn't that what the Web's about?

Arguably, the most commonly-recognised formats are Atom and RSS - you'll usually find links to both of these formats of feeds on all the most popular web sites. Don't worry if you don't understand the principle behind RSS, because I still don't quite have my head around the finer points of the format! And this is where this Cheatsheet starts to come in handy... If you're struggling with your RSS understanding though, don't worry - it's not as hard as you'd think. In fact, the principles behind RSS are really quite nice and simple once you understand the buzzwords.

So, if you want to brush up on your tech knowledge, Commoncraft have put together a nice little tutorial for everybody - just watch this handy little video guide: "RSS in plain English" and all will be revealed. If you want to download it for future viewing, you can do that too.

...Part 1: the UK cable broadband industry.

Preface: having previously been a customer of Be* ADSL2+ for 12 months, I transitioned to cable broadband as my shockingly poor BT telephone line could only manage 10mbps max on a 24mbps service. Minor gripes aside, Be* were an excellent ISP, and they only look to have improved since I left them as a customer.

I'm currently a customer with Virgin Media (in Birmingham, so on their ex-Telewest network), on their XL package which gives up to 20mbps download and 768kbps upload. The nature of cable means that it's less susceptible to speed loss over distance, the most common affliction of DSL. ADSL2+ services offer faster throughput than DSL Max, but are even more susceptible to noise, interference or just plain poor quality lines (of which there are many in the UK, given that our ageing BT network is approaching 70 years old now).

Up until recently, Virgin Media offered their speeds without any kind of limits or throttles - one of the few remaining ISPs in the UK to do so. However, they then rolled out STM - Subscriber Traffic Management, a proprietary 'solution' from Cisco which allows them to do various things. They currently have it configured to throttle a user's available speed back for a certain amount of hours during peak times (currently 4pm-midnight) if they go over a certain threshold. This threshold is different depending on what speed package you're on, but the thresholds aren't high - just 3 gigabytes of data in an hour on their XL package will trigger STM for four hours.

This has caused much consternation amongst users, but VM are steadfastly sticking by this current setup, much to the chagrin of many (including myself). STM is also being constantly 'refined' by Virgin Media, and will soon (if not already) monitor both your download and your upload usage (it already throttles both if you go over the limit). The peak hours are set to change, as is the period of throttling (up from four to five hours).


I am totally against STM, traffic management or any kind of last-gasp 'solution' like this - it smacks of under-investment and desperation. So, prompted by a post in one of the recent discussions on the VM newsgroups about this topic, I decided to formulate my plan to save the future of the UK's cable broadband network.

The UK DSL industry is currently undergoing its own mini-revolution, with the gradual phased rollout of 21CN by BT (basically glorified ADSL2+) and, we hope eventually, fibre to the home - or at least fibre to the cabinet... So, we're leaving those guys alone for now. However, a number of the things I said in my response still apply to them.

Please note that as I originally posted this in virginmedia.discussion.broadband, references to "this company" mean "Virgin Media."

Anyway, here's my reply...

---

(in response to a previous message in the thread, discussing
how the lowest tiers of service have a much lower total download
threshold during peak hours before they become 'managed'):

Mark McIntyre wrote:
> > In fact, I think the policy would be a lot fairer if all
> the tiers had
> > the same download/upload cap AND the same throttled speed. Someone
> > fully utilising a 20Mbit connection is obviously causing far more
> > network congestion than someone using a 2Mbit connection ;
>
> I'm inclined to agree, personally.

I'm glad you're not in charge of administering STM then, because that's a quite, quite silly. Maybe if we were in a Communist state and we all paid the same for our broadband, I wouldn't mind that, but we're Western Europe last time I checked (in Soviet Russia, broadband throttles you!)

It's not hard to fully utilise a 20mbit connection for long enough to trigger STM - just go to somewhere like Stage6 (the DivX.com showcase site where anyone and everyone, from individuals to movie companies) can upload video content for in-browser playback in REALLY high quality...

Stream some music videos, HD vids or plain old 15/20minute documentaries for half an hour (easily done)... Even Google Video will achieve the same effect... Bingo, STMed.
So, what have you done to start this chain reaction? You’ve just utilised your connection how it's intended to be used - viewing rich media in a setting which could hardly be described as excessive.
Cue hassle, frustration, annoyance, gradual loss of faith in brand. This is not A Good Thing.


-- WARNING: MY PROPOSAL FOR THE FUTURE OF CABLE BROADBAND LIES AHEAD. --

I'm cutting the bull now, and I'm going to speak as frankly as I can.
Hopefully these comments will be noted and passed on.

I hate the fact that people who decide to actually use their connections get branded as the scourge of the customer base and must be punished, and I think that STM as it currently stands is wholly ineffectual and really quite insulting. You just *can't* impose strict, nonflexible limits on a next-generation, high speed, and therefore logically high-availability service such as >=20mbps broadband. People expect a little more than this, even on a residential service.
The thresholds on the lower tiers are pitifully low, and XL is only just bearable as it is.

The lack of prior notification before you trigger STM is unacceptable.
The national road network has defined and clearly indicated limits, and all cars have speedometers as standard. Thus, if you go over the limit and get penalised - that's your fault.

So, having just signed up to a tier of service (quite possibly the top tier, having been lured by the slick marketing campaign), this company expects everyone to then go and install an unverifiable, standalone after-market speed indicator so they can self-regulate their usage according to an arbitrary limit which was only brought in after someone realised that network utilisation from the increased uptake of a market-leading service would actually increase faster than their forecasts? Speaking from experience, most bandwidth meter apps don't give an accurate representation of how much cumulative bandwidth you've gone through in a particular time period ANYWAY, so you're always on your toes and having to double and triple check.
And all this on (in my case) the top, PREMIUM tier of a service which is billed as unlimited and blazingly fast?

Again, not acceptable. Personally I regard any kind of hard, automatically-enforced limit as unacceptable on a service which should offer me a high level of service (contention and other network quality factors accepted), but when push comes to shove, something is really quite wrong with the network if it can't provide a consistent level of service which is still being sold to new customers regardless of the time of day. Slowing people down for using their service is NOT a fix, investing in the network is a fix. Strategically speaking, this is a far sounder idea too from both a business and financial point of view because the initial higher spend gives the company a chance to really future proof their infrastructure to the point where they don't have to worry about spending more in the short term more times just to offer a satisfactory level of service for a shorter time before the next level of speeds come in.

-- THE ANALYSIS --

The most noticeable side effect of this company's current business strategy is that it leaves a lot of customers frustrated and locked in to a service with seemingly fluid terms and conditions which are incredibly hard to both understand and comply with.

Trialling different mixtures of STM policies without forewarning customers may help your metrics but it's not helpful to people who are complying with the first set of rules only to find that although they thought they were obeying, they STILL get penalised. People who've had problems with overzealous STM automation have also had an uphill struggle to prove that it's not their fault (for example, see broadband discussion forum for a thread from last month where one customer provided detailed info about a number of times when STM was kicking in right on the dot at 6pm and it took AGES for VM techs to actually acknowledge this was happening).

What possibly riles me the most in all of this is that this company is promoting the blazing fast speeds (note the repeated recent airings of "hot diggety damn that's fast!" Umadverts), and gaining new customers on this promise of superior speed, yet then going "well... Actually, we'll let you go this fast for a little while just to prove you *can* go this fast, then we'll give you a lower speed than if you'd stuck with your DSL Max service. Oops, sorry about that. Please stick around, won't you?
After all, you do still have 11 months left on that contract... Fix the f up VM marketing.

I'm available for paid kick-up-the-arse daily consultancy until the 26th of January if VM wish to contract me to head office.



-- THE WAKE UP CALL --

My thoughts about this are quite clear. In all of this, in all the wrangling, all the discussion, all the back and forth, I think Virgin Media as a company is failing to realise something: in the next 5 years, one factor will become the distinguishing factor in the ISP market, and that is bandwidth.

Bandwidth, Bandwidth, BANDWIDTH. Unfettered, unthrottled bandwidth. Rich media abounds, and it's only going to get richer. IPTV is gaining in popularity, as can be seen by this week's CES announcement that BT's IPTV service is going to be via the Xbox 360 platform. What's the 360? Oh yes, it's HD. What's HD? Higher resolution = higher bandwidth.

When do customers use their _high_speed_ broadband connections? ALL THE TIME.

The gradual implementation of faster core infrastructure on the DSL side of things means that, within 12-18 months, Virgin Media will be in an indefensible position with regards to its treatment of users who use their networks more than they'd like. People are getting engaged in the broadband world, people are going wireless, getting various wireless devices, streaming devices, slingboxes, internet radios, watching things over the web, downloading content to watch later, renting videos over the web, hell - this HD-DVD / Blu-Ray format war is going to be the last one we see, because come next decade, all content will be high definition and digitally delivered over YOUR BROADBAND CONNECTION to a does-everything device underneath your (high def) TV.

You are currently taking the market in the direction where people are going to end up quite literally SCARED to use all their equipment with their broadband connection lest it suddenly (and unannounced) drop in speed for hours, to the extent where they're already thinking "why did I switch from my previous provider? I was paying less and I had this speed guaranteed 24/7."


This is not the right direction.


Telewest managed to offer (increasingly!) high speed service for YEARS without any kind of throttling or caps. Why the sudden sea change in attitudes towards provisioning enough resources for the network? So your metrics tell you that usage is rising in accordance with the company making higher speed connections available... What do you do? (and no, the answer is NOT "implement STM").

You ACCOMMODATE this usage.
More bandwidth.
Improved network structure where required.

More investment's required to achieve this end, but you can then charge more and get your money back just as quickly. The people in control of this company's finances have to be shown that looking as far as the end of their noses for ROI is not the way to run a sustainable, future-proofed, next generation broadband network. Consumer attitudes towards broadband (along with usage patterns) are changing rapidly, the people you'll find populating these newsgroups are but the tip of the iceberg - and it's one HELL of an iceberg. You won't be avoiding this one.


See where I'm going with this? This company's official stance on high speed broadband is so utterly hypocritical that frankly I think it's a miracle they've even managed to get this far. They're lucky that the kind of usage I've described hasn't reached critical mass yet, but when it does millions of customers are going to get a VERY nasty surprise (and this company's going to get a LOT of churn). They're also lucky that Ofcom takes such a long time to initiate any kind of formative action to shape the broadband market, useless fing 'watchdog' that it is.



-- THE REQUIREMENTS --


Here's my list of key investment points for Virgin Media's accounts
department:

1) more bandwidth
2) infrastructure upgrades to handle the inexorable, meteoric rise in the kinds of activities they're presently trying to curb
3) MORE BANDWIDTH! OVER-RESOURCE LIKE THERE'S NO TOMORROW! I can guarantee that it'll all become used eventually, but you cannot afford to retrospectively rearrange your network infrastructure to try and make a years-old solution fit with today's usage trends. Those trend curves are rising ever more sharply, aren't they... Still think your network can cope?

Some home truths: people will never stop p2p sharing. People will never stop downloading. People will always continue to increase their usage amounts. People will always buy more devices that utilise this commodity, and they will expect to have service.

The IT industry is no different than any other when it comes to buying in bulk - buy a crapload of backhaul and you'll get a savage discount. Then, you monetise on that by charging more for the services and getting your profits back like that because people will be able to use connections like they expect to be able to do, and with all that spare bandwidth in the meantime you can provide some really killer value-added services for the DTV and VoIP markets.

I'm sorry if what I'm saying jars with the current situation or corporate mindset, but people talk about how the final mile isn't good enough to support the kind of use that is going on RIGHT NOW, and is only set to increase... Well, the solution's quite simple: invest. Rip out the old kit, or add to it, with equipment which you KNOW will provide a sustainable level of quality and service for the next decade at the kind of levels currently only imaginable. Plan for 200mbps home connections, and spec accordingly.

I don't care if this is going to cost money, if you can boast that your network has the guaranteed ability to eventually offer more than 100mbps for the next 10 years without any more core upgrades, you're guaranteed a huge swathe of the market. Investment seems to be almost a dirty word with this company at present, and that's A Bad Thing. Maybe you need someone to point out what industry you're in again? Contrary to what you've been told, subscriber traffic management by way of reducing speeds is *not* adequate investment.



-- THE SOLUTION --

In the interim, what's the way forward? Well, you prioritise your offerings to consumers. QoS for the win. Why wasn't this run with in the first place? It makes so much more sense.

Classic example: a three-tiered offering. The same amount of bandwidth for everyone, no time-based limits, just different guaranteed level of quality:

Tier 1: "hey, you can get 50mbps of bandwidth with a high contention ratio for £25 a month, we make no guarantees for things like latency (within reason) but it's the cheapo package so it's good for web surfing, email checking etc, downloading your TV shows from iplayer/sky/kangaroo etc. Your connection will get last dibs on bandwidth, so you probably won't always get full speed."

Tier 2: "ok, so you want a bit better quality? Sure, we'll guarantee a minimum of 2/3 of your specified maximum bandwidth always available, and bandwidth preference will be second in line. Lower latency and a separate guaranteed QoS for time-sensitive apps like VoIP and video conferencing. £33 a month."

Tier 3: "you want first dibs on that 50mbps of bandwidth your connection can suck down? Sure, here ya go. Low latency, great for gamers, plus all the other previously-mentioned stuff (VoIP etc) - and we'll throw in on-demand video for you to use as a sweetener, which costs us less than you using a third party iptv/ondemand solution _anyway_ because it's coming off our own caches, so we don't have to pay any cross-network bandwidth charges. £37/38 a month."

All packages come with free wireless router and DETAILED instructions on setting up a SECURE wireless network. There's your value added, and the bulk buy discount you'll get on a decent, cheap wireless router is again going to be insane, so it's a no-brainer.

Guide prices for the tiers in the above example are roughly based on current package prices, so take that as you will. Sensitivity to price brackets may also change as the availability of high speed broadband increases, but this is just my current thinking at present.


Now, if VM was to implement that solution, I'd put MONEY on this company getting most of its customers on the Tier 1 or 2 packages, with the minority of heavy users going on to the most expensive packages and paying the most accordingly. VM makes most of its profit off lower tier customers (contention and overselling here) and the people who want that guaranteed level of service will stump up the most for it. Like they do now, their bundled multiple service offerings use the lowest or middle tier of broadband service (depending on the margins from other services like TV and phone), with an option to upgrade to the top tier for full price or MAYBE a couple of quid off (customer stickiness = lower churn = more accurate financial predictions = more guaranteed funds for investment = happy everyone!)


Have I made myself clear here? This is, for lack of a more viable solution, the only feasible way forward for the industry, as society's netizens progress from being casual users of a service which is still currently a luxury for many, to a society where broadband is the norm, it's expected, it's embedded and it's unfettered. You don't think about it being there, it's just *there*, like the gas, the water, the leccy.
(in other words, it's near-enough a utility). If you can suggest a better solution, please, be my guest (and I'll read any and all suggestions with great interest).


QUALITY OF SERVICE is set to fast become the deciding factor, not speed. You want speed? everybody gets the same maximum speed. You want quality? You pay for it. No longer are we obsessed with "whose advertised* broadband speed is the fastest", those who want the best pay the premium and get the guaranteed quality levels. The business ISP industry has had this right for a long time now, why haven't these ways of thinking been adopted in the residential ISP industry? Because everybody's too SCARED to go first, everyone's waiting for someone to jump in the pool and then they'll follow.




STM IS NOT THE ANSWER. Throttling is not the answer. Limits are not the answer. Pile it high, sell it tiered.




I've left that with lots of whitespace so hopefully it gets through to someone who has the power to effect change, because if this flawed, underplanned, unfair and archaic current method of usage 'management' is intended to be a permanent solution, then this company is going to find itself left playing on its own in the sandpit whilst the rest of the technological world grows up and realises the full potential of high speed, next-generation broadband access.

I love beginnings of years, they're really exciting technology-wise. With CES on right now, and more big tech news to come, we're going to be seeing a bucketload of new standards and revisions to existing ones, new formats (and revisions to existing ones!) and new discussions (and revisions... you get the point). Data portability and agnosticism is going to be the next big thing, and as if this wasn't enough, the topic was recently highlighted by the farce surrounding Robert Scoble being banned (and subsequently unbanned) from Facebook by scraping his contacts' details to import them into one of his other accounts... In years to come, our children will look back and laugh when they're told that we had to sign up for each web site separately!

Taken your pinch of salt yet? Right, some of the key emerging trends (I predict) are going to include:

• DRM-free audio and video content! (finally!)
  
  
• Open platforms (want to do anything with any site you're a member of? Sure, here's our API, go mad)
  
  
• Widespread CSS3 support
  
  
• Data Portability (more at dataportability.org) (which also encompasses the nascent APML and existing standards like OpenID and microformats)

... And they're just a few. Blanket HSDPA 3G coverage and the rollout of HSUPA (fast download and upload speeds, versus just fast download) is going to be the next big thing in the mobile connectivity arena - and in my opinion this has been long overdue. HSDPA is great, but the upload really stymies any real rich media interactivity - 3G video calling is awful quality because of the poor upload, and that's its main problem. As soon as the quality reaches a good enough threshold to become bearable, mobile operators will find their revenues (and usage) skyrocketing.

The TV industry's going to see a lot more convergence and shift from traditional broadcasting to IP-based shiftcasting (to coin a term) - the iPlayer's already gaining in its popularity (moreso since the site revamp, offering on-demand in-browser Flash Video streams of their downloads) and IPTV is most likely going to start to usurp your regular broadcasts, particularly given the increasing spread of high-speed broadband connections and all-in-one STBs. If you've not got some form of digital receiver in your home yet, I think it's definitely time to get one.

Broadband's just getting faster and faster, but super-fast, very short range data standards are going to be the next big thing in the home (ultra wideband support in AV devices is going to be killer when it comes to market). Sony announced its own version of short-range, high-speed (500mbps) wireless data connectivity at CES today (make that yesterday, now), so if they're bringing to market, expect others to follow suit. It's a no-brainer really, why bother with wires if you can just sit your camera on top of its dock?

Wireless charging is what I'm really holding out for, but I'm not expecting even protypes of that until at least Q1 2009.

Hopefully, 2008 will also be the year the UK begins to go fully HD (HD on Freeview is the next big step for the industry to take, though it's mired in problems and negotiations at the moment) - and hopefully I'll finally get myself an HDTV this year! That's the most important potential development of the next twelve months ;)