puremango.co.uk

(this is a slightly expanded transcript of a talk I gave at Oxford in June 2009 about my work there)

Hi! I’m Howard Yeend, my supervisor is Vasile Palade, and the title of my project is:

“Implementing Adaptive Web Sites using Machine Learning and Ajax“.

But before I talk about what all those buzzwords mean, I’d like to give a little background information about why this is an important research area, and why I feel it’s the right project for me.

When I was trying to think of a project title, I had a question in mind:

How can we improve the web?

And I think that’s a hugely important question for us to ask.

I don’t need to tell you that the web is strongly interlinked with our daily lives, it’s having a massive impact on the way we interact, the way we make decisions and the way we express ourselves. It’s a very pervasive technology. So it’s important to think about how we can make the web better.

But also in a more general way, I think it’s important for us as Computer Scientists, as students, as geeks, to ask ourselves “How can we improve things”. I mean, that’s kind of our job, isn’t it? That’s why we do what we do – we take technologies and combine them in innovative ways.

We want to make things better.

It’s important to note here that I’m not asking how we could improve the web in 5 years time, or if we had enough users, or if we had the backing of the academic community.

I’m talking about right here, right now – what can we do with the technologies we have available to us, that are robust enough, to improve the web? Well one thing we can do is to implement Adaptive Web Sites.

So that’s really the core of what this project is about. But what are Adaptive Web Sites? I keep saying those words, what do they mean?

Look at what the user is doing on a web site…
…and use that information to improve the web site.

I think that’s a nice phrase to encapsulate what Adaptive Web Sites are all about; you look at what the user is doing, and you use that information to improve the web site. It’s a nice way to explain it. But it’s a little vague, so let’s go a bit deeper into what exactly I’m doing here.

Well first of all I’m actually going to be implementing something. That’s one of the major aims of this project, to actually produce some software at the end of this that you can take away and use on your own web sites. But what am I going to be implementing?

So there was a lot of research in the mid-90′s about Adaptive Web Sites and recommender systems, which is a related field. If you go to amazon, you’ve no doubt seen the little “customers who bought this also bought” feature, that’s a nice little display of a recommender system. But the thing is, in the mid-90′s the technology wasn’t really there. Maybe you can detect that the user was searching google for “pictures of dogs”, and maybe google sent them to your “pictures of cats” page, so we can intercept that and redirect them to our dogs page, to make it more relevant for that user. That’s the kind of adaptation they’re talking about.

And that’s pretty nice, but that scenario doesn’t really happen that often, google have some smart guys working for them, they get it right most of the time.

So this is where the Ajax part comes in, because now, in 2009 we can get a much deeper look into “what the user is doing”.

Sure we can detect that the user hit the pictures of dogs page, but maybe they scrolled down to a particular paragraph.

So we can detect this, and send that information back to our server – while the user is still reading that paragraph – and we can say:

Hey, do we have anything else that this user might be interested in?

And so maybe we have another page on our site about the history of the breed they’re currently looking at, and we can just expand the section they’re reading and insert a little bit of content at the end that they might find useful, just a sentence or two from the relevant page with the option to expand further or link through to that content.

So as they’re reading the page, the page changes in response to what they’re doing.

Maybe they’re hovering over a particular link, perhaps they scrolled down to this or that section, maybe they’re selecting and copying text from the page, or they did a “Ctrl-F” to search for a particular phrase.

And that’s really what this project is all about; collecting this kind of data, theorising about the user’s intent, and then adapting the web site in real time to their needs.

So this kind of technology could be used to improve the relevance of advertising – updating a page’s adverts in real time, or to provide more relevant product and content recommendations, or even just to give web masters a richer insight into what their users are interested in.

Current analytics software, including google analytics, only breaks your content down by page; they’ll say “hey, people spend an average of 30 seconds on this page”.

But what were those users doing during those 30 seconds?

With the technology I’m developing, we’ll be able to say “hey, 50% of visitors nearly clicked this link”, or “600 people only read this one paragraph”, or “most of your visitors scroll down till they find a bold heading”. And this is really powerful data to a content publisher. And it’s my belief that browsing an adaptive web site is going to be a much more useful experience to the end user as well.

In the future, I can see a web where every site links in to every other other site in an adaptive way. Because if we go back to our user who’s interested in dogs, well maybe we actually don’t have any more content that is well matched to what they’re reading. But we can go off to google and pull down the top three results and present them inline to the user. Or we can insert the first paragraph from wikipedia about that topic.

So in this way, when I go to an adaptive web site to research a topic, I’m not just looking at whatever content that site might have, but I’m also looking at everything the web has to offer on that topic.  That’s the real power of adaptive web sites to me.

So that’s my project; Implementing Adaptive Web Sites using Machine Learning and Ajax.

I hope you enjoyed hearing about it.

---

UPDATE October 2009:

I’ve now completed and passed my MSc, attaining a distinction for this project. I will eventually be writing up my conclusions, which include some novel results. There’s also talk of publication and/or a chapter in a forthcoming book on web usage mining and adaptive web sites, so we’ll see what happens with that. If you’d like to talk about adaptive web sites, or alternative document similarity metrics (a key finding in my work) then please email

---

Related Posts:

• Google Instant Launches
• js1k: an allRGB entry in <1k of JavaScript
• XKCD Colour Survey – a 3D visualisation