Dissecting Google’s Wave Technology
Google has presented Wave at IO 2009 a couple of days ago. As the main developer of our in-house “Wave like” technology I have mixed emotions about what they have presented. To us at Corvalius it was not a surprise given the timeframe that we had put on the commoditization of such a technology. We didn’t expect to be Google the one that develop this kind of technology, but we are glad to know that we are in that same league.
Those that has been following the roadmap of Corvalius and the unpublished work that we have been doing at the Labs+Academics division in the last year where dumbfounded to see Google unveiled a similar technology and in such advanced state.
At Corvalius we knew that it was ready for primetime given the recent advance of broadband penetration and the cultural shift that is happening. And we also have found by ourselves how easy is to rethink almost every business process and application using this new model of REAL real-time collaboration.
First I have to say that I really like what I saw, and the demo was flawless in almost every possible aspect, kudos to Google’s Wave Team on that, its great to see innovation in such a raw form from the guys that did it in the first place.
Now strictly speaking of the Wave’s Technology demo I have a couple of interesting insights that I would like to share with those that didn’t have the inclination or haven’t researched the groupware algorithms in detail.
Google’s Wave is based on Jupiter, given the published work (we can just only speculate until they publish the source code of the server), they use a variation of the dOPT concurrency control where dOPT stands for “Distributed Operational Transform Algorithm“ that in theory allows achieving consistency in n-way communication patterns. However, as noted by several researchers dOPT fail in what it is called the TP2 Puzzle (picture).
The Waves team, avoided this problem using Jupiter’s approach where you have a central server that acts as the authority and therefore create partition the space into several 2-way connection. The problem is that there is still another variant of the TP2 puzzle that can be created using 2-way communication; the approach for that has been to rely on the ordering and not being able to send more than 1 update at the same time (that is why the protocol wait for the server to acknowledge –ACK- the delta).
There are other way to solve the TP2 Puzzle, that is to rely on a safe Operational Transformation that is more costly performance wise, but that is formally proved that it can handle real n-way communication without a central authority. At Corvalius we didn’t want people to rely on a central server so even though the approach has greater complexity we moved on that path. The biggest difference is that we delegate to the clients the transformation resolution and therefore do not require big data-centers (those that Google has already in place), even though we can easily make use of them for other interesting uses 
.
Both approaches have their pros and cons, the only question that I really have (so if someone of the wave’s team likes to shed some light on the issue I would really appreciate it) is: “In the Federated example, Wave have a potential n-way communication scenario, even though the approach with the server’s protocol has already ruled out the TP2 by construction. Can it be ensured that is the case when you move into a more P2P scenario with multi-servers environments you will still be able to satisfy the TP2 property?”.
This is an old technology example (not integrated into an actual product) of our implementation of the “wave like” intentional preserving resolution algorithm, that we showed at the Demofest of the Latin American Microsoft Research Summit 2009. This video is the demo proposal that we sent for evaluation, so it was supposed to be amateurish, but shows what the core technology provides.
Now on more concrete details over the demo, given that we have implemented a similar technology of what the waves team is using, we can say 2 things:
- The technology itself is: “A hell of technology” no matter who the provider of it is.
- The Waves demo is just a glimpse of what this technology can do in the RIA space.
For the first assertion I can say that nothing of what you saw in the Wave’s Demo is possible without the reconciliation algorithm and the properties it satisfies. So expect to see a complete new breed of applications and ways of doing things (we are at the dawn of a revolution similar to that of the personal computer). Moreover, the most impressive work that the Waves team did is in the UI space not on the tech-side mostly by leveraging the properties that the reconciliation process bring with it. That points straight into the next assertion.
For the second I would like to dig deeper into what we have been seeing lately, there are 2 very different camps regarding RIA technology; those that believes that the browser should be the desktop (Google’s camp) and those that believe the browser is just a little too limited to be the desktop (Microsoft’s camp). I personally can say, I DONT CARE the form it takes, as chance favor neither one nor the other is completely right but I see things are changing and who knows where we will end up?.
With Silverlight 3 introduction with out of browser experience I foresee that the distinction is going to get blurry pretty fast, moreover Google’s Gear already has moved to fill some of that space too. So we are in pretty interesting times.
Now with that in mind, what if you move out of the browser and start using this technology outside of it? That is our view at Corvalius, we don’t care if you use it inside the browser (through Silverlight in our case) or natively outside the browser in a typical Winforms application; we don’t care if you rely on a central server to resolve the changes as Waves do, or you have the OT Framework embedded into your own applications for your own use. What we care about is that new ways of interacting create huge possibilities for new and impressive applications and ways to do your stuff.
We believe that even thou
gh Google’s Wave is an excellent portrait of how technology is already changing our culture, as in the Transformer’s movie, “There is more than meets the eye”.
Hans Riesnik
What’s your opinion on Google’s next steps?
Federico Lois
What they have done is to show the world what can be done. It is the very same exercise that we did internally with our first prototype when we were only able to have a better notepad. Then we started to figure out that most of our daily activities require some sort of collaboration and that collaborative work is pervasive to almost any human activity. There is a huge space needed to be filled, but there is also plenty space to screw it up.
The more pervasive the technology, the most important for you to design for humans. So if I were Google (and I am speculating here that they are trying to enter the software market somehow), I would start doing software for people and not for niche markets (geeks like me for example) that would use anything that has a technological edge. As an example, Microsoft has made an OS my 80 years old grandpa is using and a communication platform (MSN) he is using to do video conference without my help. Google outside of the search engine (we can argue that Google Maps may be on that same league too), has never catered to that audience. With Waves they have an advantage, they had shown the world a different way to do things. But as a side effect, they had made easy for the competition to show the roadmap that investors werent seeing before last week.
Now more than ever, competition is going to work harder to chow a little of that open market. And there are, as you know by now, lots of people that has been working in the shadows for some time already on those very same concepts. At least to us, what they did wasnt a surprise at all. Everybody is speaking about the realtime translation, Google and Microsoft had that technology from the start; putting that to work online is pretty simple to any developer, the comparative difference ends the very same day you expose the idea.
For example, my first involvement in this space has been in 2003 while at college, but didnt have the funds or the experience to go further. But the concepts where already there layed out scattered all over the floor. I dont regret had that project dormant for 4 years, because the timing hasnt been better, and the hard problems like the TP2 had recently been solved for the general case.
And to respond to your original question, to me the next step is just plain simple, they must move to make it really massive and brain dead simple to the common user. The developers will just follow.
Soren Lassen
You ask: “In the Federated example, Wave have a potential n-way communication scenario, even though the approach with the server’s protocol has already ruled out the TP2 by construction. Can it be ensured that is the case when you move into a more P2P scenario with multi-servers environments you will still be able to satisfy the TP2 property?”
Hello Hans. It’s great to see your interest in Google Wave. I’m an engineer from the Google Sydney team and noticed your question and wanted to offer an answer.
The federation case works just like the single server case because, for a given wavelet, a single designated “authoritative” server (identified by the domain part of the wavelet id) hosts the wavelet and operates as server. All the other servers act as dumb proxies, they just forward the operations between their clients and the authoritative server.
FWIW, in the future we wish to elaborate the role of the subordinate servers, so they can transform local users’ operations locally, but it will be a while before we get around to take a closer look at this. It will not become part of the initial version of the federation protocol and implementation.
Prasad V
“Now with that in mind, what if you move out of the browser and start using this technology outside of it? That is our view at Corvalius, we don’t care if you use it inside the browser (through Silverlight in our case) or natively outside the browser in a typical Winforms application; we don’t care if you rely on a central server to resolve the changes as Waves do, or you have the OT Framework embedded into your own applications for your own use. What we care about is that new ways of interacting”
With due respect, Jupiter collaborative system paper came out in 1995. Implementing such a realtime system using native windows apis in 2009 is a shame and not research. A grad school project makes sense, but not by a research institution. What makes google’s implementation impressive is the usage of “browser” and sticking to standards.
Federico Lois
Jupiter came out in 1995, and I must say they made a huge advance in the field, but it had restrictions. Furthermore, lots of things happened for 12 years, until it was posible to prove formally that there exists an Operations Transformation function that is consistent, and intentionally preserving at the same time. And in fact, in the process they proved that ALL the OT functions published up to that point had consistency problems in the unrestricted case.
We definitely dont care about the technology, I think I was pretty clear about it when I said: “The technology itself is: A hell of technology no matter who the provider of it is”, what we really care is about the new interaction posibilities that as you said was available since 1995
(and almost nobody noticed). In that context, until Google waves or something else is live, most people is out of the equation, so either you implement the technology yourself (as we did) or you wouldn’t have been able to feel first hand the deep implications in interaction the technology carries with it.
About the technology stack, it makes sense to Google to use Java and Javascript and because it is Google to make it standard (well mostly because they can
). And in fact I would personally like to have an unrestricting collaboration standard, something I think google is looking forward to.
The browser is google’s business, it is google’s heritage and more importantly is google’s expertise. We are a Microsoft shop, so to us it makes more than sense to use Silverlight/.Net because it is OUR business, it is OUR heritage and more importantly it is OUR expertise.
At the end your will have to account for your technology decisions (or not if they proved to be an asset), but that doesnt means that those decisions weren’t made in context.
I still dont understand what you mean about a research institution. We are a business like Google. We just happen to have an small division that works very closely with Academia, in things that may never be profitable, with the idea of taking research out of the labs and into the public as fast as possible.
Just as a side note, what makes you think that what you saw in the video is using Jupiter’s approach?
Federico Lois
@Soren Lassen
Soren thanks to respond the question, cause that shed some light over the approach you are following on federation. Do you have any idea where we will be able to see or even contribute in the discussions about P2P federation?
Based on the actual approach to federation, do you have any idea about how you are going to deal with network partitioning when dealing with federated waves?
Greetings,
Federico
Prasad V
@Federico Lois
I guess what one could appreciate is the truly remarkable achievement in Wave; ie a combination of Real Time collaboration with inherent time and social aspects, coupled with a powerful capability through bots, and with a federation model that is as simple as email.
In an increasingly social web adding a time aspect to each social interaction, adding bots to assist or even build intelligence from those interactions is a fundamental shift in the way we use and collaborate on the web.
Dissecting the implementation of Wave’s realtime algorithms and more or less downplaying it as not being a suprise is actually not honourable from a Computer Scientist’s perspective.
I thought you guys have seen that this is a step towards semantic web, unless blinded by one’s own business motivations.
“I have a dream for the Web [in which computers] become capable of analyzing all the data on the Web – the content, links, and transactions between people and computers. A ‘Semantic Web’, which should make this possible, has yet to emerge, but when it does, the day-to-day mechanisms of trade, bureaucracy and our daily lives will be handled by machines talking to machines. The ‘intelligent agents’ people have touted for ages will finally materialize.
– Tim Berners-Lee, 1999″
Federico Lois
@Prasad V
Again not being a surprise is quite different to downplaying. I was pretty quick to acknowledge that the real innovation is on taking something like email that everybody take for granted and find ways to improve on it. The technology itself is just an enabler with very cool side properties, but as noted the field is generally not known (even among computer scientists). We started from an idea, and end up in that very same technology too, if you had being exposed to the underlaying ideas sooner or later you end up in the right place. The barrier to entry may be higher that with other technology but nothing that cannot be overcome with time, dedication and good engineering practices.
Technology decisions must be made in context, their context called for dOpt, ours not. Technology is technology, it really doesnt make a difference (just for the record, we would use Waves both on the human and the technical grounds alike). If now I tell you that you can do photographic collaborative editing and manipulation, after being exposed you will say: “of course”, the typical response for the unexposed (I got several of those) is why do I want to collaborate on that, it cant be done workflow wise or some other variant.
The real innovation was finding a way to bring to the masses concepts that has been confined to the lab for years. As you noted, the technology itself is revolutionary on the social aspects, not on the technical ground.
In fact that is why I said explicitly: “First I have to say that I really like what I saw, and the demo was flawless in almost every possible aspect, kudos to Google’s Wave Team on that, its great to see innovation in such a raw form from the guys that did it in the first place”, because we think they did an amazing job on putting those ideas to work and why not overcome the initial skepticism of people (We were there too).
About the semantic web, we definitely have seen that google and others are on the right track but I would put as current exponents Wolfram Alpha, or the Human Language Processing people at both Google, Microsoft Research and several well known universities. Waves is more about human-human interaction than computer-computer, even if we have already found some interesting scenarios where it can play a very interesting role that are not really human oriented. Posting something to my blog, using the right platform from the computer-computer standpoint is like 10 lines of code, things get really interesting when there is people on the other side
Soren Lassen
Please join the discussion at
http://groups.google.com/group/wave-protocol
where you can reach both the Google Wave engineers and many others who are interested in real-time collaboration.
With regard to network partitioning, for any given wavelet and any given “remote” wave provider the question is whether or not the provider can reach the “host” wave provider which hosts wavelet. If it can, it can share in the collaboration. It it cannot, it only has whatever read-only copy of the wavelet it may have obtained before it lost connectivity. In the last case, the users of the disconnected remote wave provider must make do with a read-only view of the wavelet. If they want to continue the conversation, they can initiate a (locally hosted) private reply, which will become available to other wave providers when connectivity is reestablished.
In future versions of the protocol, if remote wave providers transform their users’ operations locally, network partitioning becomes a more complex problem.
» Despídase de la Colaboración
[...] Como saben, entonces, desde Corvalius estamos trabajando en desarrollar una tecnología base que permita la aplicación de Co-Operación en todos los dominios expuestos. Suena ambicioso, y lo es
. Realmente tenemos las expectativas altas y creemos estar a la altura de las expectativas. Claramente no poseemos los recursos de Google, pero eso no es un impedimento significativo aún. [...]
sim-sim
“or you have the OT Framework embedded into your own applications for your own use”
Whats a concrete or recommend OT Frameworks for applications means?
Federico Lois
@sim-sim
Hi sim-sim,
As Seba told me that you are translating the blog entries into russian, I will try to explain in detail the phrases… You can also add this comments as references too, to explain a little better the context.
OT = Operational Transformation. There are several approaches to perform the transformation required to allow several parties to modify a shared state data structure; one of those is the Operational Transformation. Nonetheless there are also quite a few OT Frameworks arround, in the sense that the operational transformation functions are useless without a Operational Transformation Framework (dOpt is just one of those). There are also extra issues to keep in mind, as where do you perform the Operational Transformation (in a client, in a server, in both?). So basically what we are saying here is that you are free to embed the OT Framework (including the transformation) to be done directly by your application as if it were a server in itself. I have been working on this quite some time, so to me is pretty clear, did it makes sense?
Greetings,
Federico
sim-sim
here this article in russin:
http://moscow-gtug.org/index.php/topic,20.0.html
Sebastian Fernández Quezada
Thank you very much sim-sim!! This is very valuable to us!!