Real-time wiki

From XMPP WIKI
Jump to navigation Jump to search

« back to Summer of Code 2006 «


Author: Grzegorz Grasza ›xek‹

Mentor: Mickaël Rémond

xmpp/mail: groz at gryf info

repos: svn://gryf.info/realtimewiki/trunk

test: http://wiki.gryf.info/ (not yet available)

blog: http://blog.xek.pl/ (future developments)

Abstract

Real-time wiki: a Wiki like it was predicted by Stanislaw Lem (a famous Sci-Fi writer):

"In an extreme instance, in which there is a Propervirt of less than 0.9%, the TEXT OF THE PRESENT PROSPECTUS may likewise undergo an ABRUPT change. If, while you are reading these sentences, the words begin to jump about, and the letters quiver and blur, please interrupt your reading for ten or twenty seconds to wipe your glasses, adjust your clothing, or the like, and then start reading AGAIN from the beginning, and NOT JUST from the place where your reading was interrupted, since such a TRANSFORMATION indicates that a correction of DEFICIENCIES is now taking place."

To achieve this, I will use a JavaScript jabber client library and write an XMPP client supporting Publish-Subscribe for notification of the page changes. The server side code serving the Wiki will be programmed as an extension to the ejabberd server, in Erlang.

Why jabber?

I think the JSJaC library is a good tool for this, since it provides a stable, authenticated XMPP connection, over which you can send content. It also overcomes many possible web-browser restrictions, such as a limit of 2 connections to a server, with a well designed HTTP Binding protocol, providing an easy way of real-time communication. Furthermore, the Publish-Subscribe module of ejabberd is very suitable for notifying of the Wiki modifications and maintaining the edit history.

Implementation

The implementation will consist of two parts: a JavaScript XMPP client with Publish-Subscribe capabilities and an ejabberd HTTP module, which will serve pages via http. The pages will be served directly from the Publish-Subscribe module.

The AJAX XMPP client will fetch the complete current version of a page, and then it will subscribe to this page's node. It will unsubscribe on the event of leaving the page, or when the user chooses to. Notice that in an event of a browser crash the currently viewed (or selected) pages will be memorized and they can be opened on the reentry to the website. Any edits of the page will be sent via pub-sub and distributed to everyone interested.

Deliverables

  • ejabberd Wiki module (probably independent from the current HTTP module, using yaws or other Erlang HTTP server) serving the Wiki pages
  • JavaScript jabber client, which will implement XEP-0060 for notifying of changes of currently viewed Wiki pages, submitting changes and subscribing/unsubscribing with a node-page

next versions:

(some of these features might turn-out simple and I will implement them in the first version)

  • the basic version should work with the current Erlang implementation of pub-sub, but probably it will need some polishing
  • history of previously visited pages
  • subscribing to not currently viewed pages
  • good looking animations of the page changes
  • more user oriented GUI changes and extensions
  • XEP-0070: Verifying HTTP Requests via XMPP (private Wiki)
  • support for non-HTTP XMPP clients?
  • XML diff algorithm and support?

Progress

I will have exams untill 30th June so I also have to study...

Week 1 (24th-31st May)

  • I made a repository where I commit my work: svn://gryf.info/realtimewiki/trunk
  • a basic layout and test environment
  • I found some helper functions, but it turned out they were buggy, so I rewrote them
  • I implemented a cross-window eval function which permits me to execute code across many open windows, this will permit browsing with only one connection and multiple tabs and windows!!! It is made with cookies, so it has some implications, like the cookie is sent to the server when it doesn't need to, and a limited number of cookies a browser can store (http://www.ietf.org/rfc/rfc2109.txt), but I made a message passing interface between windows I call Portal Cookies (reminds me of Erlang, anyone willing to implement Erlang in JavaScript?) and now there is a main window which controls others and can redirect messages from plain windows to others. It limits the needed number of cookies, but it complicates the later implementation, still I think it's worth it. I don't know of any similar cross-window message passing interface (apart from a similar commercial application I'm also writing) so it's quite unique and may be of interest to you if you do JavaScript. It may still have some bugs though.
  • and here is a round presence icon theme I made :)

File:Presence.png

Week 2 (1st-7th June)

  • I commited configuration for lighttpd
  • added some debbuging code with logging of all XML trafic in a nice window
  • sessions are ready
  • I found a useful mozilla option: javascript.options.strict which probably will let me write better code
  • there are some further fixes for div scrolling in mozilla which I will commit today
  • added CSS
  • starting to write pub-sub support

Week 3 (8th-14th June)

  • I decided to make a major design change - the pub-sub service will store full page versions instead of diffs, this will make the implementation of both the JavaScript client and the ejabberd module simpler. It's also in favor with the Jabber design philosophy: move complexity to the server (in this case the jabber server will need more storage ;) ). It will also make the ejabberd module more useful as a web front-end for other pub-sub services. I also looked at how some other wiki engines are implemented, and I think most of them store whole page versions, this way they are a lot easyer to manage.
In case you are wondering: I still plan tu make nice animations on page updates.
  • I tested the ejabberd pub-sub implementation
  • you can perform all basic queries to pubsub (create nodes, subscribe, publish, browse nodes, get items)
  • I made an atom feed to my svn commit log (done with XSLT)
  • I've chosen the javascript diff algorithm and implementation (http://neil.fraser.name/software/diff_match_patch/), and I'm off with coding the gui

Week 4 (15th-21th June)

Wiki basically works: there is gui for subscribing, unsubscribing, browsing nodes by their ids, publishing changes, managing nodes... There is a progress bar, which makes the thing look pretty responsive. I also tested it under internet explorer (using ies4linux) and it works, there are some problems under opera though.

Next things to do are linking, better crosswindow integration (for now don't open more than one window), gui for discovering nodes, animating diffs. But these are all extra features, so I will focus on the server part from now on.

Week 5 (22th-28th June)

The demo is up at http://wiki.gryf.info/

I'm starting to write the server...

Weeks 6-9 (29th June - 26th July)

I made myself a little break, but nevertheless I did some researching, reading safari online books that O'Reilly made available for soc participants (I couldn't find any useful Erlang or Jabber related books though :> ).

The server code is now mostly ready (it works, but needs some cleanup), it is not available on the demo page yet.

Week 10 (27th July - 2nd August)

I added linking across nodes and to other pages in erlang and JavaScript, you can bookmark both.

Summer of Code part of the project is now finished, look at future for future plans

Project Schedule

part I

  • a basic JavaScript client (100%)
    • working JavaScript environment (100%)
    • some helper functions and thinking (100%)
    • implementation of crosswindow eval function (100%)
    • session management and authentication (100%)
  • fetch pages (100%)
    • research pub-sub and it's Erlang implementation (100%)
    • simple edits (100%)
    • diff algorithm (100%)
  • managing wiki pages (100%)
    • subscribing (100%)
    • adding nodes (100%)

part II

  • Erlang module serving wiki pages (60%)
    • Erlang environment (100%)
    • Research (100%)
    • retreving pages from Mnesia (100%)
    • web-server (yaws 100%)
    • integrate JavaScipts (100%)
  • internal and external links (100%)

future

  • make the ejabberd pub-sub implementation more suitable for forum-like and wiki-like applications
  • HTTP Binding - YAWS integration
  • atom feeds
  • more configuration options - forum-like, blog-like behaviors

Bio

I am a 20 year old student of 'Faculty of Mathematics and Natural Sciences. College of Science' in UKSW University in Warsaw ( http://www.uksw.edu.pl/ ). I started programming in the third grade of high school (c/c++), and in the same year I qualified to the next to last stage of Polish Olimpiad in Informatics ( http://www.oi.edu.pl/ ) and it happened again the next year (2002/2003 2003/2004). Before going to university I started mobber ( http://mobber.gryf.info ). It was my first real-use application and it is the first open source mobile jabber client. Since then I have been busy studying, making commercial web sites in python and sending patches to open source projects from time to time. In the last few months I also gained experience programming in JavaScript and Erlang.


I think a real-time Wiki (or similar technologies) can revolutionize the way people will navigate the web - the whole Internet could be one day viewed in real-time, just like TV.

Links


  • jotlive - somewhat similar, commercial application
  • synchroedit - alpha, written in java, I didn't test it
Retrieved from "https://wiki.xmpp.org/web/index.php?title=Real-time_wiki&oldid=5676"