RSS feeds"Real-time information isn’t,” wrote someone who listened to a series of lectures about real-time search on a speaker evaluation form. Search technology is not as resilient as a £50 Timex watch on the high street.
The observation that something in the world of electronic information is not what it seems makes miscommunication the baseline for many discussions. One example is the paradox of Twitter messages. On one hand, it is bursting with nonsense; on the other, it contains useful intelligence for some organisations. Twitter is a waste of time, yet it can slash the time and cost of promoting certain types of events. Paradox? Paradigm?
Organisations have to look at real-time information and understand what specific benefits can be gained from low-latency systems. Low latency – not quite real-time but delays of milliseconds only – is a useful concept.
Famed for 15 minutes
Exalead, a Paris-based information systems company, built a system search-enabled for logistics company Jefco. Before its implementation, an employee could spend half a day or more answering a routine question like “Where is my delivery?” The system delivers the report in milliseconds using data that is no more than 15 minutes old. Exalead makes clear that in complex distributed systems, latency is an engineering problem, and the 15 minute delay for a report that once took hours is a significant improvement. Exalead describes its system, quite accurately, in terms of specific performance metrics.
Other vendors take a different approach, using terms like “acceptable performance” and “rapid content processing”. Thus, the definition of what is meant by “low latency” becomes one of the key ways of determining what real-time means.
Real-time is becoming an important component of business intelligence, enterprise resource planning systems, and consumer services. For these, delays between receiving fresh information and its being available to the user of an information access tool have become an engineering challenge.
Google Wave (http://wave.google.com) enables a business to collect information and have it updated and available to authorised users of that digital container with low latency. A test product from Google’s engineering labs, it provides a glimpse of one way to make communications and collaborative work more real-time; that is, tasks can be accomplished without the standard hunting for a specific document or flipping through a calendar looking for the date on which a specific decision was taken.
A second glimpse of low-latency applications foreshadows how some complex concepts will be presented to students, and to staff on training programs. Using Google’s mapping technology you can explore the ruins of Pompeii in real time (http://tinyurl.com/yl7jkwu). In this low-latency application, the notion of “time” is that of a tourist exploring the city devastated by a volcanic eruption centuries ago. Google delivers an interactive experience that puts the user in control of what is viewed either from a distance or close up. With a mouse, the visitor to the ancient city can walk down its streets in real-time. Without wandering into the philosophical aspects of a real-time exploration of an ancient city, let me point out that many complex concepts can be conveyed with this type of rich, low-latency system. A person learning the Byzantine pathways of London to pass the test for “the knowledge”, or a doctor learning a new medical procedure, can use these low-latency systems in useful ways.
Different approaches
Microsoft and Yahoo have added content from Facebook and Twitter to their searches. Microsoft offers BingTweets (http://bingtweets.com) which is a combination of Twitter trends with Bing search results and BigTwitter (http://bing.com/twitter), which is Twitter hot topics. Microsoft licenses content directly and uses its own method for processing and ranking results from the sources. Most users perceive the Tweets and Facebook content as real-time i nformation.
Some latency is introduced but, speaking broadly, the content in the Bing services is available within minutes of becoming available to Microsoft’s crawler.
Yahoo has integrated the OneRiot (http://www.oneriot.com) real-time system into Yahoo. OneRiot processes a wide range of content, emphasising that it is a “real-time search service”, although it does not define precisely what that means. My tests indicate content freshness is on a par with the Microsoft system’s index.
The inclusion of these real-time results makes it clear that users have an appetite for content that comes from non-traditional sources without the lengthy editorial production cycles that were once the norm for print and online information services. Consumer expectations have changed.
When I look at what these examples trigger in my work, I see several important glimpses of real-time (low latency) systems in the months ahead. First, users expect online systems to deliver snappy performance and fresh content. Users may not know what “low latency” means, but a failure to deliver on both counts could damage companies or even put them out of business.
Second, vendors have a market signal to which to respond. Saying a system is real-time when it delivers outdated information is risky. My view is that users’ expectations will make vendors more accountable. Perhaps the disinformation and outright fabrications in some marketing collateral will be replaced with clear, factual statements about what a system can deliver. In my experience, vendors exploit buzzwords and the customer often learns too late that misdirection or fuzziness was an intent of the vendor.
Finally, business professionals have an opportunity to make use of technologies that can present related information in a way that is compelling and more accessible to some constituents. Awareness of companies like Apnoti (http://www.apnoti.com) – a German firm able to index real-time pricing data – helps trigger new ways of thinking about solving somewhat routine, often time-intensive business tasks.
Looking forward, real-time information is one of the next frontiers in online access. You are welcome to Tweet this opinion, post it on your Facebook wall, and distribute the idea via RSS. Real-time has “taken a licking but keeps on ticking” as John Cameron Swaze said in the 1950s.
Stephen E Arnold is an IT consultant
