Archive for December, 2008

The Formalizer Revolution

Wednesday, December 31st, 2008

I’ve already discussed formalizers briefly a couple of days ago, and I’ll go on exploring them in future posts as well; but in order to be thorough, I’ll paraphrase:

Formalizers are form fields that accept text input of a specific kind, and allow the users some flexibility to input text in a more natural way.  The formalizer mechanism then takes the input, interprets it, and converts it into formal input that the application can understand.

For example, take a look at Remember The Milk; an amazing web application, hands down.  One of its nicest features is the ability to make task creation a little easier for the user. Most of their fields require some sort of format – like the Due date field shown below.  Editing this field is not done traditionally through drop down lists, or even a calendar popup, but simply by typing in your preference in natural language.  The screencapture below shows how I simply typed the word “tomorrow”, and the formalizer engine understood it and returned the correct date:

Remember The Milk due-date formalizer

The Phenomenon

Science fiction movies have always shown computers that you can actually talk to like human beings; but alongside that fantasy lies a disappointing reality of a long history of people talking down to computers, asking them to do their bidding in the most unintelligent way imaginable.  We’ve actually developed a new way of communicating with computers – mainly through forms – since we understood very early on that using our natural language around computers was like asking a banana to peal itself.

However, in recent years we’ve seen applications start to make an effort in order to actually understand us users and how we express ourselves naturallyNatural Language Processing (NLP) is a fascinating field of computer science that strives to help applications both express themselves to humans in a natural form, and to understand what they’re saying.  The science fiction fantasy of HAL actually speaking and understanding people is slowly becoming a reality.

So what’s the big deal?

I realize that many product marketing and product development people out there ask themselves “why should we care?  This is all technical stuff – let the lab-rats deal with it!”  That couldn’t be farther from the truth.  When you’ll understand formalizers and NLP, you’ll realize that a true revolution is taking place in user experience.

We are experiencing a surge of applications that allow people to share content for free with anyone they like.  I believe mass content is not a passing trend, but is the truest form the internet has taken so far.  It’s here to stay, at least for a while.

Some of these applications work with rich media, like YouTube and Flickr, and some with simple text, like Wikipedia and Remember The Milk.  Sharing rich media content is easy – you simply upload a file and you’re done, and provide some metadata to it, such as a title, description and tags.  However, when uploading simple text-based content, we usually get stuck with the most tedious yet annoyingly common way of text-based content acquisition – forms.

With the constant evolution of formalizer fields, and the incorporation of NLP into traditional forms, things are starting to look slightly brighter for us poor users.  We can begin to actually think and act as human beings around these applications, and rely on them to understand us a little better than their predecessors.  It’s amazing how gratifying it is to use your own words with an application when telling it that the due date for a specific task is “tomorrow” rather than filling in a date form packed with drop-down lists.

Formalizer fields, I believe, will only evolve to be more sophisticated and capable of accepting text in a natural language.  At the very least, they should evolve to a point where more computer illiterate people can start communicating properly with applications.

Any product department that is involved with a text-based, content related product should focus their efforts on providing more formalized fields to users – not only to make their application more usable, but to lower the bar for more people so that they can generate more content.  Needless to say, the more content we have as a company, and the more users we have, the more interesting and useful our product is.  And as content, especially online content, becomes an increasingly common thing in our lives, we’re bound to encounter more and more NLP formalizers as time goes by.

So play it again, Sam.

Support Wikipedia

Tuesday, December 30th, 2008

Wikipedia Timeline

Wikipedia is probably one of the single products that have significantly changed the world.  This website, which accumulates knowledge and provides it to everyone for free, joins a long line of revolutionary products that had made us who we are, and shaped civilization in a way that changed human life forever.

Let’s put them in context for a second:

  • The Alphabet. The first Egyptians and Phoenicians, who revolutionized writing from hieroglyph-based to phonemic script, and ultimately given us the alphabet.  This made reading and writing a significantly easier skill to master, and instead of learning thousands of images and hieroglyphs, reading and writing were reduced to identifying a couple of dozen symbols.  Knowledge became more accessible to people all around the ancient world, and knowledge started to become the realm of humanity.
  • Numerals. The ancient Babylonians gave us algebra, and opened up a new way for us to view the world with an advanced arithmetic system.  And with the numeral system, which was first developed in ancient India, humanity was exposed to the wonders of mathematics, and the infinite possibilities it created.
  • Law. Lawmakers and lawgivers, like Dracon and Solon of ancient Greece, had given us access to the law of the land.  Law, which was a private practice known only to selected elite, was made public, readable, and accessible by all.  Our lives as citizens were revolutionized with this basic freedom of knowledge
  • Print. Johannes Gutenberg found an efficient way to spread the written word throughout the world with his printing press.  By utilizing printing, books were no longer the sole property of monasteries and royalty, but spread across nations and continents.  Literacy levels sprung almost immediately, allowing us to learn, improve and achieve.
  • Public knowledge. Wikipedia’s achievement is no lesser than any of the above.  This website has managed to turn human wisdom, learning and education into an organized, categorized and free enterprise.  Knowledge is no longer the province of the few, but the realm of all humanity.  It is available from any place on the planet, in numerous languages, and can be enriched by any one of us.  Wikipedia has given us the freedom to learn, to grow, and to become better people by acquainting ourselves with our world and the civilizations within it.

Imagine a world without any of the above discoveries and developments – a world without literacy, science, knowledge, learning, law, and civilization. Denying the world of Wikipedia is no different.

We owe much of our children’s future to Wikipedia.  So let’s help them help us, and let’s all take part in this effort to make knowledge a public domain.

The 7 basic rules for creating advanced forms

Sunday, December 28th, 2008

FormForms are a pain. They’re boring, tedious, unattractive, and downright annoying. Filling out a form is like bending your mind in order to tell a slow person how to get somewhere fast. But unfortunately, they’re also quite necessary; they’re sometimes the only way for us to communicate with an application.

Once, we used to fill out a form, submit it, and wait for its-majesty-the-server to come back with a favorable response. The less benevolent answer would be a series of fields that were filled in incorrectly. This whole process feels quite bureaucratic, with us sitting at the edge of our seats while our form is being reviewed. Also, when it comes back with multiple errors, we need to go one by one and correct them, usually by looking for flaming red indicators telling us how silly we are for not selecting a username with 6-12 characters in it, or a password that contains uppercase and lowercase letters, or an about field that is formatted like a proper Haiku. And to add insult to injury, security related fields, such as passwords, and authorizing fields, or “I accept the terms” checkboxes, would have to be filled in again. So, basically, the user would sometimes go through numerous submits before moving forward.

Many product development people realize this, which is why we see forms that are becoming smarter and more helpful. It’s basically an “if you can’t get rid of them, improve them” approach. So new methods were created to make forms more user-friendly, and to make them appear as if they’re watching over you and correcting you while you fill in the form.

Here are a few ways to improve forms that could work with the user, and not vice versa:

  1. Tips. Very simple to implement, and quite helpful. When the user gets to fill in a specific field (focuses on it, in browser terms), the field shows a tip, usually on what the field is about and how best to fill it out.
  2. Real-time validity checks. Takes care of approving or disapproving your input as you type it, providing useful tips and suggestions in the process.
  3. Availability checks. This has mostly to do with username fields, where you select a username and are allowed to check its availability without actually submitting the form. When combined with real-time validity check, you don’t need to take your fingers off the keyboard to get an approval for your username selection.
  4. Password strength. With the rise of online security awareness, this simple indicator might do wonders to the user’s peace of mind. A bar-layout-color-coded feedback is displayed next to the field, telling the user how weak or strong the password is. This very responsive gadget encourages the user to do well, to get to the green, and to select a harder-to-break password.
  5. Auto-suggest. This goes the whole nine yards in terms of helping the user accomplish the task of filling in a field. While typing, the form actually suggests a possible value for the field based on the input typed so far. The user can then accept it, and dodge the most annoying task there is – typing. It’s not an easy thing to implement, but quite worth it.
  6. “Formalizers”. Many fields require a specific format, like a date field or a location field. Most forms use drop-down lists to provide the user with options, but mainly to make sure they fill in the field correctly. “Formalizers” are field mechanisms that accept input written in natural language, and automatically convert them to a properly filled-in field. For example, a date formalizer could accept dates written in natural language, rather than selected from dropdown lists, and also accept relative terms, such as “tomorrow” or “next Monday”. Again, quite an advanced feature, which is becoming ever more popular.
  7. One-time humanity check. We all know the terror of spam, and how forms are an easy target for such attacks. Many have adopted anti-spam checks, or humanity tests, to combat this phenomenon. As users, we accept this, though it makes a dull task even more wearisome. But there’s no reason why we should go through this more than once at a given form. If a form has returned with a response, for some reason (when all of the above failed), and the user has already passed the humanity test, it assumes that no sudden transformation has occurred, and that the human is still there; so the anti-spam field does not appear for the second time.

Under the hood

Apart from making the form-filling process less of a burden, it actually saves resources for the application itself by utilizing the processing power of the user’s computer to do some of these checks (validity checks and password strength). This means that instead of using the server to make these validity checks, the browser itself performs these operations, relieving the server of a substantial amount of processing (when multiplied by many-many users).

Even when the server is needed to verify a field (for availability checks, for example), the data posted and received is smaller in comparison to a whole page being sent to the user’s browser.  So instead of asking the server to verify a whole form, and to send an entire page, formatted and all, back to the browser, it is only used to verify a single field, and usually responds very succinctly.  This ultimately removes quite a lot of load from the server, and has a significant impact on the site as an operation.