Posts Tagged ‘form design’

reCAPTCHA Recycles Human Effort

Friday, January 9th, 2009

The American astronauts were rumored to be using a special Space Pen, which was developed specifically to work in zero-gravity conditions. It used a pressurized ink cartridge, enabling it to write upside down (Seinfeld fans should smile by now) and, of course, be used in space. It is believed that NASA spent a total amount of $11 million dollars to develop that pen.

The Russians gave their Cosmonauts a pencil.

Whether this story is accurate or not, it brings out a common product development truth – all things being equal, the simplest solution tends to be the right one. Hence the abbreviation K.I.S.S. – Keep It Simple, Stupid! And reCAPTCHA certainly did achieve a simple solution to several problems at once.

reCAPTCHA is a special captcha form component for identifying humans and rejecting spam and bots.  It is done through the traditional means of displaying distorted words for the users to type and verify. However, instead of generating a random word, reCAPTCHA displays words that were scanned from real text books, thus helping in the global effort of digitizing old texts.

I ran into reCAPTCHA a few months ago and was impressed by the concept of “recycling” human actions – taking something we do so commonly, almost without thinking, like verifying our humanity with forms through captchas, and harnessing it to both combat evil (spam and bots) and do good (digitize texts).

Name: reCAPTCHA
Website: recaptcha.net
Type: Captcha component
reCAPTCHA is a system that uses captcha to help digitize the text of books while protecting websites from bots attempting to access restricted areas.
reCAPTCHA

Value

Let’s take a second to ponder over a very disturbing fact: if we didn’t have spam or bots that were driving our websites off the wall – we wouldn’t have had reCAPTCHA, and as a result, many textbooks wouldn’t have been digitized as part of a universal effort.

When thinking of it that way, reCAPTCHA’s value starts to get a whole new meaning. It becomes very much like modern medicine – it wouldn’t have been discovered had we not gotten ill; I suppose we owe plenty of our discoveries to urgent needs or even accidents, such as the genome project, antibiotics, Champagne, and so forth. It’s strange to vocalize it, but many people around the world will be exposed to important texts digitally thanks to the plague called spam and bots.

That being said, lets examine reCAPTCHA’s value in two aspects: intrinsic value and global contribution.

Intrinsic value

reCAPTCHA has a very granular offering – it provides your product with a solution to something you’ll probably need as part of your registration form, or any other form that requires user authentication. You need to sort out the humans from the bots that attack your site.

Your R&D will most likely tell you that developing such a component is very simple, and there are plenty of open-source solutions out there that you can use. And they’ll be right to say so. Basically, all you need is a tiny utility that generates a random word, and displays it in a funny way. You can add colored squiggly lines if you’re overly creative. And that’s about that.

But reCAPTCHA’s solution has two main advantages:

  1. It’s off the shelf. You just cut and paste a little piece of code, and you get all of the intrinsic functionality already contained within it; also, the processing is done on the reCAPTCHA servers, which might free your own servers slightly, but on the other hand relies on a server-farm that is out of your control latency-wise. However, if Twitter and Facebook use it, then I guess it’s all right.
  2. It’s fully loaded. Two paragraphs ago, it sounded quite simple to make a workable captcha. However, to make it right, you need to consider people who might not be able to read your distorted image. Sometimes, the text is illegible, and needs to be changed. Again – implementing something like this is not like inventing the space pen, but why bother when this elegant component already exists?

Unreadable

(I’m not sure my dad would be able to read this one…)

Global contribution

I love recycling; not only in my trash habits at home, but also as a goal to live a greener life. Don’t worry – I don’t spend my spare time hugging trees, but I definitely believe that there’s a practical approach to recycling, which can ultimately be applied to business.

“I think there is a world market for maybe five computers,” according to Thomas J. Watson Sr., formerly the president of IBM. I’m afraid Mr. Watson may have missed the mark slightly. Our collective processing power today is gargantuan, and some applications make use of it very effectively.

File sharing, legal or not, is a perfect example for recycling processing power, as well as the wonderful SETI@Home project at Berkeley, which is currently the largest distributed computing effort around the world, utilizing over 3 million computers worldwide in order to analyze data from radio telescopes in search for extra-terrestrial intelligence. reCAPTCHA goes along those lines, but instead of recycling wasted processing time, they’re recycling wasted human effort.

Call me an idealist, but as a product junkie I think that part of making the world a better place revolves around reducing waste and channeling effort. Most of us share their hosing with others, use platforms developed by the online community, incorporate maps drawn by other companies, and make it an effort to make sure we reduce our costs and increase the productivity of our company.

With reCAPTCHA we have a double benefit – we get a ready-to-use component, and join a world effort to bring knowledge to the masses. That alone is a value in itself.

Value summary: Provides an off-the-shelf component that answers a specific requirement, while helping in a global effort to digitize texts.

Usability

reCAPTCHA

One thing to consider when creating such a generic and simple component is not to overdo it. I’m sure product development people would have loved to sink their teeth into this product and design their heart out with plenty of cool features, like changing the image size, making it full-screen, creating a hovering magnify-glass to make the letters bigger, etc. But that would have turned this component into a monster.

When dealing with a component so basic in its functionality, the product developer needs to constantly take a step back and look at the big picture. It’s terribly easy to look at your product under the microscope and be so caught up with its tiny world – I’m sure many of us have been there and realized our overzealousness.

One has to remember that this type of granular component lives inside the context of a foreign web page, which contains a form in it. It’s mostly the last, and least favorite, field the user needs to fill in. So it’s better to keep it as simple and as non-intrusive as possible. We’ve already agreed that forms are a pain – let’s not make them any worse.

reCAPTCHA managed to keep it simple. That’s their first great achievement.

Other than that – the usability is quite basic; after all, this is merely a form field. You can change the text with a new set of words, in case the image that’s presented to you is too hard to read, or you can change the text into an audio challenge taken from an old time radio show (therefore also transcribing old recorded shows).

That’s it; in terms of usability, the product fits the requirements.

Usability summary: Simple and complete.

Layout and design

Once again – such a simple widget doesn’t have much room for an elaborate layout and design schemes. However, one thing that I would like to see is customizable display: my website and forms are usually of a specific design, and I would have liked to make this field look and feel as a native field to my site. I have no problem with sharing credit or double branding, but reCAPTCHA’s choice of Bordeaux colors does not necessarily fit my overall website design.

Layout and design summary: Simple, though cannot be customized.

Under the hood

When I initially heard of reCAPTCHA, the first question that popped into my mind was “how does it work?” If the words that are shown to me are the ones that are used to digitize book, then how does the captcha verify my input as correct? Or else – if the captcha engine already knows the meaning of the word, then what part am I playing here exactly?

And here is where reCAPTCHA came up with the pencil solution.

After the texts are scanned, they go through an OCR process to interpret it. However, OCR is never perfect, and many words remain unresolved. Yet the OCR mechanism is clever enough to at least indicate which words were not interpreted correctly.

So reCAPTCHA came up with a very simple, yet ingenious solution: each user is presented with two words, not one. One word is a word that was successfully acquired by the OCR, while the other one was not. After typing these words, reCAPTCHA verifies the known word with your input; if the match is correct, it assumes that the second, unknown word is also correct.

This user-interpreted word is then given to a few other users for recapturing, and then, after cross verification, the word is declared as properly interpreted.

Why was I so impressed? Because, as a former engineer, I would definitely have overcomplicated something this simple. Can’t help but to doff your cap – well done!

Bottom line

When creating your online forms, it is great to be able to take part in a worldwide effort, and harness the power of the masses to a global endeavor. When your website joins the ranks of websites like Facebook, TicketMaster, Twitter, StumbleUpon and Craigslist, who already use reCAPTCHA, you know that you’re with good company.

Formalizers and Search Engines

Thursday, January 8th, 2009

Google search

Search engines are possibly the most commonly used, and perhaps the oldest type of formalizers out there, which many of us take for granted.

A search form has one major formalizer field, which requires a search query. The user can be quite expressive when filling in this field, while the formalizer field takes care of certain types of input which is expected. Google have been true pioneers in this sense of formalizer usage.

Surely, we all know that there is a specific format for search – we use quotation marks (“”) to group words together, we use logical operators such as OR and AND, or sometimes even use the plus (+) and minus signs (-), especially if you’re an advanced user. However, there are a few things popular search engines have taken care of:

  • Stop words. Also known as noise words in some circles, they are words that are filtered out of search queries, usually since they’re too common and appear far too many times in searchable documents. These words, like “is”, “are”, “the”, “you”, “me”, and many others, are omitted from search queries automatically by the search engine, in order to provide a more relevant search result list.
  • Spelling. Google really made a difference here, both by finding spelling mistakes in your search query and suggesting a correction for you in the search results, and providing the means for other applications to utilize their spelling mechanism with their SOAP search API.
  • Contextual search. When typing a specific search phrase that requires a specific type of data, Google makes sure they provide you with this data as quickly as they can, with the tools that they have. You can get stock quotes, airline schedules and word definitions simply by asking for them. This saves a lot of time and hassle by simply thinking like people do, so a simple phrase like “kilometer in mile” typed into the Google search box will make the simple conversion for us, and display it at the top of the page.

What can we learn from Google?

Many applications, especially web applications, provide search capabilities to their users. With search being such a common use-case, not to mention a perfect way to learn of your users’ habits, product development managers need to make their search engines as usable and formalizer-friendly as they possibly can.

Here’s a short checklist:

  1. Auto-suggest. When users start typing search queries, try to assist them with suggestions based on common search queries.
  2. Spelling. As mentioned before, there are services for checking the spelling of search queries out there, like the one Google offers us. Therefore, there’s no need to develop it in-house, and you can literally use the best tool out there. This way, users won’t get frustrated when a misspelled query gets sent back with no results.
  3. Natural language. This is probably the most specific, and therefore most complicated aspect of formalizer-friendly search engine. It requires you to define a way for the user to communicate with your application’s search engine in a natural language. Try to find common search phrases and to identify patterns that occur the most, and you’ll be able to see how you can go towards your users and help them by understanding their queries better.

Combining these three elements together creates a powerful search user experience, and positions you immediately above your competitors in terms of usability and barrier of entry.

For example: say you have a dating website, and your search box allows users to look for possible people to meet. You could provide a boring form that would require an “advanced search” option to be clicked (possibly the worst use-case a search engine could have, but more on that in a later post).

On the other hand, you can define a few search patterns that upgrades your search field into a formalizer. Such fields are not search-boxes in their classical sense, and are usually preceded with a question, such as “what kind of date are you looking for?” to which the user can answer “middle-aged men in the greater LA area.”

And now, the hard work begins: your language processor should be able to understand this sentence and extract meaningful information out of it.  It should understand that “middle-aged” means a certain age group, and that “the greater LA area” is a geographic region. Again – not the easiest of tasks, but once implemented, it goes a very long way in terms of usability.

This, combined with a spelling service and an auto-suggest utility, makes sure the user types the words correctly, or at least minimizes the number of mistakes the user makes. This allows your formalizer mechanism to be right more times than wrong and ultimately makes your users happier. And happy users will return again tomorrow, no doubt.

The Formalizer Revolution

Wednesday, December 31st, 2008

I’ve already discussed formalizers briefly a couple of days ago, and I’ll go on exploring them in future posts as well; but in order to be thorough, I’ll paraphrase:

Formalizers are form fields that accept text input of a specific kind, and allow the users some flexibility to input text in a more natural way.  The formalizer mechanism then takes the input, interprets it, and converts it into formal input that the application can understand.

For example, take a look at Remember The Milk; an amazing web application, hands down.  One of its nicest features is the ability to make task creation a little easier for the user. Most of their fields require some sort of format – like the Due date field shown below.  Editing this field is not done traditionally through drop down lists, or even a calendar popup, but simply by typing in your preference in natural language.  The screencapture below shows how I simply typed the word “tomorrow”, and the formalizer engine understood it and returned the correct date:

Remember The Milk due-date formalizer

The Phenomenon

Science fiction movies have always shown computers that you can actually talk to like human beings; but alongside that fantasy lies a disappointing reality of a long history of people talking down to computers, asking them to do their bidding in the most unintelligent way imaginable.  We’ve actually developed a new way of communicating with computers – mainly through forms – since we understood very early on that using our natural language around computers was like asking a banana to peal itself.

However, in recent years we’ve seen applications start to make an effort in order to actually understand us users and how we express ourselves naturallyNatural Language Processing (NLP) is a fascinating field of computer science that strives to help applications both express themselves to humans in a natural form, and to understand what they’re saying.  The science fiction fantasy of HAL actually speaking and understanding people is slowly becoming a reality.

So what’s the big deal?

I realize that many product marketing and product development people out there ask themselves “why should we care?  This is all technical stuff – let the lab-rats deal with it!”  That couldn’t be farther from the truth.  When you’ll understand formalizers and NLP, you’ll realize that a true revolution is taking place in user experience.

We are experiencing a surge of applications that allow people to share content for free with anyone they like.  I believe mass content is not a passing trend, but is the truest form the internet has taken so far.  It’s here to stay, at least for a while.

Some of these applications work with rich media, like YouTube and Flickr, and some with simple text, like Wikipedia and Remember The Milk.  Sharing rich media content is easy – you simply upload a file and you’re done, and provide some metadata to it, such as a title, description and tags.  However, when uploading simple text-based content, we usually get stuck with the most tedious yet annoyingly common way of text-based content acquisition – forms.

With the constant evolution of formalizer fields, and the incorporation of NLP into traditional forms, things are starting to look slightly brighter for us poor users.  We can begin to actually think and act as human beings around these applications, and rely on them to understand us a little better than their predecessors.  It’s amazing how gratifying it is to use your own words with an application when telling it that the due date for a specific task is “tomorrow” rather than filling in a date form packed with drop-down lists.

Formalizer fields, I believe, will only evolve to be more sophisticated and capable of accepting text in a natural language.  At the very least, they should evolve to a point where more computer illiterate people can start communicating properly with applications.

Any product department that is involved with a text-based, content related product should focus their efforts on providing more formalized fields to users – not only to make their application more usable, but to lower the bar for more people so that they can generate more content.  Needless to say, the more content we have as a company, and the more users we have, the more interesting and useful our product is.  And as content, especially online content, becomes an increasingly common thing in our lives, we’re bound to encounter more and more NLP formalizers as time goes by.

So play it again, Sam.

The 7 basic rules for creating advanced forms

Sunday, December 28th, 2008

FormForms are a pain. They’re boring, tedious, unattractive, and downright annoying. Filling out a form is like bending your mind in order to tell a slow person how to get somewhere fast. But unfortunately, they’re also quite necessary; they’re sometimes the only way for us to communicate with an application.

Once, we used to fill out a form, submit it, and wait for its-majesty-the-server to come back with a favorable response. The less benevolent answer would be a series of fields that were filled in incorrectly. This whole process feels quite bureaucratic, with us sitting at the edge of our seats while our form is being reviewed. Also, when it comes back with multiple errors, we need to go one by one and correct them, usually by looking for flaming red indicators telling us how silly we are for not selecting a username with 6-12 characters in it, or a password that contains uppercase and lowercase letters, or an about field that is formatted like a proper Haiku. And to add insult to injury, security related fields, such as passwords, and authorizing fields, or “I accept the terms” checkboxes, would have to be filled in again. So, basically, the user would sometimes go through numerous submits before moving forward.

Many product development people realize this, which is why we see forms that are becoming smarter and more helpful. It’s basically an “if you can’t get rid of them, improve them” approach. So new methods were created to make forms more user-friendly, and to make them appear as if they’re watching over you and correcting you while you fill in the form.

Here are a few ways to improve forms that could work with the user, and not vice versa:

  1. Tips. Very simple to implement, and quite helpful. When the user gets to fill in a specific field (focuses on it, in browser terms), the field shows a tip, usually on what the field is about and how best to fill it out.
  2. Real-time validity checks. Takes care of approving or disapproving your input as you type it, providing useful tips and suggestions in the process.
  3. Availability checks. This has mostly to do with username fields, where you select a username and are allowed to check its availability without actually submitting the form. When combined with real-time validity check, you don’t need to take your fingers off the keyboard to get an approval for your username selection.
  4. Password strength. With the rise of online security awareness, this simple indicator might do wonders to the user’s peace of mind. A bar-layout-color-coded feedback is displayed next to the field, telling the user how weak or strong the password is. This very responsive gadget encourages the user to do well, to get to the green, and to select a harder-to-break password.
  5. Auto-suggest. This goes the whole nine yards in terms of helping the user accomplish the task of filling in a field. While typing, the form actually suggests a possible value for the field based on the input typed so far. The user can then accept it, and dodge the most annoying task there is – typing. It’s not an easy thing to implement, but quite worth it.
  6. “Formalizers”. Many fields require a specific format, like a date field or a location field. Most forms use drop-down lists to provide the user with options, but mainly to make sure they fill in the field correctly. “Formalizers” are field mechanisms that accept input written in natural language, and automatically convert them to a properly filled-in field. For example, a date formalizer could accept dates written in natural language, rather than selected from dropdown lists, and also accept relative terms, such as “tomorrow” or “next Monday”. Again, quite an advanced feature, which is becoming ever more popular.
  7. One-time humanity check. We all know the terror of spam, and how forms are an easy target for such attacks. Many have adopted anti-spam checks, or humanity tests, to combat this phenomenon. As users, we accept this, though it makes a dull task even more wearisome. But there’s no reason why we should go through this more than once at a given form. If a form has returned with a response, for some reason (when all of the above failed), and the user has already passed the humanity test, it assumes that no sudden transformation has occurred, and that the human is still there; so the anti-spam field does not appear for the second time.

Under the hood

Apart from making the form-filling process less of a burden, it actually saves resources for the application itself by utilizing the processing power of the user’s computer to do some of these checks (validity checks and password strength). This means that instead of using the server to make these validity checks, the browser itself performs these operations, relieving the server of a substantial amount of processing (when multiplied by many-many users).

Even when the server is needed to verify a field (for availability checks, for example), the data posted and received is smaller in comparison to a whole page being sent to the user’s browser.  So instead of asking the server to verify a whole form, and to send an entire page, formatted and all, back to the browser, it is only used to verify a single field, and usually responds very succinctly.  This ultimately removes quite a lot of load from the server, and has a significant impact on the site as an operation.