Posts Tagged ‘Formalizers’

Formalizers and Search Engines

Thursday, January 8th, 2009

Google search

Search engines are possibly the most commonly used, and perhaps the oldest type of formalizers out there, which many of us take for granted.

A search form has one major formalizer field, which requires a search query. The user can be quite expressive when filling in this field, while the formalizer field takes care of certain types of input which is expected. Google have been true pioneers in this sense of formalizer usage.

Surely, we all know that there is a specific format for search – we use quotation marks (“”) to group words together, we use logical operators such as OR and AND, or sometimes even use the plus (+) and minus signs (-), especially if you’re an advanced user. However, there are a few things popular search engines have taken care of:

  • Stop words. Also known as noise words in some circles, they are words that are filtered out of search queries, usually since they’re too common and appear far too many times in searchable documents. These words, like “is”, “are”, “the”, “you”, “me”, and many others, are omitted from search queries automatically by the search engine, in order to provide a more relevant search result list.
  • Spelling. Google really made a difference here, both by finding spelling mistakes in your search query and suggesting a correction for you in the search results, and providing the means for other applications to utilize their spelling mechanism with their SOAP search API.
  • Contextual search. When typing a specific search phrase that requires a specific type of data, Google makes sure they provide you with this data as quickly as they can, with the tools that they have. You can get stock quotes, airline schedules and word definitions simply by asking for them. This saves a lot of time and hassle by simply thinking like people do, so a simple phrase like “kilometer in mile” typed into the Google search box will make the simple conversion for us, and display it at the top of the page.

What can we learn from Google?

Many applications, especially web applications, provide search capabilities to their users. With search being such a common use-case, not to mention a perfect way to learn of your users’ habits, product development managers need to make their search engines as usable and formalizer-friendly as they possibly can.

Here’s a short checklist:

  1. Auto-suggest. When users start typing search queries, try to assist them with suggestions based on common search queries.
  2. Spelling. As mentioned before, there are services for checking the spelling of search queries out there, like the one Google offers us. Therefore, there’s no need to develop it in-house, and you can literally use the best tool out there. This way, users won’t get frustrated when a misspelled query gets sent back with no results.
  3. Natural language. This is probably the most specific, and therefore most complicated aspect of formalizer-friendly search engine. It requires you to define a way for the user to communicate with your application’s search engine in a natural language. Try to find common search phrases and to identify patterns that occur the most, and you’ll be able to see how you can go towards your users and help them by understanding their queries better.

Combining these three elements together creates a powerful search user experience, and positions you immediately above your competitors in terms of usability and barrier of entry.

For example: say you have a dating website, and your search box allows users to look for possible people to meet. You could provide a boring form that would require an “advanced search” option to be clicked (possibly the worst use-case a search engine could have, but more on that in a later post).

On the other hand, you can define a few search patterns that upgrades your search field into a formalizer. Such fields are not search-boxes in their classical sense, and are usually preceded with a question, such as “what kind of date are you looking for?” to which the user can answer “middle-aged men in the greater LA area.”

And now, the hard work begins: your language processor should be able to understand this sentence and extract meaningful information out of it.  It should understand that “middle-aged” means a certain age group, and that “the greater LA area” is a geographic region. Again – not the easiest of tasks, but once implemented, it goes a very long way in terms of usability.

This, combined with a spelling service and an auto-suggest utility, makes sure the user types the words correctly, or at least minimizes the number of mistakes the user makes. This allows your formalizer mechanism to be right more times than wrong and ultimately makes your users happier. And happy users will return again tomorrow, no doubt.

The Formalizer Revolution

Wednesday, December 31st, 2008

I’ve already discussed formalizers briefly a couple of days ago, and I’ll go on exploring them in future posts as well; but in order to be thorough, I’ll paraphrase:

Formalizers are form fields that accept text input of a specific kind, and allow the users some flexibility to input text in a more natural way.  The formalizer mechanism then takes the input, interprets it, and converts it into formal input that the application can understand.

For example, take a look at Remember The Milk; an amazing web application, hands down.  One of its nicest features is the ability to make task creation a little easier for the user. Most of their fields require some sort of format – like the Due date field shown below.  Editing this field is not done traditionally through drop down lists, or even a calendar popup, but simply by typing in your preference in natural language.  The screencapture below shows how I simply typed the word “tomorrow”, and the formalizer engine understood it and returned the correct date:

Remember The Milk due-date formalizer

The Phenomenon

Science fiction movies have always shown computers that you can actually talk to like human beings; but alongside that fantasy lies a disappointing reality of a long history of people talking down to computers, asking them to do their bidding in the most unintelligent way imaginable.  We’ve actually developed a new way of communicating with computers – mainly through forms – since we understood very early on that using our natural language around computers was like asking a banana to peal itself.

However, in recent years we’ve seen applications start to make an effort in order to actually understand us users and how we express ourselves naturallyNatural Language Processing (NLP) is a fascinating field of computer science that strives to help applications both express themselves to humans in a natural form, and to understand what they’re saying.  The science fiction fantasy of HAL actually speaking and understanding people is slowly becoming a reality.

So what’s the big deal?

I realize that many product marketing and product development people out there ask themselves “why should we care?  This is all technical stuff – let the lab-rats deal with it!”  That couldn’t be farther from the truth.  When you’ll understand formalizers and NLP, you’ll realize that a true revolution is taking place in user experience.

We are experiencing a surge of applications that allow people to share content for free with anyone they like.  I believe mass content is not a passing trend, but is the truest form the internet has taken so far.  It’s here to stay, at least for a while.

Some of these applications work with rich media, like YouTube and Flickr, and some with simple text, like Wikipedia and Remember The Milk.  Sharing rich media content is easy – you simply upload a file and you’re done, and provide some metadata to it, such as a title, description and tags.  However, when uploading simple text-based content, we usually get stuck with the most tedious yet annoyingly common way of text-based content acquisition – forms.

With the constant evolution of formalizer fields, and the incorporation of NLP into traditional forms, things are starting to look slightly brighter for us poor users.  We can begin to actually think and act as human beings around these applications, and rely on them to understand us a little better than their predecessors.  It’s amazing how gratifying it is to use your own words with an application when telling it that the due date for a specific task is “tomorrow” rather than filling in a date form packed with drop-down lists.

Formalizer fields, I believe, will only evolve to be more sophisticated and capable of accepting text in a natural language.  At the very least, they should evolve to a point where more computer illiterate people can start communicating properly with applications.

Any product department that is involved with a text-based, content related product should focus their efforts on providing more formalized fields to users – not only to make their application more usable, but to lower the bar for more people so that they can generate more content.  Needless to say, the more content we have as a company, and the more users we have, the more interesting and useful our product is.  And as content, especially online content, becomes an increasingly common thing in our lives, we’re bound to encounter more and more NLP formalizers as time goes by.

So play it again, Sam.