SharedBook: On Demand

The Magic Formula for the Right Balance

Lately I’ve been reading about some new methodologies that claim QA resources should be reduced to a minimum.

These so called “advanced” methodologies, mainly related to Web 2.0, argue that although QA is a “necessary evil,” reducing it will allow the development process to be shorter and more dynamic and flexible, giving companies greater strength in the market.

IMHO these progressive approaches have progressed a bit too much. It seems to me they do not take into consideration all the risks involved in such an act.

They do present one proposed solution – developers should become more responsible for their code. They should strive to write without bugs and perform the minimal necessary tests by themselves.

This solution totally contradicts the main purpose of this approach – flexibility, dynamics and quickness – it will take a developer twice as much time to develop a new feature if he tries to fulfill the above tasks, and the development process will become even more complicated and slower as a result.

Although no one knows the feature better than the person who wrote it, a QA person is a trained professional and will be able to produce twice the productivity in half the time.

Our basic assumption is that not all bugs will be found. Therefore QA’s goal is to find as many bugs as possible in a designated time and make sure every bug gets the proper attention.

But as you reduce the QA effort, more and more critical bugs will slip through and find their way into the production site. Those bugs will never go away. They are only hiding, lurking somewhere in the system waiting for the first customer to find them. Such bugs have a tendency to reveal themselves at the worst place and always at the worst time.

Critical bugs need to be fixed at any stage, and it is a known fact that the later you find the bug, the more painful and expensive it will be to fix it.

My experience has shown me that whatever you save in QA you will pay in tech support, and big time. The equation is simple – the less QA you have, the more tech support you will need, and as a QA & tech support manager, I can see both sides of the equation very clearly.

Shorter QA time and effort will enable end users to enjoy a more dynamic and enhanced application BUT … a customer will always be less frustrated by an absent feature than by one that doesn’t work.

In conclusion, no matter how much QA time and effort you invest, you will always need tech support. But our main goal is to find the magic formulas that enable us to balance the equation between the two.

New Year's Resolution

Yes there are several weeks until 2008, but I can't allow that fact to ruin my chance for yet another bad pun. The topic is image resolution and the way our system evaluates user submitted photos.

The way that digital photography has overtaken the world is truly extraordinary. I think of my parents who still won't use an ATM, but both are on at least their second generation of digital camera. Being a strong advocate of using technology for everyday life, this should warm my heart. But as it turns out, it creates a whole new set of challenges for a Web to print publishing company like SharedBook.

Though you'd be hard-pressed to find a camera today that takes a picture with less than 1M of pixels, there are so many ways that an image can become degraded.  People e-mail them, upload and download on file sharing sites, embed them in their blogs and any number of other things that can alter them from their original high resolution.  The problem is that it takes a keen eye to see the difference and browsers were built to make sure that images can be kept at a low resolution without the user seeing a difference.

As a publishing company we know that once a low resolution image gets printed, the eye will see it very differently.  So our application is built to monitor this and warn the user whenever an image falls below a set standard.  Such images are marked with a warning sign and the whole system is quite smart and dynamic.

But how do you set a standard?  This will always be a matter of opinion so we try to set our standards within a range.  And these standards are always under review to make sure that we are balancing the wants and needs of our customers.

But I'm finding the issue is even more difficult than just setting the right standard.  From a user experience, I'm struggling with a broader question:  What do you tell a user who has a substandard photo?  Suppose that the image is a treasured one and there is no other copy?  Do we scare away a sale saying that the image isn't good enough?

This is at the core of the complexity of Web site design.  There will never be a perfect answer that satisfies every user's need.  Instead it is more a matter of straddling the fence and hoping that the most users will derive the most benefit without sacrificing the experience of everyone else.
   

Today I made a chair.

Working for a high-tech company can be very frustrating sometimes. Basically, what we all do is actually move virtual text from one place to another.

In all my years of experience I have worked for quite a few companies that developed various products. Products for billing, banking, e-commerce, online catalogs, etc ...

I never felt what I was doing was very exciting, and was never able to show others (family, friends) what I was actually doing. When asked, I usually answered "something with computers." Somewhere deep inside I always wished I was a carpenter, so I would be able to show everyone the new chair I just finished.

A CEO in one of the companies I worked for once told me a nice story. He was working as the VP of sales for a very big media & TV corporation. His daughter asked what he was doing at work and he began to explain all about the large corporation and all the responsibilities he had. The little child got confused and bored, but eventually understood two words: sale & TV. So she asked him very excitedly, "So Daddy, you are selling TVs? That's great!" imagining her Dad going from door to door.

I also have a hard time explaining to my three kids what I do at work. Somehow, Quality Assurance & Tech Support always comes down to – "Daddy is checking and fixing computers.”

All this has changed in SharedBook. Today I can come home and show them all the great memory books we are creating here, which are getting better and more beautiful with each new client. I have even created baby books for all of them, which came out wonderful and were praised by everyone who saw them.

Thanks to SharedBook, today I can finally show my family and friends the “chairs” I am making.

The Customer Is Always Right?

When it comes to Usability, there is always a lingering question for me:

How do you know you aren't just building a self-serving Web site?

Surely, there will always be biases by the information designer or the creative studio as to what is the simplest and most engaging design.  The goal, of course, is to build a site that an average user can successfully navigate.  And an average user would never have the detailed knowledge of a Web site that the information designer has.

A common solution involves the use of "personas." You create a fictionalized customer with a personality and a list of goals.  Then you pretend to be this persona and try to make use of the Web site you are designing.  The idea is that accessing things from a very specific agenda will give you new insight into the usability of your site and allow you to make changes to accommodate your persona.

But does this really work?  I was reading a blog entry by Mark Hurst recently ('On product management, personas, and customer focus').  He points out that nothing can replace actually speaking with your customers (or at least listening to their feedback).  And I have to agree with him.

There are, in my opinion, two critical issues with "persona" based design.  The first one is choosing the right personas.   If you pick based on your intended demographic instead of your actual demographic, then you aren't looking at your average user.  It becomes a matter of your biases being migrated from your design beliefs to your persona beliefs.

And suppose you do successfully get your personas right.  The second issue is your ability as an actor.  The underlying assumption has to be that the persona is someone other than you.  So how well you are able to get into someone else's head and behave as they would becomes the challenge.  And even good actors don't always envision the role the same way.

Bottom line is that creating a good user experience must always start with the customer.  Then it is the role of a good designer to take their feedback and channel it into a good design.  The customer IS always right in Web site design.

Learning the Lingo

As the new kid on the block (I'm just into my third week now), I have already complied an extensive list of new words and phrases.  Some of them are standard publishing terms- like “full bleed” (this refers to an image appearing on a page without any borders).  Some are SharedBook vernacular like "book making space" (a BMS is the storage area on the SharedBook server where we store your work while you are creating your new book).

This all reminds me of my days at Hertz.  When I started work to design their Web site, I had already worked in car rental for over 10 years.  I was so fluent in "rent-a-car speak" that I didn't even know it was a foreign language to the rest of the world.  It took a customer to correct my thinking.

One of our most commonly used terms was 'fleet' (this refers to all the vehicles the company offers).  The Web site had a popular section called "Fleet Guide." This particular customer innocently asked, "What do you mean fleet?  Do you now rent ships or something?"

It turned out every customer we spoke to thereafter informed us that they found the term odd as well.  And the popular section of the site wasn't always hitting its target audience.  So we changed the words, and ever since it has been known as the "Vehicle Guide."  No more confusion.

In a Web 2.0 world, there are new words coined everyday.  At SharedBook, we have new terms for our customers to learn.  The trick will be to limit our lingo as much as possible and, when we need to introduce new terms, make sure that our customers get the benefit of a handy definition for each.

Testing Open API -- Boldly Going Where No One Has Gone Before

I mentioned last week that we face two major challenges in SharedBook QA. The second is testing an "open" application.  Some parts of our application are based on data integration from third party sites in addition to our new open API.

When testing those parts, we are required to test unfamiliar and sometimes non-existent material. In our data integration projects, a variety of content will enter the SharedBook application that could cause miscellaneous conflicts and failures.

SharedBook's open API enables programmers from all over the Web to interface with our software, using almost any kind of programming language or environment. The development task here is to make sure SharedBook remains as tolerable, adjusted and "open" as possible. The QA task is to locate in advance all those places where the system fails to meet these requirements.

Simulating the real world in the lab is always hard, and when the real world is diversified and unpredictable it is twice as difficult. Our way of dealing with this, in the data integration projects, is to try and conduct as many integration tests as possible prior to launch. We try to test the application with many different potential content types, before we open the application to the public.

Nevertheless, as always, the best tests are done when a mass of users begin to use the system.  As soon as “real” users start to encounter “real” problems, our main goal is to locate those problems and fix them as soon as we can, sometimes even before the users notice them.

I'll discuss the ways we do that in my next entry when I talk about our Tech Support.

Expect the Unexpected – SharedBook QA

Quality assurance (QA) is always a challenge. The fact that we are the last barrier between development and the customer is enough to place a huge responsibility on every QA person, in any environment.

QA is like a gate keeper, blocking all the bugs and preventing them from reaching the real world – the end users.

In SharedBook, this responsibility includes two additional major challenges. The first one has already been mentioned in this Blog a few times - many frequent upgrades.

Recently, we have been upgrading our beta site once, and sometimes even twice, a week. It must be very challenging for development to meet such tight schedules, and it is no less difficult for us to test in such an intensive atmosphere.  Especially when we have to verify that a feature which was developed so rapidly will meet the requirements and quality as requested.

Therefore, it is essential to make the most of the short time that we have, by knowing where to be strict and where to skip. For that reason we must be very familiar with the new feature we are testing in order to:

1. Detect the important and/or weak parts which will require "heavy" testing.
2. Realize which parts of the system are affected by the new feature, so we can conduct specific and precise sanity or regression tests.

Good QA is to find as many major bugs as possible and to make sure each one of them receives the proper attention. But it is not enough to find all the right bugs, they must also be fixed.

With such a tight schedule it is impossible to fix all the bugs that are found immediately, therefore we need to decide which bugs to fix.  Such decisions are based on the severity of the bug and the likelihood that a user will actually encounter it.

A wise man once said that the right way to spell "test" is U-S-E-R. It is very difficult to accept this determination as a QA person (even though it might be true), so we continue our tests even after the new feature has launched, in order to be the user who finds the bug “hiding” in the system, and make sure it will be fixed before “real” users find it.

Stay tuned. I hope to talk about the second major challenge next time.