Testing in the software lab – bring the field in!

Dennis Stevenson has written about Getting Software Development out of the lab.

As a software developer in a previous life, I found the article very interesting and wanted to add some points that came to mind as I read his piece (Btw – the first comment also has a very interesting story to narrate). Dennis’ point is that software tested in the lab is often “very protected” (my interpretation) – and that the real user testing/acceptance testing does not happen. Some thoughts:

  • Beta and GA:: ISVs often release software in beta prior to having a GA version available. The beta is meant to alert the user that there are likely edge cases/kinks in the software that need to be addressed. ISVs normally listen to beta customers very carefully for things to iron out workflow issues, specific configurations, installation problems etc. There are some companies that have products in Beta forever!
    • The field engineering team along with early product adopters can help transform a beta product into a GA (and paying customers). However you need a product manager to build something from scratch/evolve a product roadmap that will generate revenue.
  • Domain Experts and their role in Requirements Specification:: IT service companies, i.e., those that provide software solutions to enterprise customers have business analysts (BA) and domain experts to help those building/integrating software products. The BAs are expected to understand user workflows and expectations and provide this input to the engineering team. Typically BAs have significant work experience in a specific domain (financial/retail/…), so they understand the business in a more holistic manner and not necessarily just some individual workflows.
    • Domain experts play some of the roles of a product manager in the services world.

With technologies like virtual lab automation and products like VMLogix LabManager you can get to a lot more user/field cases than was traditionally possible in test labs. I have written in the past that virtual lab automation may be able to help you make a better decision on when to ship software. The test matrix coverage is significantly better — and you can re-produce a host of field scenarios prior to the beta release. But these are traditional points related to ‘software quality’ (i.e., as a tester you know what needs to be tested).

In Dennis’ context – Virtual lab automation solutions provide an additional benefit – you can let the field team get to the lab via a web browser! They can create and share scenarios, use test environments, templates, use the software, report bugs/issues etc. and be an integral part of the lab. You can get the software out of the lab – through virtual lab automation solutions!

Do you allow the field in your lab? If so, when do you normally allow the field into the lab? And how do you do this?

– Srihari Palangala

Bookmark and Share


2 Responses to Testing in the software lab – bring the field in!

  1. Srihari,

    Interesting notion of “bringing the field into the lab”. When I wrote my original blog post, I was mostly thinking about getting the lab out into the field. Recognizing that the lab isn’t everything, and is in fact quite removed from the field.

    Not sure about the product here… it’s something I’d have to better understand before I could really comment.


  2. vmlogix says:

    Hi Dennis,

    Thank you for your comment. Personally, I am convinced that server virtualization and associated lab management products will dramatically change the way software engineering cycles work. Couple of relevant capabilities that virtual lab automation enables in this context is – “self serve IT” and the ability to “collaborate between teams”. This functionality will bring the field into the fold and help improve software quality and accelerate time to market.

    – Srihari Palangala

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: