Customer Experience — Automating the creation of build and test environments works best in VMLogix LabManager

We recently caught up with a senior QA engineer working at a world renowned development company with a suite of products that they offer to the software development community. This particular customer has leveraged VMLogix LabManager to improve their internal software engineering test and build process.

Here are excerpts from the conversation.

[Q1: VMLogix] Can you provide us with context around the project that you are working on currently?

[A1: Customer response] We use agile development methodologies here. Our developers have created a large suite of automated unit tests. In addition they have also automated functional and regression test suites. We want to leverage the current infrastructure in terms of the tests created and use it to automate testing on multiple platforms, web application services and databases. So we were looking for a tool that would allow us to run our build and tests on whatever combination of platforms that we require.

[Q2: VMLogix] So, just how complex is your matrix of test platforms?

[A2: Customer response] As a Java development shop, we historically do not want to tell our customers what databases/services/platforms that they have to use. So we wanted to support as much as we possibly could. Over the years, the number of application servers that we are trying to support is roughly 12* (and that is just the number of application servers and not the different versions that we support). Again for databases there are 10* vendors that we support (and again different number of versions). On the browser side, we support multiple versions of IE, Firefox, Safari, Opera. So if you add them all up combinatorially, you end up with hundreds and thousands of different test environment configurations.

[Q3: VMLogix] So what prompted you to look for a VLA solution?

[A3: Customer response] We wanted to maximise the return from our existing hardware and eliminate the need for extra physical machines specifically for interoperability testing.

[Q4: VMLogix] Why did you find VMLogix LabManager as a good fit for the problem that you are trying to solve?

[A4: Customer response] In addition to being able to create and manage virtual machines and control the installation and configuration of software on to them, we specifically wanted the ability to manage multi-machine configurations and we also wanted the ability to take snapshots of these multi machine configurations. And we found that VMLogix LabManager ticked all those boxes, unlike other offerings in the market.

[Q5: VMLogix] Can you provide an overview of how you have deployed VMLogix LabManager?

[A5: Customer response] We are running LabManager within a virtual machine. We have created a plug-in for our build server, that allows us to mark a build in there as a LabManager build. As that build queues up, our plug-in connects with LabManager and gets a configuration started up. The plug-in also installs a remote agent in the configuration (VM) and talks back to our build server. This agent takes the delivery of the build, runs a bunch of automated tests (through LabManager support), passes back any artefacts such as logs or test results back to our build server. Once the build completes, our server communicates with LabManager server to tear down the configuration.

[Q6: VMLogix] So are you using the LabManager API capabilities extensively?

[A6: Customer response] Yes, we are.

[Q7: VMLogix] How large is the deployment, how many users do you have with LabManager?

[A7: Customer response] Currently, for our purposes for automating interoperability testing there really will not be a large number of actual users. It will really be our build servers talking back and forth with LabManager server to run the automated tests and setup/tear down configurations. But, we do have plans for the future once this above setup is fully operational, to use LabManager as a QA/test environment tool to build test environments for testers and I suspect our support engineers will also look at LabManager to help debug and troubleshoot customer issues.

[Q8: VMLogix] What are your broader/longer term plans with the VLA solution? How do you independently see this technology growing in the community?

[A8: Customer response] At the moment LabManager is part of our automated build and testing process, but in the future we see it assisting manual testing, QA engineers with rapid test environments and also for support engineers to create adhoc environments in which to import customer data to recreate bugs or troubleshoot customer issues. So currently we interface with LabManager via the API, but in the future we will rely more on the LabManager web interface.

Does your test and build environment sound similar to what our customer describes above? If you would like to replicate his success and would like a FREE 30 day evaluation of VMLogix LabManager, register and make your request here.

Footnote for A2:

*Web Apps: Tomcat, Geronimo, Weblogic, Resin, Websphere, JBoss, Jetty, JRun, Orion, Oracle OC4J, Pramati, Glassfish
*Databases: DB2, Firebird, HSQL, MaxDB, MySQL, Oracle, Postgres, SQL Server, Sybase ASE, Derby

Note – some of this is work in progress at the time of this writing, not all web apps and databases have been configured for the tests yet.


Bookmark and Share

Advertisements

3 Responses to Customer Experience — Automating the creation of build and test environments works best in VMLogix LabManager

  1. […] functionality, VMLogix LabManager has built in support to run automated operations. As an example, you will find this VMLogix LabManager customer who has immensely benefited from the automation capability. Previously I have written about the […]

  2. Peter says:

    I managed a team that integrated Lag Manager into an automated build and test system. See http://peterwilliams97.blogspot.com/2008/10/test-automation-for-complex-systems.html .

    Lab Manager allowed us to fully enqueue the build+test+fix cycle: provision VMs, install software on them, run tests, check results saved failed VMs to disk for later examination. The build+test +fix duty cycle was close to 100%, INDEPENDENT OF THE NUMBER OF DEVELOPERS, COMPLEXITY OF THE CODE UNDER TEST AND THE NUMBER OF RELEASES PER YEAR.

    This was a vast improvement for our development organization of 300 developers working on 3M lines of C/C++ code and making 50-100 product releases/year. Running a development organization. Anyone who has worked in a development organization of this size knows that the amount of time spent on communicating between teams and developers and the hand-offs of code between teams dominates schedules. Attaining quality releases requires thorough testing at each hand-off which in turn required synchronizing many people: the developer submiting code, the build team building it, the testers testing it and report bugs, and the developers fixing the bugs. This high level of synchronization reduces flow of work through the system dramatically, and the fraction of time spent on syncrhonization increases with the size of the development organization

    Lab Manager allowed us to fully enqueue the hand-offs between teams so the synchonization overhead almost disappeared. Code check-ins triggered the build+install+test+check process and after that the code change was either accepted or rejected with a reason and a failed VM for the developer to debug when he/she had time.

  3. […] there are umpteen uses of virtualization in testing. Regression tests, performance testing, testing under varied environment conditions/permutations/combinations, integration testing etc. are all cases where virtualization is useful. You will find this question […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: