We recently caught up with a senior QA engineer working at a world renowned development company with a suite of products that they offer to the software development community. This particular customer has leveraged VMLogix LabManager to improve their internal software engineering test and build process.
Here are excerpts from the conversation.
[Q1: VMLogix] Can you provide us with context around the project that you are working on currently?
[A1: Customer response] We use agile development methodologies here. Our developers have created a large suite of automated unit tests. In addition they have also automated functional and regression test suites. We want to leverage the current infrastructure in terms of the tests created and use it to automate testing on multiple platforms, web application services and databases. So we were looking for a tool that would allow us to run our build and tests on whatever combination of platforms that we require.
[Q2: VMLogix] So, just how complex is your matrix of test platforms?
[A2: Customer response] As a Java development shop, we historically do not want to tell our customers what databases/services/platforms that they have to use. So we wanted to support as much as we possibly could. Over the years, the number of application servers that we are trying to support is roughly 12* (and that is just the number of application servers and not the different versions that we support). Again for databases there are 10* vendors that we support (and again different number of versions). On the browser side, we support multiple versions of IE, Firefox, Safari, Opera. So if you add them all up combinatorially, you end up with hundreds and thousands of different test environment configurations.
[Q3: VMLogix] So what prompted you to look for a VLA solution?
[A3: Customer response] We wanted to maximise the return from our existing hardware and eliminate the need for extra physical machines specifically for interoperability testing.
[Q4: VMLogix] Why did you find VMLogix LabManager as a good fit for the problem that you are trying to solve?
[A4: Customer response] In addition to being able to create and manage virtual machines and control the installation and configuration of software on to them, we specifically wanted the ability to manage multi-machine configurations and we also wanted the ability to take snapshots of these multi machine configurations. And we found that VMLogix LabManager ticked all those boxes, unlike other offerings in the market.
[Q5: VMLogix] Can you provide an overview of how you have deployed VMLogix LabManager?
[A5: Customer response] We are running LabManager within a virtual machine. We have created a plug-in for our build server, that allows us to mark a build in there as a LabManager build. As that build queues up, our plug-in connects with LabManager and gets a configuration started up. The plug-in also installs a remote agent in the configuration (VM) and talks back to our build server. This agent takes the delivery of the build, runs a bunch of automated tests (through LabManager support), passes back any artefacts such as logs or test results back to our build server. Once the build completes, our server communicates with LabManager server to tear down the configuration.
[Q6: VMLogix] So are you using the LabManager API capabilities extensively?
[A6: Customer response] Yes, we are.
[Q7: VMLogix] How large is the deployment, how many users do you have with LabManager?
[A7: Customer response] Currently, for our purposes for automating interoperability testing there really will not be a large number of actual users. It will really be our build servers talking back and forth with LabManager server to run the automated tests and setup/tear down configurations. But, we do have plans for the future once this above setup is fully operational, to use LabManager as a QA/test environment tool to build test environments for testers and I suspect our support engineers will also look at LabManager to help debug and troubleshoot customer issues.
[Q8: VMLogix] What are your broader/longer term plans with the VLA solution? How do you independently see this technology growing in the community?
[A8: Customer response] At the moment LabManager is part of our automated build and testing process, but in the future we see it assisting manual testing, QA engineers with rapid test environments and also for support engineers to create adhoc environments in which to import customer data to recreate bugs or troubleshoot customer issues. So currently we interface with LabManager via the API, but in the future we will rely more on the LabManager web interface.
Does your test and build environment sound similar to what our customer describes above? If you would like to replicate his success and would like a FREE 30 day evaluation of VMLogix LabManager, register and make your request here.
Footnote for A2:
*Web Apps: Tomcat, Geronimo, Weblogic, Resin, Websphere, JBoss, Jetty, JRun, Orion, Oracle OC4J, Pramati, Glassfish
*Databases: DB2, Firebird, HSQL, MaxDB, MySQL, Oracle, Postgres, SQL Server, Sybase ASE, Derby
Note – some of this is work in progress at the time of this writing, not all web apps and databases have been configured for the tests yet.