Rapid Test Environments – Supporting automation processes in your guest virtual machines

August 28, 2008

Over the past few days I have come across multiple instances where users have wanted to run processes/commands within guest virtual machines in a software test lab environment. The ability to run these commands/processes provides immense flexibility to test and QA and makes rapid test environment creation easy and with minimal/no manual intervention.

The current limitations and customer problems

First, let me start with a couple of links and instances of what customers have been trying to do to provide you with some context. It just so happens that in these cases customers have been trying to achieve this with VMware Lab Manager and have failed to achieve success.

  • This is an example of a customer posting on the VMware Lab Manager forum requesting for help to run processes in guest VMs. In this case, the customer would like to bring up a VM, install software on the VM, run some tests and bring down the VM once the test is complete. Clearly, the customer is facing some difficulty achieving this.
  • I believe the customer has tried to post this question elsewhere and has limited luck in getting a truly native Lab Manager solution. The SOAP APIs provided do not help him run guest VM operations.
  • Yesterday we met an ISV customer who has a dynamic build and test environment (yeah, right, whats new eh?). The customer would like to built out a set of commands on the fly (run time) and pass parameters (such as the build number, product etc.) as environment variables. Further more the customer would like to use the output of one command (e.g., success/failure etc.) as input to the next command in the sequence. It gets really fancy when the customer wants to leverage a multi-machine scenario and have VMs in the scenario be aware of each other as the bootup happens. E.g., the client waits for the server to come up first before starting the automated web test. Now, try doing all of this without guest VM automation support!

Do these above problems resonate with your scenario and what you are trying to achieve? Clearly, from the above links you will see that VMware Lab Manager has some limitations if you want to run automated processes/commands or operations within each of the guest VMs.

The VMLogix LabManager support for Guest OS automation

Among all the other rich set of functionality, VMLogix LabManager has built in support to run automated operations. As an example, you will find this VMLogix LabManager customer who has immensely benefited from the automation capability. Previously I have written about the difference between automation support for provisioning and guest OS automation — check out what exactly we mean by guest OS automation in VMLogix LabManager. Let me highlight a few of the operations in context of what customers have asked or raised above (see the 2 images below).

In the first image below, I have setup the following set of operations to run in the VM – “Machine Role 1”.

  1. Install software. You can choose from a drop down list the software package to be installed. The software packages can be created and uploaded one time in the ‘software packages’ section of VMLogix LabManager.
  2. Capture the newly created Virtual machine as a template. Note – that once this is done, it will be available as a standard template in the VMLogix LM library for other users. Further – this will be created as a difference disk/template, and the size will be significantly smaller than a fully created template. This helps optimize your storage utilization.
  3. Take a snapshot of the image. You can setup multiple snapshots, here I have shown the first snapshot created.
  4. Set an environment variable in the guest VM.
  5. Run a command on the guest VM.
  6. Assuming the command provides some output logs, you can upload the files to the LM server.
  7. Run a user script – for example some automated test case. You might choose to have another upload command after this to upload the results of your test case. Note, LM anyway maintains the logs of commands executed which you can view at a later point.
  8. Revert back the machine to the ‘clean’ state of step 3.
  9. And continue executing the next set of tests (after changing the environment variable to the next build/product/…)

The above set of steps created are only an example – obviously you can choose to create a sequence that makes sense in your case/scenario choosing operations from the set in the palette. You can obviously create these set of operations for every VM that you have available in a multi-VM configuration if you so choose. You can use the “Sync” operation to keep these multi-machine setups in sync and cohesion and to setup any dependencies in the sequence of operations on each machine.

There are a lot more guest OS operations available to users, you can see the full list in the second image below. Further more, you can create custom operations – i.e., defining a new operation which will then be available via the UI to other LM users.

When I deploy the configuration below, VMLogix LabManager will deploy the VM and execute the set of operations I have specified in sequence. For a user, this is really a single click deployment and all the operations are executed automatically.

Drop me a note or leave a comment here if there is something specific that you are trying to get automated in your test lab – around creating/setting up software stacks on your test VMs. Would love to hear from you.

Sample set of operations setup to be run in sequence on a guest Virtual Machine

Sample set of operations setup to be run in sequence on a guest Virtual Machine

Set of guest OS operations available in VMLogix LabManager. You can extend this by creating custom operations as well.

Set of guest OS operations available in VMLogix LabManager. You can extend this by creating custom operations as well.

[Updated] — Here is a movie of LabManager deploying a configuration with guest operations (watch the “Deploy a Multi-Machine Configuration” movie).
Bookmark and Share


IBM Rational Quality Manager/RTLM Demos

August 26, 2008

IBM is in the process of developing Rational Quality Manager – the next generation test management tool from IBM. You can access the beta program here. Rational Test Lab Management is a new component being introduced with RQM. You can leverage the following resources to view demos and learn more about these products:

VMLogix LabManager has traditionally supported integrations of Rational products, including Build Forge and ClearQuest which are currently available. VMLogix is also in the process of developing integrations with other IBM Rational products being released such as RQM and RTLM above. The power of a combined collaborative and automated solution of Rational RQM/RTLM products with VMLogix LabManager will provide immense benefit to testers in due course!

References


Bookmark and Share


Management tools available for Hyper-V

August 21, 2008

Here are the various management and automation products available to users of the Microsoft Hyper-V environment. If there are any platforms that I am missing, please feel free to drop me a note and alert me to it!

  • Microsoft Hyper-V Microsoft Management Console (MMC) – Basic management tool available to manage Hyper-V hosts which are either local or remote. You can manage and set the Hyper-V server settings using this. With this administration tool, you can also do basic tasks on your Hyper-V host like creating a VM, changing the settings of a VM, getting to the settings of disk, memory, network etc. Once the VM is created, you can start/stop/snapshot a VM and also connect to the VM using a built in virtual machine connector. All in all, a basic tool to get you started with Hyper-V.
  • Microsoft Systems Center Virtual Machine Manager (SCVMM) – This is a more advanced management product that is currently available in beta from Microsoft. The final GA of the product is expected in September 2008. You can read more about the functionality of SCVMM here and here. Some of the key points to note about SCVMM are:
    1. Support for managing Microsoft and VMware VM environments (Full integration with Virtual Center (VC). In fact, SCVMM requires VC in order to integrate with VMware)
    2. P2V, V2V, Manage physical and virtual machines
    3. Features to handle failure scenarios, HA, cluster awareness
    4. Administrative control (identifying consolidation candidates, intelligent placement, quotas, library, templates, scripting etc.)

    All in all, SCVMM is geared well as a management application if you are deploying VMs in production. (Added 9/9) – You can read a full post on SCVMM on the Microsoft technet site.

  • VMLogix LabManager – If you are interested in using Hyper-V in your software development or test lab, then VMLogix LabManager can help with the management. If you are not sure why you need a management product in your dev/test lab, then consider reading this. If you are new to virtual lab automation, then you might want to read this guest post to get some familiarity with VLA. VMLogix will provide you with the following management benefits – self-service IT (hands off management allowing your lab users to gain access to lab resources based on some bounded policies/quotas etc.), centralized lab management across multiple hypervisors (which will control VM sprawl), powerful automation and optimization of lab resources (including storage via linked clones).

Demo of VMLogix LabManager working with Microsoft Hyper-V, with a highlight on the benefits of the solution.

Beyond these, I have come across the following 2 projects on CodePlex: PowerShell management library for Hyper-V and Hyper-V Web Manager.

References:

  1. Windows Server 2008 Hyper-V Key Features
  2. Creating and managing virtual machines with Microsoft Hyper-V
  3. rakeshm’s VM Management blog
  4. What’s new in System Center VMM 2008
  5. System Center Features
  6. Microsoft Hyper-V guide (SearchServerVirtualization.com)


Bookmark and Share


Quantifying the savings on your test cycles from your virtual lab automation deployment

August 18, 2008

We have several posts on this blog discussing various aspects of virtual lab automation including specific product features, benefits, use cases, product integrations, customer experiences and so on. Sometime back I had mentioned that we would also look into a model to quantify the savings (and hence your return on investment) you should expect on your testing/software engineering cycles from deploying a virtual lab automation solution like VMLogix LabManager.

We do believe that virtualization is a must have technology for software testers and that Virtual Lab Automation offers a lot more over a hypervisor (like VMware ESX, Microsoft Hyper-V or Citrix XenServer) – making it a compelling deployment in your software test lab.

For the model, we looked at two important parameters of the software testing cycle — (1) Manual provisioning costs and (2) Bug reproduction costs. The model first helps you determine your current costs for these two parameters and then projects a savings based on experiences with deploying a virtual lab automation solution.

Here is our model:

Quantifying your current costs and projected savings from a VLA deployment

Quantifying your current costs and projected savings from a VLA deployment

Does this help you make a projection of the savings you should expect to see from the deployment of a virtual lab automation solution like VMLogix LabManager? How can VLA help control costs of your software testing efforts?

If you would like to learn more about VMLogix LabManager, go ahead and register on our virtual portal. You will also have access to a free whitepaper “Five Best Practices for Improving Software Development and Testing“.

Alternatively, you can also make a request for a free LabManager 30 day evaluation (VMLogix LabManager supports all leading hypervisors from Citrix, Microsoft and VMware, including the recently announced Microsoft Hyper-V. You can read more about the LabManager system requirements).

– Srihari Palangala


Bookmark and Share


Virtual lab automation to manage your software assets in the lab

August 13, 2008

There is a continuous drive against illegal/pirated software and this article presents recent developments.

Virtual lab automation solutions like VMLogix LabManager can help manage software license compliance requirements in the test lab. Since VLA solutions help centralize lab operations (from across many locations, business units, geographies, divisions, etc.) it is a natural choice to maintain a record of the licenses available in the lab. This is often half the battle — since a lot of organizations find it difficult to maintain a summary of the number of licenses owned. This is especially important in a virtual environment since VMs can be brought “up” and “down” very easily and sans a central VLA solution, a user can be hit with VM sprawl and license management problems very easily.

Like the article mentions, the upside to accurate license management is that organizations will not end up spending money on licenses when it is not really necessary (i.e., when there is a possibility of having too many licenses).

Here is a screenshot from VMLogix LabManager where users can specify licenses owned:

License managment screen in VMLogix LabManager

License managment screen in VMLogix LabManager

License management in LabManager can be configured to be ‘hard’ or ‘soft’. With a hard license management setting, LabManager enforces the license use limit on machine templates and job configurations. This is useful in environments running under strict compliance requirements. The ‘soft’ setting does not enforce the license limit but only tracks licenses used in the lab.

– Srihari Palangala


Bookmark and Share


Customer Experience — Automating the creation of build and test environments works best in VMLogix LabManager

August 10, 2008

We recently caught up with a senior QA engineer working at a world renowned development company with a suite of products that they offer to the software development community. This particular customer has leveraged VMLogix LabManager to improve their internal software engineering test and build process.

Here are excerpts from the conversation.

[Q1: VMLogix] Can you provide us with context around the project that you are working on currently?

[A1: Customer response] We use agile development methodologies here. Our developers have created a large suite of automated unit tests. In addition they have also automated functional and regression test suites. We want to leverage the current infrastructure in terms of the tests created and use it to automate testing on multiple platforms, web application services and databases. So we were looking for a tool that would allow us to run our build and tests on whatever combination of platforms that we require.

[Q2: VMLogix] So, just how complex is your matrix of test platforms?

[A2: Customer response] As a Java development shop, we historically do not want to tell our customers what databases/services/platforms that they have to use. So we wanted to support as much as we possibly could. Over the years, the number of application servers that we are trying to support is roughly 12* (and that is just the number of application servers and not the different versions that we support). Again for databases there are 10* vendors that we support (and again different number of versions). On the browser side, we support multiple versions of IE, Firefox, Safari, Opera. So if you add them all up combinatorially, you end up with hundreds and thousands of different test environment configurations.

[Q3: VMLogix] So what prompted you to look for a VLA solution?

[A3: Customer response] We wanted to maximise the return from our existing hardware and eliminate the need for extra physical machines specifically for interoperability testing.

[Q4: VMLogix] Why did you find VMLogix LabManager as a good fit for the problem that you are trying to solve?

[A4: Customer response] In addition to being able to create and manage virtual machines and control the installation and configuration of software on to them, we specifically wanted the ability to manage multi-machine configurations and we also wanted the ability to take snapshots of these multi machine configurations. And we found that VMLogix LabManager ticked all those boxes, unlike other offerings in the market.

[Q5: VMLogix] Can you provide an overview of how you have deployed VMLogix LabManager?

[A5: Customer response] We are running LabManager within a virtual machine. We have created a plug-in for our build server, that allows us to mark a build in there as a LabManager build. As that build queues up, our plug-in connects with LabManager and gets a configuration started up. The plug-in also installs a remote agent in the configuration (VM) and talks back to our build server. This agent takes the delivery of the build, runs a bunch of automated tests (through LabManager support), passes back any artefacts such as logs or test results back to our build server. Once the build completes, our server communicates with LabManager server to tear down the configuration.

[Q6: VMLogix] So are you using the LabManager API capabilities extensively?

[A6: Customer response] Yes, we are.

[Q7: VMLogix] How large is the deployment, how many users do you have with LabManager?

[A7: Customer response] Currently, for our purposes for automating interoperability testing there really will not be a large number of actual users. It will really be our build servers talking back and forth with LabManager server to run the automated tests and setup/tear down configurations. But, we do have plans for the future once this above setup is fully operational, to use LabManager as a QA/test environment tool to build test environments for testers and I suspect our support engineers will also look at LabManager to help debug and troubleshoot customer issues.

[Q8: VMLogix] What are your broader/longer term plans with the VLA solution? How do you independently see this technology growing in the community?

[A8: Customer response] At the moment LabManager is part of our automated build and testing process, but in the future we see it assisting manual testing, QA engineers with rapid test environments and also for support engineers to create adhoc environments in which to import customer data to recreate bugs or troubleshoot customer issues. So currently we interface with LabManager via the API, but in the future we will rely more on the LabManager web interface.

Does your test and build environment sound similar to what our customer describes above? If you would like to replicate his success and would like a FREE 30 day evaluation of VMLogix LabManager, register and make your request here.

Footnote for A2:

*Web Apps: Tomcat, Geronimo, Weblogic, Resin, Websphere, JBoss, Jetty, JRun, Orion, Oracle OC4J, Pramati, Glassfish
*Databases: DB2, Firebird, HSQL, MaxDB, MySQL, Oracle, Postgres, SQL Server, Sybase ASE, Derby

Note – some of this is work in progress at the time of this writing, not all web apps and databases have been configured for the tests yet.


Bookmark and Share


Best Practices for Improving Software Development and Testing

August 7, 2008

A couple of days ago we put up a new whitepaper on our website around best practices for improving software development and testing. This whitepaper discusses how virtual lab automation is transforming the test and development lab, and provides best practices for automating the test and dev process to maintain or strengthen your competitive advantage by getting better products to market faster.

You can access and read the whitepaper by registering here.

– Srihari Palangala (srihari@vmlogix.com)


Bookmark and Share