For several years now I've had the pleasure and frustration of talking about and demonstrating what App-DNA™ AppTitude™ does for application compatibility and application lifecycle management. Pleasure because there comes a moment when people see AppTitude perform application testing and application remediation and just go "Wow" and "Where were you guys when I moved to XP". That’s the moment when people "get it". The frustration comes from being pioneers in the technique we use to assist with application compatibility for Windows 7, IE8, App-V and XenApp projects, and that means a lot of people have not yet heard about the AppTitude platform approach or don't yet realise that there is another way to go about things.
So, I wanted to take the opportunity to talk a bit about this technique and why it so important for organisations who manage large volumes of apps (and for me large is really 150+ all the way up to and beyond 5,000).
The old way
Traditional wisdom says that to understand an application and detect any defects you need to install it, run it and plug it into tools and systems that will monitor various aspects of how the app behaves and performs - analogous to the telemetry for racing cars. And, make no mistake, you can determine a great deal about an application in this way. The problem is it just does not scale to large volumes of apps and becomes very complex and long winded when you need to test for multiple platforms or things like dependencies and suitability for virtualisation.
So what ends up happening is that only a subset of the apps get thoroughly tested, the rest get light testing, if any. This makes for an inconsistent and risky project - nobody wants to be responsible for that! Some of the key shortcomings of the manual testing approach are:
1. Not all functionality is likely to be used during testing and that leaves aspects untested, allowing for latent defects
2. The install phase is often excluded or not adequately monitored (especially with agent based options which monitor 'live' environments)
3. Time consuming - this can make for very long preparation and testing cycles, running into months or years
4. Requires functional and technical knowledge
5. Context specific - you test for an objective, once you're done you have to re-test for new objectives
The new way
Our approach takes the application installer, extracts all of the files and performs a deep collection of a huge array of data relating to both binary and text files. We touch every single file in the application and can thus be highly accurate. Another key aspect of this approach is that it de-couples the analysis from the processing of the application. When we import the app we are just learning about it, not 'testing' it - our import collects tens of thousands of data points (for example: API calls, file imports and exports, configuration file contents and much, much more).
Once all of the application data or 'DNA' is in the database we then run analysis. Running the analysis in a separate phase means we can adapt to future changes without revisiting the apps. Collecting all of the detailed application data in the database means we also get to see how the apps relate to each other (not just the internal state of each app). Furthermore, we have the OS reference as part of our 'dataverse' or 'portfolio genome' and this means we have a unique ability to model the impact of a huge variety of challenges including application compatibility, dependencies, quality and suitability for virtualization.
The benefits of this approach are now well established:
1. Speed - there are many speed gains, we can process an application in just minutes, thousands in just days
2. Accuracy - touching all of the files exposes more defects than manual testing and our view of the entire portfolio and OS extends the traditional view of the issue space.
3. Breadth - report on Desktop and server OS compatibility, Virtualisation, Citrix and Terminal Services, 64bit, dependencies and interoperability in one process. This massively reduces the technical and application knowledge required in the testing processes.
4. Depth - Installation, files, runtimes, Operating System components as well as external data. Its all 'in the box' with AppTitude
5. Stability - projects executing on time without issues makes for a stable environment where potential defects are identified and resolved before change is released to the user environment
6. Reduced time, reduced cost - this degree of optimization translates directly into savings. This has been demonstrated time and again on successful projects
Silver bullet?
Wouldn't it be nice! As powerful as this approach is the more technically minded of you will have spotted various scenarios under which static analysis simply cannot identify issues (runtime activity not published, late binding instantiation, hard coded activity). We recognise that for certain environments (e.g. multi user terminal services / XenApp hosted) this can be vital information and so we extended our import process to allow for a run cycle where we collect the runtime telemetry to compete the picture as needed. The decision to go this route is generally determined by the nature of the information required, the volume of applications and the available time to execute.
As the AppTitude platform has matured it has become a complete solution for high speed, high volume projects as well as optimized day-to-day Business As Usual application management. The automation efficiencies are driven by our static analysis which feeds and interrogates our rich database formed of Application "DNA".
Questions or comments about the AppTitude approach? Please post them in the comments or on our Twitter@appdna
af260451-ecd5-4869-b671-64e4473589be|8|4.4