Strategic Testing – Shockoe and SOASTA TouchTest©

Strategic Testing – Shockoe and SOASTA TouchTest©

At Shockoe, application testing is a strategic component in our business offerings that provides our clients with a competitive advantage.

Traditionally, testing is a tactically intensive process.  By its hands-on nature, human resources required to perform tests consume budgets and starve strategic resources needed to define, conduct, and analyze quality testing objectives.

Today’s mobile apps require exhaustive levels of effort and resources to conduct comprehensive, confident, bullet-proof testing.  The sheer number of device types, makes, models, operating system versions, original equipment manufacturer modifications, and carrier alterations strangle resources required for testing, drive-up development costs, delay deployment, and often limits tests to key app areas and select devices.

So, how is Shockoe able to focus on strategic solutions without being bogged down by the nearly endless tactical resources required to ensure proper app testing? … By using SOASTA TouchTest© to automate the testing process.

So, what is TouchTest and what strategic advantages does it deliver?

Shockoe employs TouchTest to execute the majority of testing tasks traditionally performed by humans.  As such, our efforts can be focused on developing strategic test objectives for real-world scenarios, using real-world data, on actual devices.

Fundamentally, TouchTest records user interactions with an app.  These recordings can be played back on a single device or simultaneously on multiple devices.  Playbacks can be incorporated into continuous integration environments and automatically performed as part of an app build process.  During play back, app and device information is collected for analysis. Using TouchTest to carry-out test objectives enables Shockoe to expand test planning, conduct tests on more devices, shorten product cycles and increases reliability as human error is removed from the hands on, tactical testing processes.

Strategic test objective planning is the first step in Shockoe’s process.  Complex tests, such as full regression, are broken down into many smaller unit tests.  Identifying edge cases where the unit test is pushed to extremes by data values, user interaction, and system demands, are crucial for solid testing.

After test objectives are defined, a human Shockoe tester conducts each unit test on a TouchTest enabled app and device.  TouchTest records each gesture, event, and invoked user interface element as an “app action.”

App actions provide validations, condition matching waits, and access to property sets.  Additional elements can be inserted before and after app actions.  These elements include checkpoints, time delays, controls, waits, composition launchers, and comments.

If-then-else and switch conditional elements can also be added to control the flow of a test.  These conditional statements can evaluate properties associated with app actions, device, and data provided as part of the test environment.  Results from embedded or linked scripts can also be used by conditional statements.  The ability to verify app action behavior and change the course of testing based on conditions is a powerful, strategic advantage of TouchTest.

Determining what input data will yield the desired results is another strategic aspect in defining solid test objectives.  Should the test pass?  Should the test fail?  What data best exposes edge tests?  TouchTest allows “seed data” from databases and/or .csv files to be injected into the test playback process at the app action level.  While TouchTest allows for manually entered data during tests, seed data provides fully automated testing and ensures exact data use.

App actions and elements can produce output for recording test execution detail.  Screen shots, app action object properties such class name and text, as well as device information can be saved for post test analysis.  Strategic data capturing greatly reduces time required for analyzing results, decreases the time to identify faulty code, and can be used to further refine tests.

App actions and any inserted elements comprise a test “clip.”  Typically, a clip defines a single unit test case, for example, actions required to perform a login.

Clips are assigned to a “track” in a “composition.”  As its name implies, a composition allows many parts to become a whole. Playing back a composition comprised in many clips provides the ability to perform any test scenario – from unit to full regression testing.  A composition track is assigned to a physical device.  Multiple tracks can be added to a single composition allowing a single test composition playback to be performed simultaneously on multiple devices

During playback, TouchTest records key metrics from both the app and device.  These statistics are accessed in the TouchTest Dashboard.  The ability to maintain test results and drill down into details enables Shockoe to devote resources towards strategic analysis of defects, determining defect cause and effect, and efficiently identifying and making corrective actions.

Shockoe’s use of TouchTest dramatically reduces the hands-on tactical resources required for testing, allowing efforts to be directed towards strategic test development, deployment, and analysis.

TouchTest directly benefits our clients by shortening product cycles as tests and corrective actions are implemented through out the development cycle, ensuring device platform reliability as tests are performed simultaneously on multiple devices, improving app performance as emphasis is focused on analysis, and cost reductions as more tests can be conducted with fewer human resources and in less time.

Contact us to learn how TouchTest can benefit your existing or future app.

For additional information on SOASTA TouchTest, visit here.

Can you QA test this? …

Can you QA test this? …

… is the first question an agile developer should ask of them self.

Agile development is a continuous, flexible feedback loop driven by collaboration, adaptive planning, dynamic development, early delivery, and constant improvement. This continuous integration approach to development has a dark side – the potential to continuously integrate defects.

Successful agile development requires testing as a strategic component of each project: Agile development requires agile testing.  As code architects and engineers, developers are best qualified to identify areas of potential risks. Developers are critical in creating and implementing an efficient, effective testing strategy.

Developers are the first line of defense in detecting and eradicating defects. Applying best practices during coding reduces the number of introduced errors. Catching and correcting defects before they are introduced into a distributed app is far more productive than pinpointing and fixing a multitude of problems days or hours before delivery deadlines.

Testing on developer environment simulators is the first means of testing an app, however, it does not expose all defects. Simulators do not represent the actual device or user events encountered in real device use. Actual devices are the only means to test real user experience – have you ever seen an end-user run their app on a simulator?

After passing initial testing on the development simulator, the app is ready to be built on the continuous integration (CI) environment. Builds deployed to devices from the developer environment may not exhibit defects resulting from different CI configurations. While CI performs testing on various levels, developers should verify and validate CI builds prior to passing apps to third persons for manual testing or automated QA frameworks outside the CI.

Developers using cross-platform solutions, such as those used by Shockoe, can often decrease development time, testing complexity, and costs. Today’s multitude of device chipsets, memory, display, sensors, OS versions, OEM and carrier modifications, and third party software dependencies can make “test often” overwhelming, expensive, and dramatically reduces the time a developer can spend developing.  Shockoe employs tools to streamline the seemingly endless possibilities of testing required to assure stable products and heightened user experience.

Automated testing on actual devices can be performed in-house and by cloud-based services. These automated testing frameworks increase the number of tests that can be run in a time frame, reduce the time it takes to expose defects, and shortens overall time to market. Automated testing frameworks do not make errors. Developers should witness the first few automated passes, as they are most familiar with the product’s performance expectations and nuances.

Human testing comes with long time cycles, high costs, and is prone to errors. While automated testing solves these human detriments, they do not accurately test for design issues and overall user interaction experience. Whether testing in development simulators, performing manual tests on CI builds, or watching the first few passes of an automated test, only a human can test for human experience factors.

Developers produce the code that is the app. They are integral to the testing processes at all levels of production. Before a developer asks, “Can someone QA test this?” they should employ development best practices, testing techniques, and tools to ensure those “QA-ing” find few, if any defects.

Appcelerator Titanium iOS Debugging with XCode

Appcelerator Titanium iOS Debugging with XCode

Earlier this week, I was debugging and I was reminded of the sheer power of the XCode developer tools, even in the context of a not-quite-native application like a titanium application.

The Problem

Andrew was working on an application that will load in a large number of images and PDFs from a remote server and display them to the user, in-app. However, when we got to the point that we would be displaying a certain one of our images, we saw this in the Titanium console:

Sorry, what? It seems like something odd is happening at a native level, and Titanium is getting too confused to return a sensible error message. Well, guess it’s time to open a support ticket with Appcelerator and wait for them to figure out what the issue could be, right?

Wrong. One of my favorite things about Appcelerator Titanium is its open-source nature. What we can do from here, is open up the native project generated by Titanium and debug it with the normal native debugging tools. When you build a Titanium application for iOS, a (pretty much) normal XCode project is generated from your project, compiled, and run on whatever test device you have selected. In situations like this, we can take that project and manually re-build it in XCode for debugging purposes.

Opening your project in XCode

To open your project in XCode, first run

in your project’s directory. This will ensure you have a native project generated for your Titanium project. From here, all you need to do is open XCode, and open up the XCode project in the build/iphone folder.

Path to a Titanium project's compiled XCode Project

Setting Native Breakpoints

Now that we’ve got the project in XCode, we need to set up a native breakpoint so that we can see what the issue is with the Objective-C code that Titanium is executing on our behalf. Fortunately, the message that Titanium printed out gave us a selector name:

. Let’s go ahead and set up a symbolic breakpoint for that selector:

Add a breakpoint in XCode by clicking the second icon from the right in the side bar, and then clicking the plus in the bottom right

Add the symbol from the error as the symbol name

Enter the XCode debugger

Now that we’ve got our breakpoints set up, we can run the project in XCode, and execution will stop when our breakpoint is hit in the Titanium SDK code.

The debugger will automatically pause execution when you hit a breakpoint.

Let’s go ahead and step over a few commands and see if we can figure out exactly what’s going wrong.

Step Over

Huh, it looks like we’re having some issue turning our Titanium file into a UIImage that we can apply to the native UIImageView. Let’s use the variable inspector to figure out exactly why we’re failing to convert this into an iOS image.

XCode variable inspection

Well, one look at the MIME type is enough to see exactly what’s wrong. Our file isn’t an image! Even though this didn’t tell us exactly where the issue was, it was enough to direct our debugging (we eventually figured out that we were accidentally saving a PDF file as an image – oops!). Issues like this are why I’m very quick to reach for XCode when I see a native iOS issue – it makes it much easier to figure out what parts of your code might be incorrect when you can easily trace through Appcelerator’s code!

API Description Language for the Enterprise

API Description Language for the Enterprise

REST, now estimated to be used by 90% of new APIs, is great for groups within a corporation that can engineer their API as quick as lean startups. However, REST doesn’t fully meet the needs for enterprise businesses. With their slow-moving review and approval processes, internal politics, and sometimes a large user base, enterprise businesses require an API that can be simple to manage, quick design process and also be able to use standardization across potentially hundreds of other APIs across multiple line of business.

It’s been more than a couple years now since we started to look for API Description Languages that can fit small to large companies. Anything that can help the communication between Back-End and Front-End developers to be fast, easy and standardized, it’s been something that we always strive for. Swagger and RAML, both are API Description Languages that meets those needs.

It has been suggested that RAML may be the perfect solution for designing the next generation of enterprise level APIs, but as you might know enterprises still using SOAP based APIs and, of course, WSDLs are still the king.

With Swagger or RAML, at the end of the day, you are trying to describe an API by writing JSON, and since it’s JSON, there are many tools out there that will help you generate these documents via nice editors (GUIs).


Rapid Titanium WebView debugging with Chrome Developer Tools

Rapid Titanium WebView debugging with Chrome Developer Tools

Any Web Developer worth their salt is already familiar with the awesome Chrome Developer Tools package that ships with Google’s Chrome Browser.  What you might not know is that Android 4.4 introduced a powerful new native debugging tool for native Android/HTML5 hybrid apps that allows you to attach Chrome Developer Tools to a Web View running on a physical android device.  As part of our continuing mission to help you tame the Mobile Monster, we’ve taken the liberty of wrapping the code to enable this functionality in a Titanium-ready mobile module.  To make use of this new functionality in Titanium, you will need :

  • A laptop with the Google Chrome Browser
  • A phone running Android 4.4+ (and a charging cable)
  • An app built with Titanium SDK 3.4.1.GA +

Getting Started

First, download the pre-built module below.

[ No module needed on Android 4.4.x+ ] (update)

To make use of the module within your Titanium project, you just need to un-zip the module into your modules folder, and add it to your modules tag in your tiapp.xml.  We recommend only including this module for the development deploy type, since this is purely a debug tool.

Now, just build your app to an Android 4.4+ device, and you will be able to attach chrome developer tools to any web view within your app.

Attaching Chrome Developer Tools

Once you have your app on your test device, and have your test device connected to your development laptop, open Google Chrome and type chrome://inspect into the address bar.  This should open you up in the devices tab.


The Chrome developer tools device perspective

Then just click inspect.  A normal Chrome developer tools window will open, allowing you to inspect your element as normal.  In addition, you can click the new icon in the top-right of the screen to get a live preview of your WebView directly within the inspector window.

The Chrome Developer Tools inspector with on-device preview


And that’s it. You now have a full-featured Chrome Developer Tools window attached to your Titanium app’s WebViews, allowing you to use all of the same web development tools that you are familiar with from normal HTML/CSS/JavaScript apps to develop Titanium/Mobile Web hybrid apps.

Page 1 of 2012345...1020...Last »