UI Automation Gotchas (part I)

I have recently started to implement user automation in Rex Lists, as a prove of concept test for the applications I am currently working on. I won't try to explain UI automation in this article, since there are are some good sources available like Alex Vollmer article or session 306 of WWDC10. Instead, I will focus on some of the things that I stumbled upon.

What is UI Automation

User Interface Automation provides developers with the ability to create test cases for the user interface that can be re-executed with almost no human intervention in order to have repeatable behavior.

As Graham Lee (@iamleeg) pointed out in the recent episode 18 of iDeveloper Live about TDD, this will help your Q&A, but cannot be considered as part of TDD, because they take a long time to be performed and thus cannot be part of the normal building process.

The things you want to know

Solve your accessibility first

Since UI Automation accesses the items using their accessibility labels, it is a good idea to have your application accessible first. You probably want to go there anyhow for the best customer experience, and it will make your live much easier too.

Use the proper language

I am not referring here to the programming language (or swearing for what is worth), but to the language that is displayed when running the program, either in the simulator or in the device. UI Automation uses the accessibility information, and if you have done your work well, that information is internationalized and displayed or spoken in the current localization.

If you want to simulate a tap on the information button that is at the left of the navigation bar, you will include some javascript code similar to:

var window = UIATarget.localTarget().frontMostApp().mainWindow();
window.navigationBar().buttons()["Information"].tap()

Unfortunately, if you run the application in Spanish (for example), the accessibility label of that item will be "Información" instead of "Information" and the script will not work.

The easiest way to go is to have all the names of your UI automation script written on the default language, and ensure that all the tests are run with that language set. This means that you have to check your internationalization preferences before starting the tests.

Find the right elements

If you are not used to write UI Automation scripts, and don't know the name of the elements in any of the views of your application you can ask JavaScript to show the name of the elements.

var window = UIATarget.localTarget().frontMostApp().mainWindow();
window.logElementTree();

When unsure about the hierarchical relation of the elements, use this to get this information displayed in the output of the test as shown in the following image:

The numbers that are shown at the beginning of each line are the level in the hierarchy. In this case, the application contains a window, that contains a tableview with a table group that has its section title.

Obtain the labels for the system items

Some items, like the add button that is present in many navigation bars and toolbars, are created using the defaults provided by the system.

If you want to see the name (i.e., accessibility label) of an item that is defined by UIKit, or one of yours for what is worth, you can turn on the accessibility inspector of the simulator. In order to do so, go to the Preferences application of the simulator, tap on "General" and then on "Accessibility".

But don't forget to toggle the accessibility inspector of before running your script. Otherwise, the application will behave as if accessibility were turned on in the phone, and controls are then selected with a tap and used with a double tap.

Conclusions

I hope this information is useful to you when implementing your UI Automation scripts. I will be glad to have any comments or corrections. Share your experiences here too!

In my next entry I will explain other aspects of UI automation that are more related to its integration with Xcode. Come back and read it!