II. Introduction to Airtest

1.Preface

By reading this section of the tutorial, you will learn about the following:

• An introduction to
• Beginner's guide: a detailed instruction on Airtest scripts
• How to call Airtest APIs in Python script?
• How to access the platform- related API documentation and call the APIs?
• An introduction to parameters related to image statements

2.Introduction to Airtest

Airtest is a Python-based, cross-platform automated testing framework for UI. It is based on image recognition principles and is applicable to games and apps.

Visit the Airtest source code address on Github for more information. You are more than welcome to help improve the project by submitting PRs or submitting bugs or suggestions on the issues page.

1) How to quickly get started?

First of all, to write Airtest scripts, you need to have some basic knowledge of Python syntax. Although the recording function of AirtestIDE allows you to record scripts that can be replayed effortlessly following the instruction, generally speaking, proficiency in Python syntax can help you write scripts that are more widely applicable and less error prone. If you are not familiar with Python syntax, there are many excellent Python tutorials available online that you can make use of, such as Liao Xuefeng's Python Tutorial for Beginners. For the installation, basic usage, and simple examples of the Airtest project, please refer to the Quick Start Guide section in the Airtest documentation.

3.Introduction to .air Script

1) What is an .air script?

After downloading and unzipping the exclusive IDE for Airtest script - AirtestIDE, click on the New Script button to create a script file with the .air extension by default. The .air extension is exclusive for Airtest scripts.

Now let's open the folder where the newly created script is located. We can see that the .air script file is actually a regular folder, with a same-named .py file in it.

When AirtestIDE runs the script, it is actually running the .py file. This means that although Airtest scripts have their own file extension, they are essentially Python scripts that follow Python syntax. You can freely import other third-party Python libraries as needed.

It should be noted that there must be a .py file with the same name in the .air folder, otherwise it will lead to failure in executing the command "airtest run test.air" in the command line.

2) How to use AirtestIDE to record Airtest scripts?

If you have already read the Quick Start Guide, you should know that a device needs to be connected before recording scripts. This device can be an Android phone, a Windows window, an iOS device, etc. Please refer to the Device Connection section and connect a device to AirtestIDE as needed. After the successful connection to a device, you can start recording the scripts you need with the two functions we introduce in the Airtest Script Recording document: manual recording and automatic recording. Meanwhile, by using Python's conditional statements, loops, and other syntax, you can make your scripts achieve more complex functions and fulfill the needs of automated testing.

3) Airtest script examples

This is a simple script example. In AirtestIDE, it looks like this:

In fact, the script is such a Python script (the Template(xxxx) is automatically rendered as an image in AirtestIDE):

# -*- encoding=utf8 -*-

__author__ = "user"
​

# Initialize the environment

from airtest.core.api import *
auto_setup(__file__)
​
start_app("org.cocos2d.blackjack")
​

# Simulation of clicks

touch(Template(r"tpl1556019871196.png", record_pos=(0.204, -0.153), resolution=(1280, 720)))
sleep(2)
swipe(Template(r"tpl1561952588795.png", record_pos=(-0.067, 0.134), resolution=(1280, 720)), vector=[0.2783, 0.0374])
wait(Template(r"tpl1561952704834.png", record_pos=(-0.186, -0.093), resolution=(1280, 720)))
​
keyevent("BACK")
​

# Some basic logical judgement

if exists(Template(r"tpl1559100640980.png", record_pos=(-0.33, -0.105), resolution=(1920, 1080))):
    text("test")
​

# Assertion

assert_exists(Template(r"tpl1561952631660.png", record_pos=(-0.373, -0.108), resolution=(1280, 720)), "Assert the content exists")
​
stop_app("org.cocos2d.blackjack")

Next, we will explain in detail what each line of code in the example script does.

① Initialize the environment

First of all, just like a regular Python script, we need to write from airtest.core.api import * at the very beginning of the code file to import all the core APIs of Airtest for the subsequent script writing.

The auto_setup API is used for initializing the environment. Here is the API documentation. It accepts 5 parameters, allowing us to set the path of the script, specify the device to run the script on, set the default log path, set the parent path of the script, and set the compression ratio for the snapshots.

• If no parameters are passed to auto_setup (i.e. auto_setup(file)), Airtest will read the various parameters passed in the command line at runtime to initialize the environment.
• When creating a script in AirtestIDE, the default generated code contains the simplest initialization code auto_setup(file), which means that the script file is passed as the script path, and other parameter contents will be read by default from the command line during runtime.
• Please try to call the auto_setup API during script initialization. This can ensure that the environment is initialized as correctly as possible and log files are generated in a correct way. Otherwise, log content will not be generated by default.

There are two forms of running scripts from the command line, whose parameters include device, log, etc.

• Example of running Airtest script from the command line: >airtest run untitled.air --device Android:///device string --log log/. For more information, please refer to this documentation.
• When using AirtestIDE to run scripts, a usable command line will be automatically generated in the Log Viewer window for your reference. "D:\AirtestIDE-path\AirtestIDE" runner "D:\script-path\untitled.air" --device Android://127.0.0.1:5037/5PZTQWQOGES8RWUG --log "C:\Users\username\AppData\Local\Temp\AirtestIDE\scripts\aa8c71acdfa70c3068b862cb42ffb8dc""

② Device connection

• If a device parameter like --device Android:/// is passed in the command line during runtime, the script will automatically connect to the corresponding device during initialization without an additional connection code.

• If the device was not connected during initialization, you can specify the device to run the script on with the auto_setup API(__file__,devices=["Android://127.0.0.1:5037/5PZTQWQOGES8RWUG"]), or connect the device with the connect_device API.

• Airtest supports connecting multiple devices in one script. The set_current API enable the switching between multiple devices, and the device() API can retrieve the currently used device. With this API, you can easily write use case code for interactions between multiple devices, such as adding friends with each other on two phones.

③ Simulation of clicks

As an automated testing framework, Airtest simulates human operations. Common APIs include:

• touch: Click on a specific position to set parameters such as the location and number of clicks, duration of holding.
• swipe: Swipe from one position to another.
• text: Call the input method to enter specified text.
• keyevent: Enter a keystroke, such as the Enter key or Delete key.
• wait: Wait for a specified image element to appear.
• snapshot: Take a snapshot of the current screen.
• And so on.

Please refer to this document for the core APIs. All APIs that appear on this document are cross-platform APIs. Since we have imported all the APIs from airtest.core.api in the first line of the code, these APIs can be directly called in the code, like this:

from airtest.core.api import *
touch((x, y))

Many APIs supports passing in Template images as parameters. During runtime, it will click on the location of the image on the screen:

# Equivalent to touch((x, y)), where (x, y) is the center point of the image.

touch(Template(r"tpl1556019871196.png", record_pos=(0.204, -0.153), resolution=(1280, 720)))

The Template object is an image class. Airtest will first try to find the position in the current screen that matches this image. If it succeeds, it will click on its coordinate. If it fails, it will throw a recognition exception. We will introduce the Template image class in more detail later.

④ Platform-specific APIs

The APIs in airtest.core.api mentioned above are all cross-platform, but the specific platforms supported by each API may vary. For example, the install API only supports Android as listed in the documentation, which means it cannot be used to install applications when a Windows window is connected. For details on the supported platforms of each API, please refer to the platform-specific APIs module in the document directory. In this module, you can find APIs under a platform-based categorization, as well as some exclusive APIs available for a specific platform. For example, under the Android platform:

from airtest.core.api import *

# The touch API on Android platform supports an additional duration parameter to set the length of time for clicking on the screen.

# Refer to the touch method under the Android platform in airtest.core.android.android to know more about its parameter information.

touch((600, 500), duration=1)
​

# An Android-exclusive API called list_app can list all the installed applications.

dev = device() # First, get the current device object. The dev here refers to an Android object.
print(dev.list_app()) # You can then call platform-specific APIs.

⑤ Assertion statement

Assertions are very important in unit testing code, so we recommend using assertion statements in your scripts to determine if the tested application is in the state you expect. Airtest provides two APIs, assert_exists and assert_not_exists, to assert whether an image exists or not on the current screen. Another two statements, assert_equal and assert_not_equal, are also provided to assert whether the two values passed in are equal or not.

4.How to use Airtest in Python scripts?

When creating a new script in AirtestIDE, you can also directly create a .py script file, for which you need to fill in some specific parameters in the pop-up window. Based on our previous knowledge of the auto_setup API, we know that these parameters are to be passed to it and initialize the Airtest running environment. Therefore, the initialization code for a pure .py script can be like this:

from airtest.core.api import *
from airtest.cli.parser import cli_setup
​
if not cli_setup():
 auto_setup(__file__, logdir=True, devices=[
     "Android:///?cap_method=javacap&ori_method=adbori",
 ])
​

# do something

# touch((x, y))

The code above means that if you run this file using python xxx.py without any command line arguments, it will automatically use the auto_setup API to initialize the Airtest-related parameters. In this way, by filling in the specified parameters when writing the Python script, you can directly run the script using the python xx.py command. Meanwhile, the original script running method from command line oid:/// will stay unaffected. As long as the script detects the input of command line arguments (namely the if not cli_setup() conditional statement in the code), it will prioritize using the command line arguments to initialize the Airtest environment. Of course, those who are proficient in using the Airtest APIs can also call them in your Python scripts, just like using any other third-party libraries in Python.

5.Introduction to Parameters for Image Class Templates

In AirtestIDE, statements with snapshots are able to display the corresponding images, making it convenient to know what snapshot is being used in the statement. By clicking the Image/Code Mode Switch in the right-click menu in the editor, you can switch the image code in the current editing window to text code. Then, we will see that a touch(image) statement may be become like this:

touch(Template(r"tpl1556019871196.png", record_pos=(0.204, -0.153), resolution=(1280, 720)))

Template is the image class encapsulated by Airtest. During runtime, the program will read this image and then find the result coordinates that best match this image in the current screen before finally executing touch. A snapshot in AirtestIDE comes with three default parameters. The first parameter is the image path (displayed as a relative path), the second parameter record_pos records the location where the snapshot was taken, and the third parameter

• resolution records the screen resolution when the snapshot was taken. The record_pos allows Airtest to prioritize searching near the location where the recording was made during script playback. If it cannot find a suitable image, it will expand the search area to the entire screen. This improves the speed and accuracy of image searching.
• Resolution records the screen resolution. If the script is replayed on different devices, Airtest will scale the current screen resolution proportionally to facilitate cross-resolution matching of images.
• Although the image path can also be used to initialize a Template class and run the code, we recommend that you fill in the parameters to adapt to devices with different resolutions and to improve image search speed. The images captured by AirtestIDE will automatically generate corresponding parameters. If you are not satisfied with the captured images, you can use the image editing function to further modify the image parameters. In addition to the above three parameters, more parameters are supported, such as modifying the image recognition thresholds, the click position, and grayscale settings during operation. Among them, the threshold value for image matching is closely related to the success rate of image recognition. Please read this document for more details.

1) Global configuration options for image recognition

In the previous section, we have introduced the various parameters of the image class Template. When we modify those parameters, it will only take effect on the corresponding image. For example:

touch(Template(r"tpl1556019871196.png", threshold=0.9)

In this line of code, we set the recognition threshold of an image to 0.9, which means that we only consider the image recognition match successful when the confidence of the recognition result is greater than or equal to 90%. This is a rather strict setting. If you want to extend this setting to all images in the entire script without modifying the code for each image one by one, you can consider modifying the global configuration of Airtest:

from airtest.core.api import *

# airtest.core.api contains a variable named ST, which is a global setting.

ST.THRESHOLD = 0.8
​

# No image threshold is specified, defaulting to 0.8 in ST.THRESHOLD.

touch(Template(r"tpl1532588127987.png", record_pos=(0.779, 0.382), resolution=(407, 264)))

# Manually set the image threshold, subject to the value of 0.6 set by the image.

touch(Template(r"tpl1532588127987.png", record_pos=(0.779, 0.382), resolution=(407, 264), threshold=0.6))

There are more customizable global configuration options available. Please refer to the document Global Configuration Scripts for more information.

6.Summary

We hope that you can have a better understanding of Airtest scripts after reading this tutorial. It not only allows you to generate recording scripts through a few clicks on AirtestIDE, but also can be combined with Python and programming skills to write code that can achieve more complex requirements. Therefore, to master Airtest, like other Python third-party libraries, you need to read the Airtest API documentation carefully. We also provide its source code on Github which is clear and easy to understand. Welcome to read the source code and communicate with us.