Skip to content

Annotation

This document is a supplementary document of Airtest framework and AirtestIDE editor. Please refer to Airtest project documentation for more detailed API information during script writing.

5.1 Airtest introduction

Airtest is a cross-platform UI automated test framework, based on the principle of image recognition, suitable for games and apps.

Visit Airtest source address on Github for more information. You are also welcome to help improve the project, submit a PR, or submit bugs or suggestions in the issues page.

The project documentation of Airtest can be accessed here.

5.1.1 How to get started quickly

For installation, basic usage and simple examples, please check the Quick Start section of the Airtest documentation.

At the same time, we also provide an tutorial Airtest introduction and script introduction for everyone to learn.

5.1.2 Airtest API

The module airtest.core.api in Airtest provides a series of cross-platform API for calling, including touch operation touch, drag operation swipe, input text operationtext and other operations. Please refer to the Cross-platform API documentation to see the specific API for call parameters and return values.

In the cross-platform API documentation, the interface parameters viewed are cross-platform common parameters. If specific to a platform, you can consult the API documentation of the corresponding platform for more parameter information.

Take the touch interface as an example. Assuming that you need to click on the Android platform, you can write: 

# Import all API from airtest.core.api, be sure to keep this line
from airtest.core.api import *
# The touch interface on the Android platform supports additional parameter duration to control the duration of the screen tap.
# Check the touch method included in airtest.core.android.android for more parameter information
touch((600, 500), duration=1)

If you want to call a public function encapsulated in another .air script in the current .air script, you can do this: 

from airtest.core.api import using
using("common.air")

from common import common_function

common_function()

For more information, please refer to the Airtest framework complete documentation. In this document, we only provide the documentation of some interfaces.

5.1.3 Airtest scripting in AirtestIDE

In Airtest, you need to write a lot of screenshot statements, like this: 

    touch(Template("image_of_a_button.png"))

The Template (" xx.png ") is an image object. If you need to manually take screenshots and write code, the workload will be relatively large. So we recommend that you use the special AirtestIDE for the development of screenshot statements which makes scripting easier.

Download AirtestIDE on Official Website and use it after decompression. The attached Airtest auxiliary window is very convenient for writing Airtest script statements:

image

Hover the mouse over the button of the Airtest auxiliary window, and you can see the parameter information of the related interface. Just click the button to generate the corresponding sentence. Or you can take a screenshot directly on the screen of the currently connected device. After the screenshot is completed, a sentence with a screenshot will be generated in the script window.

5.1.4 Advanced: Improve the success rate of the script

Airtest is not complicated to get started. It can be said that as long as you have a little basic knowledge of Python, you can easily get started. However, as more and more script code is produced, we will find a very serious problem: Sometimes the running result of our script seems out of our control.For example, we want to determine whether there is an icon on the current screen, and then perform the next operation only when it exists.But it is clearly not on the screen. Airtest still thinks that this icon exists. After opening the report, it is found that it judges that the content of another area on the screen is the icon we want.Another example is a more common situation: we selected a few words on the screen and wanted to recognize them, but the running results were sometimes good or bad. Airtest often thinks that the content we choose does not exist, but sometimes it does work successfully.

This problem is caused by the operating principle of Airtest. We use image recognition technology to find the corresponding picture in the current game screen.However, image recognition is not as accurate as human eye recognition. It can only find the most likely result as much as possible.This often leads to situations where we consider a picture that doesn't exist, it thinks it exists, or what we think can be found at a glance on the screen, Airtest thinks it doesn't exist.

In other words, image recognition is not everything! !! !! It has a success rate. Assuming there are 10 pictures in a script, the recognition success rate of each picture can reach 95%, and the probability of all 10 pictures being correctly recognized is only 60%. That's it.What's more, there are many pictures, for various reasons, the recognition success rate is much lower than 95%, and it is even more difficult for the script to run 100% correctly.

Therefore, after writing the script, we can let the script run a few more times and then improve the part with a low success rate. These are a few notable improvements:

  • When taking a screenshot, try to ensure that the captured image is highly recognizable, independent and clear. For example, when capturing an image of a button, try not to bring too many noisy background patterns which can avoid the problems that it is difficult to successfully identify after changing the background.
  • The algorithm used for image recognition is more suitable for identifying images of buttons (with borders) and icons. The recognition success rate of only intercepting a few texts is very low. Please try to adjust the content of the picture to achieve better Recognition, and avoid capturing poorly-recognized content.
  • Airtest will try to adapt to phones with different resolution as much as possible. However, in some games, there are custom resolution adaptation rules. You can customize them according to the resolution adaptation of your game. The strategy is here.
  • If there are a lot of repeated, very similar icons stacked on top of each other, the recognition effect may be poor. In our eyes, the text on each icon may be different, but in the eyes of Airtest, they are too similar . We can try to modify the screenshot and use some other background styles to modify it into a more recognizable picture.
  • Although we provide a convenient automatic recording function, which can directly convert all current operations into code step by step, in this case, the automatically captured picture is often not ideal, and you need to manually adjust the screenshot.

Configuration items for image recognition

In the process of image recognition by Airtest, there are some common configuration items that can be adjusted to improve the success rate of script execution:

  • In AirtestIDE, you can change the value of threshold to modify the threshold of image recognition by double-clicking the image . The higher the threshold, the higher the accuracy requirements for image matching.
  • When recognizing an image, Airtest will first convert the image into a gray scale image for recognition. So if there are two buttons with the same shape and content but only different colors, Airtest will consider them to be the same content. However, we can force the use of color image recognition by double-clicking the picture and checking the RGB option in the settings.
  • In addition to modifying the two values of threshold andrgb, double-clicking the picture can also set the target position of the image we click on after the recognition is successful. For example, after identifying an icon, we can specify Airtest to click on the lower right corner or upper left corner of the icon.
  • The specific script configuration method can be viewed in 5.2 Airtest script related configuration