III. Poco Window¶

1. Viewing UI Element Trees of Different Projects¶

Poco is a testing framework based on UI recognition. AirtestIDE supports the use of the Poco framework, providing functions such as UI viewing and script automatic recording. Poco's support for different types of applications:

Platforms	Airtest	Poco
Native Android app	√	Directly use
Native iOS app	√	Direct use
Unity3D	√	Poco-SDK Integration Document
Cocos2dx-lua	√	Poco-SDK Integration Document
Cocos2dx-js	√	Poco-SDK Integration Document
Cocos Creator	√	Poco-SDK Integration Document
Egret	√	Poco-SDK Integration Document
UE4	√	Poco-SDK Integration Document
Other engines	√	Self-integration
WeChat mini programs & webview	√	Reference documents (may become invalid with WeChat updates)
Windows, MacOS	√	Under development, please stay tuned.
Netease	√	NetEase internal engine help documentation

To view the UI element tree of the application under test, you need to open the device screen in the IDE and select the corresponding Poco mode in the Poco window. When selecting the mode, you can choose to insert the corresponding initialization code by selecting "Yes" in the script editing window.

Note

For Android and iOS native apps, there is no need to integrate poco-sdk, and you can view UI element tree of the app by simply choosing the corresponding mode in the Poco window. For other game applications rendered by engines, you need to integrate the poco-sdk in the project source code of the game application beforehand, and then package and install it on the test device to choose the corresponding mode in the Poco window and view the UI element tree of the application.

2.Viewing Controls in the Freeze Mode¶

After the successful display of the UI element tree in the Poco window, you can use the freeze mode to view the detailed properties of the controls:

freeze

1) Main purpose¶

To view the UI presentation range for each position in the current screen.

2) Entry method¶

Click the freeze button or any item on the UI rendering tree.

3) Specific manifestation¶

The screen will freeze, device operations will fail, and the data of the UI rendering tree will also stop refreshing.
As the mousing moves across the screen, the UI elements in the corresponding position will be highlighted with a bounding box.
Left click the mouse to view the detailed properties of the corresponding control in the log viewer window.

4) Cascading UI¶

UI with different cascading orders can be selected through the right-click menu.

5) Exit method¶

freeze-2

Click the freeze button again or right-click and select "Poco Mode" - "Normal" from the menu.

3.Viewing Controls in the Check Mode¶

Or we can use the check mode to view the control details of the application. Its biggest difference from the freeze mode is that when viewing the control in this mode, if we operate the control, the device screen will also change accordingly.

check

1) Main purpose¶

Check the rendering status of different UI pages while operating the device.

2) Entry method¶

Click on the check mode button.

3) Specific manifestation¶

The device screen is interactive and the data in the UI rendering tree is refreshing properly.
As the mouse moves across the screen, corresponding UI elements will be highlighted with a bounding box.
Left click the mouse and you will view the detailed properties of the corresponding control in the log viewer window.

4) Cascading UI¶

UI with different cascading orders can be selected through the right-click menu.

5) Exit method¶

Click the check mode button again or right-click and select "Poco Mode" - "Normal" from the menu.

4.Searching for Nodes¶

Click on the control in the UI element tree. Press Ctrl+F can hide/show the search bar (fuzzy Pinyin search is supported).

5.Recording Poco Scripts¶

The script recording function can greatly improve the efficiency of Poco script writing. Currently, AirtestIDE provides two types of Poco recording functions: step recording and automatic recording with operation.

1) Step recording¶

Step recording of Poco scripts can be done in two ways:

The first way is to generate Poco code for a UI node You need to double-click the target item in the UI element tree to insert the corresponding Poco code.
The second way is to generate UI node x-path code To do this, you need to right-click on the target item in the UI element tree and select "UI path-code" from the menu to generate complete parent-child chain of Poco code.

record01

Note

The operation codes (such as ".click()") for the step recording need to be manually added.

2) Automatic recording¶

record02

① Entry method¶

Click on the Poco automatic recording button.

② Specific manifestation¶

The device screen is interactive and the data in the UI rendering tree is refreshing properly.
As the mousing moves across the screen, the UI elements in the corresponding position will be highlighted with a bounding box.
Simply operate the device (such as click and swipe) to insert the corresponding Poco code for the UI node.

③ Exit method¶

Click the automatic recording button again / or: right-click menu - "Poco Mode" - "Normal".

6.Poco's Features for Windows Apps¶

For some Windows apps, such as Unity game windows, which have already integrated Poco-SDK, you can also use AirtestIDE to easily view and select UI nodes. However, before using this feature, you need to inform AirtestIDE of the location and size of the Windows window.

In "Options" - "Settings" - "Poco", select "Windows Content Area Rect", then select the window by drawing a box and double-click to confirm (right-click/esc to cancel). Then, you can use Poco-related features, as shown in the following figure:

Note

Currently, Poco only supports a few Windows apps that have integrated Poco-SDK. More support for Windows apps will come in future updates.

7.Common Issues with Poco Window¶

1) Uncertain which Poco mode to choose for your application¶

Choose Android mode for native Android apps.
Choose iOS mode for native iOS apps.
When testing game applications, it is recommended to consult on which rendering engine to use before integrating with Poco-SDK and selecting the corresponding mode. For example, for Unity games, you should choose the Unity mode.

2) Poco window fails to display UI element tree¶

① Native apps¶

Currently, native Android and iOS apps can be tested without integrating the Poco-SDK. During Poco initialization, relevant APKs will be installed on the phone. Please ensure that the phone allows the installation of these APKs. For some phone models, you may need to enable options such as "Allow installation of apps through ADB" in settings before installation can proceed. Some phone models require additional settings, such as manually setting the Yosemite input method as the default. For more details, please refer to the Special Issues of Certain Phone Models.

② Game applications¶

Game applications are all rendered by various engines and require prior integration with the Poco-SDK to obtain UI structure information (the UI tree cannot be directly viewed without prior SDK integration). Please be sure to carefully read the integration documentation and proceed with using Poco after integrating the SDK.

3) The recorded script cannot be played back¶

The recorded Poco script may not necessarily be the correct/optimal script. If the recorded Poco script fails to achieve your expected results, you can refer to the UI element tree and rewrite the Poco script manually. For more details, please refer to the "Poco script writing" section.

4) The UI element tree fails to refresh correctly¶

You can first try changing to "Stop" in the dropdown menu of the Poco mode, and then select the mode you need, such as Android. The IDE will then re-establish the Poco communication with the phone. Sometimes, due to insufficient phone memory, the phone may automatically close the background pocoservice process, causing the disconnection. In this case, it is recommended to try again with a better configured phone.