III. Methods of Locating Poco Controls¶

1.Preface¶

This article will explain in detail the various methods of locating Poco controls. With these methods, you can write scripts to locate the target controls. The common Poco scripts recorded in the IDE are like poco("star_single").click(), where the poco("star_single") belongs to the Poco control location script.

2.Three Targeting Selectors¶

The most basic three targeting selectors for Poco controls are:

Basic Selector
Relative Selector
Sequence Selector

1) Basic selector¶

After adding a pair of parentheses to the Poco instance, we can perform element selection. The selector will traverse all elements to select those that meet the given conditions and return them.

The parameters in parentheses represent the given conditions, expressed as attribute name-value pairs. The first parameter indicates the node name, just like poco("star_single"). Optional parameters can also be added after that to represent the node's attributes and expected attribute values.

from poco.drivers.unity3d import UnityPoco
poco = UnityPoco()

poco("star_single",type="Image")

2) Relative selector¶

If using node attributes (or basic selectors alone) cannot select the desired element, you can also select through the rendering hierarchy between elements, such as parent-child, sibling, offspring, etc:

poco("plays").child("playBasic").offspring("star_single")

For more information about the APIs related to parent-child and sibling hierarchy, please visit this page: https://poco.readthedocs.io/zh_CN/latest/source/poco.proxy.html?highlight=offspring#poco.proxy.UIObjectProxy.offspring.

3) Sequence selector¶

Selecting in sequence always follows the spatial arrangement, starting from left to right and then from top to bottom row by row. As shown in the figure below, we use the selector to select many elements with type="Text", and then use the index order to select each element one by one:

name0 = poco("Content").child(type="Text")[0].get_name()
name1 = poco("Content").child(type="Text")[1].get_name()
name2 = poco("Content").child(type="Text")[2].get_name()

print(name0+" "+name1+" "+name2)

There is a special case for index selection. If the position of the element changes after it is selected, the index number will still be the same as when it is selected. For example, if element ① is selected and then moves to the position of element ③, it is still accessed by poco(...)[0], not poco(...)[2]. If an element disappears (removed from the interface or hidden) after selection, accessing that element again may cause an exception, but the rest of the elements can still be accessed.

3.Match Controls Using Regular Expressions¶

In addition to the three common methods mentioned above, there is another less common but very useful way of locating controls, which is to use regular expressions to match controls, as shown in the following example:

# select the UI element(s) which text attribute matches the pattern '^close.*$'

poco = Poco(...)
arb_close_btn = poco(textMatches='^close.*$')

1) A simple example¶

Let's take a closer look at how to use regular expressions for matching Poco controls with a simple example.

This is the icon control of the Taobao App. By using the basic selector, we can locate this Taobao icon control with code like poco(text="Taobao"). If we want to use regular expressions to match this control, we can do so by using the following method:

poco(textMatches="Regular expression that can match with Taobao")

There are many regular expressions that can match "Taobao", such as ".Taobao". Therefore, we can write the statement for clicking the Taobao icon control as follows:

poco(textMatches=".*Taobao").click()

In addition to textMatches, we can also use nameMatches to match the name attribute of the control: With the basic selector, we can write the following script to locate the control in the image:

poco(name="com.netease.cloudmusic:id/portalTitle",text="Daily Recommendation")

With the regular expression, we can write the script like this:

poco(nameMatches=".*portalTitle",textMatches=".*Recommendation")

2) Writing concise and efficient location scripts with regular expressions¶

When writing location scripts for controls deeply hidden in the hierarchy, regular expressions can help make the scripts concise and efficient.

As shown in the figure above, if we want to obtain detailed information for all songs on the current page by using the selectors mentioned above, the script may be as follows:

for i in poco("com.netease.cloudmusic:id/pagerListview").child("com.netease.cloudmusic:id/musicListItemContainer"):
    info = i.child("com.netease.cloudmusic:id/songNameAndInfoArea").offspring("com.netease.cloudmusic:id/songInfo")
    print(info.get_text())

It can be seen that location scripts written using basic selectors and relative selectors appear very cumbersome and demands a very clear understanding of the element hierarchy, otherwise the script is prone to errors. Let's try using regular expressions instead. After carefully observing the UI element tree, we will find that the control names of these song information are the same. So, we only need to write one regular expression to match this group of identical control names, equivalent to locating all the song information controls on the current page. Next, we can use Poco to traverse and obtain the text attribute of each control one by one.

for i in poco(nameMatches="com.*?songInfo"):
    print(i.get_text())

Apart from the most common textMatches, nameMatches, and typeMatches, most attributes can actually be passed using regular expressions in this way. As long as the control can be selected using poco(xx=expected attribute value), it can be matched and located using poco(xxMatches=regular expression for expected attribute value).

3) Tools for checking if a regular expression works¶

If you need a tutorial for regular expressions, we recommend reading this classic 30-minute quick start guide: https://deerchao.cn/tutorials/regex/regex.htm. Additionally, if you want to check if a regular expression you have written can match the expected target, you can simply test its matching result on websites such as: https://tool.oschina.net/regex/.

4.Common Issues with Poco Location Scripts¶

1) Distinguishing the Control Location Script from the Control Operation Script¶

After recording/writing the location script for Poco, many users may directly run it and find no response on the device screen at all. This is because the location script can only filter the controls, and you need to add the script for the expected operation, such as touch(), to truly execute some operations on this control.

# Poco control location script
poco("star_single")

# Poco control operation script
poco("star_single").click()

2) Non-recommended Poco location methods¶

The previous text has introduced many methods of locating Poco controls, but we usually do not recommend using the relative selector and the sequence selector because the complex hierarchy coupled with the spatial indexing sequence can easily result in poor runtime efficiency or failures to locate controls due to changes in index values. Unless absolutely necessary, we generally do not recommend using these two methods.

3) Recommended Poco location methods¶

We highly recommend using basic selectors and regular expressions to locate controls.

For example, if we want to click on the "Daily Recommendation" control on the NetEase Cloud Music homepage and write a script using text attributes for locating, it will be very concise and efficient using these methods, without worrying about the complex hierarchy and spatial index order.

In addition to the text attribute, we can also use a name attribute or other unique attribute values of a node to locate control in Poco, as long as it is unique on the current page. It is even possible to use multiple attribute values for precise control location. We highly recommend using this method to write Poco location scripts.

As for the reason for recommending the use of regular expressions to match controls, it has already been mentioned earlier. In some cases, using regular expressions to locate controls will be very efficient and concise.