doc -cn
This commit is contained in:
饭桶狼 ra～
parent
406372d603
commit
14481759c2
437
README.md
437
README.md
@ -1,104 +1,118 @@
|
||||
English | [简体中文](./README_zh-CN.md)
|
||||
简体中文 | [English](./README_en.md)
|
||||
|
||||
# GO-MOBILE-AUTOMATION SDK
|
||||
# GO手机自动化SDK
|
||||
|
||||
A full featured Android mobile automation sdk for golang developers
|
||||
功能齐全的手机自动化Golang SDK，支持编译成二进制可执行文件，适合自动化流程部署到云手机，脱离ADB连接稳定高效执行
|
||||
|
||||
## The purpose
|
||||
## 目的
|
||||
|
||||
If you are an automation developer, you may find the python/javascript echo system provide the developers great capabilities for manipulating devices and apps.
|
||||
在（手机）自动化领域，其实python是绝对的主流，其他语言诸如Golang/C#/C++在整个生态中占的份额相对来说比较少。对于安卓手机自动化开发来说，我们起码有下面的工具了：
|
||||
|
||||
For Android automation, some well known tools are:
|
||||
1. Appium（多语言/全平台自动化支持）
|
||||
2. Uiautomator2/ATX (python)
|
||||
3. 各RPA平台，比如云扩科技 (C#)
|
||||
|
||||
1. Appium (multi-language primarily javascript/python)
|
||||
2. uiautomator2 (python)
|
||||
我们有这么多python库可以用，为什么要再做个Golang自动化SDK呢？
|
||||
|
||||
Our SDK ports the uiautomation2 python library to golang.
|
||||
1. 容易部署 - Golang一般只需要部署一个二进制可执行文件，不像python/javascript需要安装一大堆依赖，在墙内如果没有一个稳定可靠的镜像(的确，javascript有CNPM，也可以自己搭镜像)，如果安装依赖挂了就没有然后了。即使下载依赖没有问题，也会有一些依赖版本冲突的问题。C#/JAVA有运行时依赖，你一般不会知道用户会使用哪个版本的.NET SDK/JDK。
|
||||
2. 部署在云手机上稳定可靠 - 由于云手机使用虚拟化技术，它不像传统机架集群会遇到各式各样的硬件文件 - 比如电池爆膨了（一般2年寿命，工作室机更差），比如电源供电不足手机断电，比如USB线接触不良，等等。云手机相对贵一点但是稳定且省人力成本。但是有些云手机使用外网远程adb连接，如果自动化脚本host在本地的PC/MACOS上远程操控云手机，稳定性非常取决于网络的稳定性。我们当然更希望使用Golang/C++的方式：编译一个二进制可执行文件，推到云手机上定时/按某种触发条件去执行，脱离ADB连接
|
||||
3. 稳定 - 事实上python能直接跑在手机上。pyto-python3这个app提供了host python脚本在手机上的可能。但是手机应用比较容易被Android系统杀死，而二进制可执行文件一般不会。
|
||||
|
||||
Why do we do this?
|
||||
所以我们采用Golang做安卓自动化语言是有道理滴～
|
||||
|
||||
1. Easy to deploy - deploy one executable only (at most several dlls) instead of resolving thousands of dependencies like javascript and python. The bigger scope of this family of projects is to provide a mechanism to orchestrate the automation "scripts" to run cross a bunch of platform/systems. Must have a fast & robustic app distribution and deployment mechanism.
|
||||
2. For using cloud phones (Android) - Some cloud phone providers have very poor quality adb connection. In case the automation process break because of adb connection failure, we would like the process run inside the phone
|
||||
3. Robust - apps can easily get killed by Android system, but executable not. That why despite the fact that apps like pyto-python3 provides hosting for python but we don't use it.
|
||||
再说说其他自动化工具坑的地方：
|
||||
1. Appium安装复杂，而且图像识别只支持模版匹配，中间层N多，上层是well known的protocol，要优化性能或者加一些底层支持非常困难。要部署Appium到一台新的主机虚拟机，别提有多烦了
|
||||
2. UiAutomator2/ATX。这个软件库我是极其推崇，它的API支持比我们的Golang SDK要丰富，毕竟人家才是原版。但是它不提供直接将流程部署到手机的能力。这不能说是缺陷，因为考虑到自动化开发的受众群体，python是绝对绝对的主流
|
||||
3. 云扩RPA。它其实本身是一个很棒的平台，但是为了做手机自动化安装N多东西不值当。但是如果有别的需求，比如使用低代码做界面层，可视化的流程管理工具，它是一个不错的选择。这个SDK首先会和云扩RPA做集成。
|
||||
|
||||
## Inspired by
|
||||
唯一要注意的是Android是只读文件系统，这意味着打日志的话我们可能需要批量收集，在程序里面做批量推送到日志服务器
|
||||
|
||||
Inspired by [OpenAtx](https://github.com/openatx) and the [Uiautomation2](https://github.com/openatx/uiautomator2) python library. We entirely use the openatx drivers, leaving the client side sdk written in golang. This benefits users, because they can use the uiautomator2 tool chain, which is super cool.
|
||||
## 项目启发
|
||||
|
||||
项目启发于[OpenAtx](https://github.com/openatx)和[Uiautomation2](https://github.com/openatx/uiautomator2) python库。我事实上在项目中首先大量使用了uiautomator2，在玩儿了云手机之后，因为有需要，自然而然就有了这个Golang库（因为Golang比C++简单，ARM Cortex A编辑不需要NDK）
|
||||
|
||||
它事实上是OpenAtx的又一个客户端，但是它可以跨平台执行。它实现了大部分Uiautomator2的API。这么做的好处是，OpenAtx的工具链，包括它的Inspector [weditor](https://github.com/alibaba/web-editor) - 这个真的很好用，而且是web应用不需要额外占我太多存储空间
|
||||
|
||||
而且有别于Appium，OpenAtx不需要依赖Session同时在一个App上执行流程，它可以同时操作一个App，点击另一个App的浮层。这是我喜欢并使用OpenAtx的原因。
|
||||
|
||||
## Quick start
|
||||
|
||||
There are 4 steps:
|
||||
我分四步讲
|
||||
|
||||
1. Setup the Android phone
|
||||
2. Setup the development environment
|
||||
3. Start creating a golang project for mobile automation
|
||||
4. deploy&run
|
||||
1. 设置安卓手机
|
||||
2. 设置开发环境
|
||||
3. 创建Golang自动化项目
|
||||
4. 部署&执行
|
||||
|
||||
### Setup the Android phone
|
||||
### 设置安卓手机
|
||||
|
||||
1. Install the specific version of the app you want to automate.
|
||||
1. 安装你要自动化的APP（注意：安装特定的版本 - 因为不同版本同一个元素的属性很可能是不一样的 - 即使同一个版本有些属性其实也会变 - 反映到脚本中就是xpath）
|
||||
```
|
||||
$ adb install [package.apk]
|
||||
```
|
||||
2. Download atx-agent from [here](https://github.com/openatx/atx-agent/releases) choose the armv7 version unless you run a x86 phone simulator
|
||||
3. Untar atx-agent and install, follow the intallation insttructions [here](https://github.com/openatx/atx-agent)
|
||||
2. 下载 atx-agent: [点这里](https://github.com/openatx/atx-agent/releases) 选择 armv7 除非你部署的手机是x86手机模拟器
|
||||
3. 解压 atx-agent 并安装, 这里有安装说明: [点这里](https://github.com/openatx/atx-agent) 或者看下面的命令行指令：
|
||||
```
|
||||
$ adb push atx-agent /data/local/tmp
|
||||
$ adb shell chmod 755 /data/local/tmp/atx-agent
|
||||
# launch atx-agent in daemon mode
|
||||
# 后台模式执行atx-agent
|
||||
$ adb shell /data/local/tmp/atx-agent server -d
|
||||
|
||||
# stop already running atx-agent and start daemon
|
||||
# 或者后台模式重启atx-agent
|
||||
$ adb shell /data/local/tmp/atx-agent server -d --stop
|
||||
```
|
||||
4. Download app-uiautomator-test.apk and app-uiautomator.apk from [here](https://github.com/openatx/android-uiautomator-server/releases) and install using adb install
|
||||
4. 下载 app-uiautomator-test.apk 和 app-uiautomator.apk [点这里](https://github.com/openatx/android-uiautomator-server/releases) 然后用adb install命令安装
|
||||
```
|
||||
$ adb install app-uiautomator-test.apk
|
||||
$ adb install app-uiautomator.apk
|
||||
```
|
||||
5. Grant all priviledges to the app "ATX"
|
||||
6. Open app "ATX" and click "启动UIAUTOMATOR", click "开启悬浮窗"
|
||||
5. 给应用 "ATX" 所有权限，并且打开 "ATX" 检查有任何运行时权限请求的话，选择一直许可。
|
||||
6. 打开应用 "ATX" 点击 "启动UIAUTOMATOR", 点击 "开启悬浮窗"
|
||||
|
||||
## Setup the development environment
|
||||
## 设置开发环境
|
||||
|
||||
1. Install Python3(version 3.6+) from [here](https://www.python.org/downloads/)
|
||||
2. Install [weditor](https://github.com/alibaba/web-editor)
|
||||
1. 安装 Python3(版本 3.6+) from [here](https://www.python.org/downloads/)
|
||||
2. 安装 [weditor](https://github.com/alibaba/web-editor)
|
||||
```
|
||||
$ pip3 install -U weditor
|
||||
```
|
||||
3. You can now open weditor as ui inspector for android applications, by typing
|
||||
3. 打开 weditor, 它就是你的UI Inspector，在命令行中输入,
|
||||
```
|
||||
$ weditor
|
||||
```
|
||||
|
||||
## Create a golang automation project
|
||||
## 创建Golang自动化项目
|
||||
|
||||
1. Create a folder called helloworld
|
||||
2. Open terminal and type in
|
||||
1. 创建文件夹 helloworld
|
||||
2. 打开命令行并输入
|
||||
```
|
||||
$ go mod init helloworld
|
||||
```
|
||||
3. Add dependency
|
||||
3. 添加依赖
|
||||
```
|
||||
$ go get github.com/fantonglang/go-mobile-automation
|
||||
```
|
||||
4. Add the program entry - create main.go file
|
||||
5. [Here](https://github.com/fantonglang/go-mobile-automation-examples/blob/main/douyin-luo-live/main.go) is an example of main.go file
|
||||
4. 添加程序入口 - 创建 main.go 文件
|
||||
5. [这里](https://github.com/fantonglang/go-mobile-automation-examples/blob/main/douyin-luo-live/main.go) 是示例 main.go 文件
|
||||
|
||||
## Deploy&Run
|
||||
## 部署&执行
|
||||
```
|
||||
# cross build linux/arm target
|
||||
# 交叉编译(cross build) linux/arm 可执行文件
|
||||
$ GOOS=linux GOARCH=arm go build
|
||||
# deploy - helloworld is the executable name, which is the same with the go module name
|
||||
# 部署 - helloworld 是可执行文件名，和Go模块名是相同的
|
||||
$ adb push helloworld /data/local/tmp
|
||||
$ adb shell chmod 755 /data/local/tmp/helloworld
|
||||
# run
|
||||
# 运行
|
||||
$ adb shell /data/local/tmp/helloworld
|
||||
```
|
||||
If you start a background process, you don't need the phone to connect with your PC/macos. Note that you can transfer your linux shell knowledge to using the adb shell.
|
||||
如果你用后台模式启动程序，程序启动之后就不需要连接电脑。adb shell命令和普通linux系统是一样的。你可以输入
|
||||
```
|
||||
# nohup保证当关闭terminal session的时候，程序不会被杀死，&保证在后台运行程序
|
||||
$ adb shell nohup /data/local/tmp/helloworld &
|
||||
```
|
||||
|
||||
## Examples
|
||||
[This](https://github.com/fantonglang/go-mobile-automation-examples/tree/main/douyin-luo-live) is a working example. Read the comments in main.go carefully. This helps to resolve all dependencies and environment requirements before you start. The comments also give the commands for compilation, deployment, and execution.
|
||||
## 例子
|
||||
[这里](https://github.com/fantonglang/go-mobile-automation-examples/blob/main/douyin-luo-live) 是一个能跑起来的例子。仔细阅读main函数的注释，里面有手机设置，环境安装，编译，调试，部署，执行的教程。
|
||||
|
||||
# APIS
|
||||
|
||||
@ -128,7 +142,7 @@ If you start a background process, you don't need the phone to connect with your
|
||||
**[XPATH](#xpath)**
|
||||
- **[Finding elements](#finding-elements)**
|
||||
- **[Xpath elements API](#xpath-elements-api)**
|
||||
|
||||
|
||||
**[UI Object](#ui-object)**
|
||||
- **[construct query](#construct-query)**
|
||||
- **[execute ui object query](#execute-ui-object-query)**
|
||||
@ -136,9 +150,9 @@ If you start a background process, you don't need the phone to connect with your
|
||||
|
||||
## Connect to a device
|
||||
|
||||
There are two types of connection:
|
||||
有两种类型的连接：
|
||||
|
||||
1. If the executable is deployed in the phone, use
|
||||
1. 如果程序是部署在手机上，使用如下代码：
|
||||
``` go
|
||||
package main
|
||||
|
||||
@ -147,10 +161,10 @@ import (
|
||||
"github.com/fantonglang/go-mobile-automation/apis"
|
||||
)
|
||||
...
|
||||
// you don't need to specify device id, because there is no PC connection
|
||||
// 不需要指定设备ID，因为程序并不需要从电脑通过ADB连接手机
|
||||
d := apis.NewNativeDevice()
|
||||
```
|
||||
2. If you debug and deploy in PC/macos, use
|
||||
2. 如果是在电脑端调试或部署，使用如下代码：
|
||||
``` go
|
||||
package main
|
||||
|
||||
@ -160,7 +174,7 @@ import (
|
||||
)
|
||||
|
||||
|
||||
//here c574dd45 is the device id, replace it with yours own
|
||||
//这里 c574dd45 是设备ID, 你可以从adb devices指令中获取到它, 把它替换成你自己的手机设备ID
|
||||
d, err := apis.NewHostDevice("c574dd45")
|
||||
if err != nil {
|
||||
log.Println("failed connecting to device")
|
||||
@ -168,7 +182,7 @@ if err != nil {
|
||||
}
|
||||
```
|
||||
|
||||
Combine 2 code snippets. The following code enables the same piece of code working on the both deployments.
|
||||
结合上述两个代码片段，下面的代码既能在电脑端(假定是x86架构)调试的时候工作，又能在Android设备中部署之后工作。它判断运行时如果是ARM，使用手机的方式连接；反之使用电脑的方式连接。特别注意苹果MAC最近的ARM架构电脑，如果这种情况，最好判断一下系统(GOOS)
|
||||
``` go
|
||||
package main
|
||||
|
||||
@ -182,7 +196,7 @@ func getDevice() *apis.Device {
|
||||
if runtime.GOARCH == "arm" {
|
||||
return apis.NewNativeDevice()
|
||||
}
|
||||
//here c574dd45 is the device id, replace it with yours own
|
||||
//这里 c574dd45 是设备ID, 你可以从adb devices指令中获取到它, 把它替换成你自己的手机设备ID
|
||||
_d, err := apis.NewHostDevice("c574dd45")
|
||||
if err != nil {
|
||||
log.Println("101: failed connecting to device")
|
||||
@ -194,24 +208,22 @@ func getDevice() *apis.Device {
|
||||
d := getDevice()
|
||||
```
|
||||
|
||||
Take extra notice if your macos is the ARM architecture. Then you may judge based on GOOS.
|
||||
|
||||
## Device APIS
|
||||
This part showcases how to perform common device operations:
|
||||
这部分展示如何执行常见的设备操作
|
||||
|
||||
### Shell commands
|
||||
Example: Force stop douyin(China tiktok) app
|
||||
示例: 强制停止抖音APP
|
||||
```go
|
||||
d.Shell(`am force-stop com.ss.android.ugc.aweme`)
|
||||
```
|
||||
Example: Start douyin app
|
||||
示例: 打开抖音APP
|
||||
```go
|
||||
// You can find the app main activity by using the dumpsys command, hence I didn't implement the uiautomator2 equivalent session API for now.
|
||||
// 使用dumpsys命令，你可以找到APP的启动Activity。所以目前并不实现uiautomator2的启动APP API，以及session API。
|
||||
d.Shell(`am start -n "com.ss.android.ugc.aweme/.main.MainActivity"`)
|
||||
```
|
||||
|
||||
### Retrieve the device info
|
||||
Get detailed app info
|
||||
获取详细设备信息
|
||||
```go
|
||||
info, err := d.DeviceInfo()
|
||||
if err != nil {
|
||||
@ -226,7 +238,7 @@ if err != nil {
|
||||
fmt.Println(string(bytes))
|
||||
```
|
||||
|
||||
Below is a possible output:
|
||||
下面是可能的输出：
|
||||
|
||||
```json
|
||||
{
|
||||
@ -241,7 +253,7 @@ Below is a possible output:
|
||||
}
|
||||
```
|
||||
|
||||
Get window size:
|
||||
获取窗口大小：
|
||||
|
||||
```GO
|
||||
w, h, err := d.WindowSize()
|
||||
@ -250,14 +262,14 @@ if err != nil {
|
||||
return
|
||||
}
|
||||
fmt.Printf("w: %d, h: %d\n", w, h)
|
||||
// device upright output example: w: 1080, h: 2340
|
||||
// device horizontal output example: w: 1080, h: 2340
|
||||
// 设备竖屏时的输出示例: w: 1080, h: 2340
|
||||
// 设备横屏时的输出示例: w: 1080, h: 2340
|
||||
```
|
||||
|
||||
### Clipboard
|
||||
Get of set clipboard content
|
||||
获取和设置剪切板内容
|
||||
|
||||
set clipboard
|
||||
设置剪切板内容
|
||||
```go
|
||||
err := d.SetClipboard("aaa")
|
||||
if err != nil {
|
||||
@ -265,7 +277,7 @@ if err != nil {
|
||||
return
|
||||
}
|
||||
```
|
||||
get clipboard: This doesn't work in Android > 9.0. Most cloud phone work on lower Android version. I don't mind.
|
||||
获取剪切板内容: 在Android大于9.0这个API并不工作. 但是大多数云手机使用低版本Android(比如7.0)，所以我并不care
|
||||
```go
|
||||
a, err := d.GetClipboard()
|
||||
if err != nil {
|
||||
@ -277,24 +289,24 @@ fmt.Println(a)
|
||||
|
||||
### Key Events
|
||||
|
||||
* Turn on/off screen
|
||||
* 打开/关闭屏幕
|
||||
```go
|
||||
err := d.KeyEvent(KEYCODE_POWER) // press power key to turn on/off screen
|
||||
err := d.KeyEvent(KEYCODE_POWER) // 打开关闭屏幕都是按电源键
|
||||
```
|
||||
* Home key
|
||||
* Home键
|
||||
```go
|
||||
err := d.KeyEvent(KEYCODE_HOME)
|
||||
```
|
||||
|
||||
d.KeyEvent is basically the Android "input keyevent " command, please refer to [this doc](https://developer.android.com/reference/android/view/KeyEvent), or if you're behind gfw, [this doc](https://blog.csdn.net/feizhixuan46789/article/details/16801429)
|
||||
d.KeyEvent 是在调用 Android 的 "input keyevent " 命令, 请参照 [这个文档](https://developer.android.com/reference/android/view/KeyEvent), 或者你如果没有翻墙工具, 参照 [这个文档](https://blog.csdn.net/feizhixuan46789/article/details/16801429)
|
||||
|
||||
### Press Key
|
||||
Example: press Home key
|
||||
示例: 按Home键
|
||||
```go
|
||||
err := d.Press("home")
|
||||
```
|
||||
|
||||
supported keys are:
|
||||
Press API支持下面的按键:
|
||||
```go
|
||||
VSK_HOME = "home"
|
||||
VSK_BACK = "back"
|
||||
@ -317,153 +329,155 @@ VSK_POWER = "power"
|
||||
```
|
||||
|
||||
### New command timeout
|
||||
How long (in seconds) will wait for a new command from the client before assuming the client quit and ending the uiautomator service
|
||||
设置Uiautomator服务的超时时间
|
||||
|
||||
```go
|
||||
err := d.SetNewCommandTimeout(300) // unit is second
|
||||
err := d.SetNewCommandTimeout(300) // 单位秒
|
||||
```
|
||||
|
||||
### Screenshot
|
||||
* Screenshot and save - notice Android has readonly file system, this API is only available for host(PC/macos).
|
||||
* 截屏并保存文件 - 注意 Android 使用只读文件系统, 这个API只有在电脑端有效.
|
||||
```go
|
||||
err := d.ScreenshotSave("sc.png")
|
||||
```
|
||||
* Screenshot and get bytes (preferred, because opencv can accept bytes directly by "cv::imdecode" function)
|
||||
* 截屏并获取[]byte字节流 (推荐, 因为opencv使用cv::imdecode函数能直接读取字节流)
|
||||
```go
|
||||
bytes, err := d.ScreenshotBytes()
|
||||
```
|
||||
* Screenshot and get image.Image object
|
||||
* 截屏并获取image.Image对象 （如果需要不同的图片编码，比如jpeg/png，使用image.Image可以帮你做到转换）
|
||||
```go
|
||||
img, format, err := d.Screenshot() // img is the image.Image object, example of format: "jpeg"
|
||||
img, format, err := d.Screenshot() // img 是 image.Image 对象, format的示例: "jpeg"
|
||||
```
|
||||
|
||||
### UI Hierarchy
|
||||
* Get hierachy text
|
||||
* 获取UI结构的XML文本
|
||||
```go
|
||||
content, err := d.DumpHierarchy(false, false) // content is the text
|
||||
content, err := d.DumpHierarchy(false, false) // content 是文本
|
||||
```
|
||||
* Transform hierachy text to *xmlquery.Node object. (This is useful if you want to do xpath query based on a snapshot - this is a lot faster)
|
||||
* 将UI结构的XML文本转换成 *xmlquery.Node 对象. (我们如果要基于snaphot执行xpath查询，那 *xmlquery.Node 对象是非常有用的 - 它不涉及Uiautomator调用，所以速度会非常快，适合广告页面识别关闭的场景)
|
||||
```go
|
||||
doc, err := FormatHierachy(content) // doc is the *xmlquery.Node object
|
||||
doc, err := FormatHierachy(content) // doc 是 *xmlquery.Node 对象
|
||||
```
|
||||
|
||||
### Touch
|
||||
Simulate "mouse press down", "mouse hold and move", "mouse up release"
|
||||
* Get touch object
|
||||
模拟“手指按下触屏”，“手指按住触屏拖动”，以及“手指离开触屏”
|
||||
* 获取 touch 对象
|
||||
```go
|
||||
touch := d.Touch()
|
||||
```
|
||||
* Mouse press down at a position
|
||||
* 手指按下触屏 - 在某个位置
|
||||
```go
|
||||
/* press at (relative to the top-left corner)
|
||||
* x: 50% position of the width
|
||||
* y: 60% position of the height
|
||||
/* （相对于屏幕左上角）手指按下触屏，位置为
|
||||
* x: 50% 宽度坐标
|
||||
* y: 60% 高度坐标
|
||||
*/
|
||||
err := touch.Down(0.5, 0.6)
|
||||
```
|
||||
* Mouse hold and move
|
||||
* 手指按住触屏拖动 - 到某个位置
|
||||
```go
|
||||
/* then move to (relative to the top-left corner)
|
||||
* x: 50% position of the width
|
||||
* y: 10% position of the height
|
||||
/* （相对于屏幕左上角）然后拖动到下述位置
|
||||
* x: 50% 宽度坐标
|
||||
* y: 10% 高度坐标
|
||||
*/
|
||||
err := touch.Move(0.5, 0.1)
|
||||
```
|
||||
* Mouse up release
|
||||
* 手指离开触屏
|
||||
```go
|
||||
/* then mouse up release at (relative to the top-left corner)
|
||||
* x: 50% position of the width
|
||||
* y: 10% position of the height
|
||||
/* （相对于屏幕左上角）然后手指离开触屏，位置为
|
||||
* x: 50% 宽度坐标
|
||||
* y: 10% 高度坐标
|
||||
*/
|
||||
err := touch.Up(0.5, 0.1)
|
||||
```
|
||||
|
||||
### Click
|
||||
Click on screen given coordinates
|
||||
在指定坐标位置点击屏幕
|
||||
|
||||
* coordinates using percentage
|
||||
* 使用百分比 - 如果x,y的任意一个值在 [0,1) 的范围内，它就是表示百分比，反之如果在[1, maxScreenWidth/maxScreenHeight]的范围内，它就是绝对坐标值
|
||||
```go
|
||||
/* click relative to the top-left corner
|
||||
* x: 48.1% position of the width
|
||||
* y: 24.6% position of the height
|
||||
/* 相对于屏幕左上角点击
|
||||
* x: 48.1% 宽度坐标
|
||||
* y: 24.6% 高度坐标
|
||||
*/
|
||||
err := d.Click(0.481, 0.246)
|
||||
```
|
||||
* coordinates using absolute pixel values
|
||||
* 使用绝对坐标 - 如果x,y的任意一个值在 [0,1) 的范围内，它就是表示百分比，反之如果在[1, maxScreenWidth/maxScreenHeight]的范围内，它就是绝对坐标值
|
||||
```go
|
||||
/* click relative to the top-left corner
|
||||
* at (x: 481, y: 246)
|
||||
*/
|
||||
// 相对于屏幕左上角点击(x: 481, y: 246)
|
||||
err := d.Click(481, 246)
|
||||
```
|
||||
|
||||
### Double Click
|
||||
双击
|
||||
```go
|
||||
err := d.DoubleClickDefault(0.481, 0.246)
|
||||
```
|
||||
|
||||
### Long Click
|
||||
Mouse click, but there is a certain time interval(0.5s) between mouse down and up
|
||||
长按点击，相当于按下和松开之间隔了一个给定时间，默认0.5s。函数名后面带Default的一般有一个非Default版本，提供更多参数选择。
|
||||
```go
|
||||
err := d.LongClickDefault(0.481, 0.246)
|
||||
```
|
||||
|
||||
### Swipe
|
||||
* Swipe from one point (fx, fy) to another (tx, ty)
|
||||
滑动
|
||||
* 从起始点 (fx, fy) 滑动到终点 (tx, ty)
|
||||
```go
|
||||
var fx, fy, tx, ty float32 = 0.5, 0.5, 0, 0
|
||||
err := d.SwipeDefault(fx, fy, tx, ty)
|
||||
```
|
||||
* Swipe points, you can specify more than 2 points
|
||||
* 多点滑动, 参数可以指定多个apis.Point4Swipe对象，代表滑动途径的坐标点
|
||||
```go
|
||||
// swipe from (x=width*0.5, y=height*0.9) to (x=width*0.5,y=height*0.1)
|
||||
// 滑动途经 起点(x=width*0.5, y=height*0.9) 到 终点(x=width*0.5,y=height*0.1)，滑动总时长0.1秒
|
||||
err := d.SwipePoints(0.1, apis.Point4Swipe{0.5, 0.9}, apis.Point4Swipe{0.5, 0.1})
|
||||
```
|
||||
Drag from one point (fx, fy) to another (tx, ty)
|
||||
* 从起点(fx, fy) 拖动到 终点(tx, ty)
|
||||
```go
|
||||
var fx, fy, tx, ty float32 = 0.5, 0.5, 0, 0
|
||||
err := d.DragDefault(fx, fy, tx, ty)
|
||||
```
|
||||
|
||||
### Set Orientation
|
||||
Accepts 4 orientation parameters:
|
||||
* "n" - means natural
|
||||
* "l" - means left
|
||||
* "u" - means upsidedown
|
||||
* "r" - means right
|
||||
设置屏幕方向，接受下面这四种参数:
|
||||
* "n" - 代表正常的竖屏
|
||||
* "l" - 代表朝左的横屏
|
||||
* "u" - 代表倒过来的竖屏
|
||||
* "r" - 代表朝右的横屏
|
||||
```go
|
||||
err := d.SetOrientation("n")
|
||||
```
|
||||
|
||||
### Open Quick Settings
|
||||
打开[快速设置菜单](https://www.lifewire.com/quick-settings-menu-android-4121299)
|
||||
```go
|
||||
err := d.OpenQuickSettings()
|
||||
```
|
||||
|
||||
### Open Url
|
||||
打开浏览器输入URL，打开网页
|
||||
```go
|
||||
err := d.OpenUrl("https://bing.com")
|
||||
```
|
||||
|
||||
### Show float window
|
||||
This operation is openatx specific - open a float window to keep the automator app in the front and prevent it from getting killed
|
||||
显示弹窗。这个操作是openatx特有的 - 它打开一个浮层来做应用保活。在这个SDK的产品级部署上面，我们在每次流程开始时都会用d.Shell函数打开ATX应用，再使用自研的分辨率无关的控件图像识别找到“启动UIAUTOMATOR”按钮并点击，再打开ATX应用用同样的方法找到“开启悬浮窗”按钮并点击。以避免在流程执行的过程中，由于uiautomator已经被杀死而重新唤起它 - 虽然这个操作会自动发生，但是有时候会很慢。
|
||||
```go
|
||||
err := d.ShowFloatWindow(true)
|
||||
```
|
||||
|
||||
|
||||
## Input Method
|
||||
Type text, (you will switch to a special input method)
|
||||
* Clear Text
|
||||
用来输入文字，(会使用一个特殊的输入法)
|
||||
* 清除文字
|
||||
```go
|
||||
err := d.ClearText()
|
||||
```
|
||||
* Send Action
|
||||
* 发送动作
|
||||
```go
|
||||
err := ime.SendAction(SENDACTION_SEARCH)
|
||||
```
|
||||
|
||||
The following actions are supported:
|
||||
支持下面的动作：
|
||||
```go
|
||||
SENDACTION_GO = 2
|
||||
SENDACTION_SEARCH = 3
|
||||
@ -473,33 +487,33 @@ SENDACTION_DONE = 6
|
||||
SENDACTION_PREVIOUS = 7
|
||||
```
|
||||
|
||||
* Send Keys - Type text
|
||||
* 发送按键 - 输入文字（包括中文以及其他Unicode）
|
||||
```go
|
||||
err := ime.SendKeys("aaa", true)
|
||||
```
|
||||
|
||||
## XPATH
|
||||
XPATH is the most important way of finding UI Element.
|
||||
在Android自动化中，XPATH是最重要的寻找UI元素的方法。快速又强大
|
||||
|
||||
### Finding elements
|
||||
* Find multiple elements by xpath
|
||||
* 通过Xpath查找多个元素
|
||||
```go
|
||||
els := d.XPath(`//*[@text="your-control-text"]`).All()
|
||||
for _, el := range els {
|
||||
...
|
||||
}
|
||||
```
|
||||
* Find one element by xpath
|
||||
* 通过Xpath查找一个元素（首个或者nil）
|
||||
```go
|
||||
el := d.XPath(`//*[@text="your-control-text"]`).First()
|
||||
```
|
||||
* Check element exists by xpath
|
||||
* 通过Xpath检查元素是否存在, el.First()函数当元素不存在的时候返回nil
|
||||
```go
|
||||
if d.XPath(`//*[@text="your-control-text"]`).First() != nil {
|
||||
...
|
||||
}
|
||||
```
|
||||
* Wait element appear
|
||||
* 等待元素出现
|
||||
```go
|
||||
el := d.XPath(`//*[@text="your-control-text"]`).Wait(time.Minute)
|
||||
if el == nil {
|
||||
@ -508,7 +522,7 @@ if el == nil {
|
||||
}
|
||||
...
|
||||
```
|
||||
* Wait element disappear
|
||||
* 等待元素消失
|
||||
```go
|
||||
ok := d.XPath(`//*[@text="your-control-text"]`).WaitGone(time.Minute)
|
||||
if !ok {
|
||||
@ -516,16 +530,16 @@ if !ok {
|
||||
return
|
||||
}
|
||||
```
|
||||
* If you want run xpath query based on ui hierachy snaphot,
|
||||
* 如果我们基于UI结构的snapshot来做Xpath查询, 这样，基于一次uiautomator的调用，我们可以执行多次Xpath查询，这在需要广告识别的场景非常有效
|
||||
```go
|
||||
content, _ := d.DumpHierarchy(false, false) // content is the text
|
||||
doc, _ := FormatHierachy(content) // doc is the *xmlquery.Node object
|
||||
content, _ := d.DumpHierarchy(false, false) // content 是文本
|
||||
doc, _ := FormatHierachy(content) // doc 是 *xmlquery.Node 对象
|
||||
...
|
||||
el := d.XPath2(`//*[@text="your-control-text"]`, doc).First()
|
||||
```
|
||||
|
||||
### Xpath elements API
|
||||
* Children of Xpath element
|
||||
* Xpath元素的子元素（子级后代）
|
||||
```go
|
||||
// el := d.XPath(`//*[@resource-id="com.taobao.taobao:id/rv_main_container"]`).First()
|
||||
children := el.Children()
|
||||
@ -533,7 +547,7 @@ for _, c := range children {
|
||||
...
|
||||
}
|
||||
```
|
||||
* Siblings of Xpath element
|
||||
* Xpath元素的兄弟元素
|
||||
```go
|
||||
// el := d.XPath(`//*[@resource-id="com.taobao.taobao:id/rv_main_container"]/android.widget.FrameLayout[1]`).First()
|
||||
siblings := el.Siblings()
|
||||
@ -541,7 +555,7 @@ for _, s := range siblings {
|
||||
...
|
||||
}
|
||||
```
|
||||
* Find descendants based on xpath
|
||||
* 通过Xpath查找Xpath元素的后代元素（它基于查找Xpath元素时获取的UI结构snapshot，所以不会涉及Uiautomator调用）
|
||||
```go
|
||||
// el := d.XPath(`//*[@resource-id="com.taobao.taobao:id/rv_main_container"]`).First()
|
||||
children := el.Find(`//android.support.v7.widget.RecyclerView`)
|
||||
@ -549,40 +563,42 @@ for _, c := range children {
|
||||
fmt.Println(*c.Info())
|
||||
}
|
||||
```
|
||||
* Find bounding rect
|
||||
This describes the bounding box surrounding this control
|
||||
* 获取 bounding rect
|
||||
|
||||
获取控件的边框，注释中解释了返回值类型
|
||||
```go
|
||||
bounds := el.Bounds()
|
||||
/* bounds has the type of *apis.Bounds:
|
||||
/* bounds 的类型为 *apis.Bounds:
|
||||
* type Bounds struct {
|
||||
* LX int // top-left-x
|
||||
* LY int // top-left-y
|
||||
* RX int // right-bottom-x
|
||||
* RY int // right-bottom-y
|
||||
* LX int // 左上角x
|
||||
* LY int // 左上角y
|
||||
* RX int // 右下角x
|
||||
* RY int // 右下角y
|
||||
* }
|
||||
*/
|
||||
```
|
||||
* Find Rect
|
||||
This also describes the bounding box surrounding this control
|
||||
* 获取 Rect
|
||||
|
||||
也是获取控件的边框，注释中解释了返回值类型
|
||||
```go
|
||||
rect := el.Rect()
|
||||
/* rect has the type of *apis.Rect:
|
||||
/* rect 的类型为 *apis.Rect:
|
||||
* type Bounds struct {
|
||||
* LX int // top-left-x
|
||||
* LY int // top-left-y
|
||||
* Width int // width of control
|
||||
* Height int // height of control
|
||||
* LX int // 左上角x
|
||||
* LY int // 左上角y
|
||||
* Width int // 控件宽度
|
||||
* Height int // 控件高度
|
||||
* }
|
||||
*/
|
||||
```
|
||||
* Get text of control - the text shown in user interface
|
||||
* 获取控件在UI中显示的文本
|
||||
```go
|
||||
text := el.Text() // text is string
|
||||
text := el.Text() // text 是文本
|
||||
```
|
||||
* Get control's info - which is everything, including text and bounding box
|
||||
* 获取控件的所有信息 - 包括文本和边框，注释中解释了返回值类型
|
||||
```go
|
||||
info := el.Info()
|
||||
/* info has the type of *apis.Info
|
||||
/* info 的类型是 *apis.Info
|
||||
* type Info struct {
|
||||
* Text string
|
||||
* Focusable bool
|
||||
@ -601,137 +617,148 @@ info := el.Info()
|
||||
* }
|
||||
*/
|
||||
```
|
||||
* Get center position
|
||||
* 获取元素中心点位置
|
||||
```go
|
||||
x, y, ok := el.Center()
|
||||
```
|
||||
* Click
|
||||
* 点击
|
||||
```go
|
||||
ok := el.Click()
|
||||
```
|
||||
* Swipe inside the control - if the control is a (Recycler)List
|
||||
* 在控件中滑动 - 如果控件是(Recycler)List
|
||||
```go
|
||||
dir := apis.SWIPE_DIR_LEFT
|
||||
var scale float32 = 0.8
|
||||
ok := el.SwipeInsideList(dir, scale)
|
||||
/* dir can use these 4 values:
|
||||
* SWIPE_DIR_LEFT = 1 // swipe right to left
|
||||
* SWIPE_DIR_RIGHT = 2 // swipe left to right
|
||||
* SWIPE_DIR_UP = 3 // swipe bottom to top
|
||||
* SWIPE_DIR_DOWN = 4 // swipe top to bottom
|
||||
/* 滑动方向dir 有四种值(int):
|
||||
* SWIPE_DIR_LEFT = 1 // 从右向左滑动
|
||||
* SWIPE_DIR_RIGHT = 2 // 从左向右滑动
|
||||
* SWIPE_DIR_UP = 3 // 从下到上滑动
|
||||
* SWIPE_DIR_DOWN = 4 // 从上到下滑动
|
||||
|
||||
scale is the percentage of width/height swiped
|
||||
scale 是滑动距离的比例，相对于在此滑动方向下，宽度或者高度。比如，再这个例子中，我们是从右到左横向滑动，那么scale = 0.8就意味着滑动80%的宽度距离
|
||||
*/
|
||||
```
|
||||
* Type text
|
||||
* 输入文字
|
||||
```go
|
||||
ok := el.Type("aaa")
|
||||
```
|
||||
* Screenshot - take screenshot of this control
|
||||
* 截图 - 控件截图
|
||||
```go
|
||||
img := el.Screenshot() // img is the image.Image type
|
||||
img := el.Screenshot() // img 是 image.Image 类型对象
|
||||
```
|
||||
|
||||
## UI Object
|
||||
Find ui elements via attribute matching search. In most platforms including iOS and windows UIA, accessibility api(UI Object) is far more efficient than xpath. For example, windows UIA, to get the full xml structure takes very long time, because fetching element's info involves cross-process COM calls which takes time. But here in Android, this is not the case. Xpath is as fast as accessibility(UI Object) and far more powerful. I would prefer suggest you to use xpath.
|
||||
通过属性匹配方式查找UI元素. 在大多数平台，包括iOS和Windows UIA, 这种典型的辅助功能API(UI Object)远比Xpath更快. 举例来说, windows UIA, 获取桌面根元素下的XML UI结构异常的慢, 因为获取元素的信息会有大量的夸进程COM调用，它是很慢的. 但是在安卓获取XML UI结构非常的快，基本和UI Object方式一样快. 而且Xpath还更强大，支持更有表现力的查询。
|
||||
|
||||
与其你使用UI Object方式查找元素，我会更建议你使用上一节的Xpath方式。我给两种方式提供了类似的元素操作API，比如Info, Click, Wait等。
|
||||
|
||||
有些API在UI Object中看起来不那么自然，比如Count，那是因为获取所有元素在UI Object可能会牵涉非常多次的Uiautomator调用。与其让用户因为一个不那么好的查询条件等太长时间，不如把底层暴露给用户，让用户自己决定实现。
|
||||
|
||||
### construct query
|
||||
UI object API doesn't effectively fetch any elements util you call (*UIObject).Get() *UiElement, (*UIObject).Wait(timeout time.Duration) int, or (*UIObject).WaitGone() bool
|
||||
UI Object API事实上在调用这几个API前是不调用Uiautomator的:
|
||||
1. (*UIObject).Get() *UiElement,
|
||||
2. (*UIObject).Wait(timeout time.Duration) int,
|
||||
3. (*UIObject).WaitGone() bool
|
||||
|
||||
* Construct query based on attribute values
|
||||
* 基于属性值构造UI Object查询
|
||||
```go
|
||||
uo := d.UiObject(apis.NewUiObjectQuery("resourceId", `com.taobao.taobao:id/rv_main_container`)) // this api accepts multiple NewUiObjectQuery in args, with and relationship
|
||||
uo := d.UiObject(apis.NewUiObjectQuery("resourceId", `com.taobao.taobao:id/rv_main_container`)) // 这个 api 接受多个 NewUiObjectQuery 参数, AND关系
|
||||
```
|
||||
* Construct sibling query - I find this api's behavior is a bit awkward. Notice that.
|
||||
* 构造兄弟元素UI Object查询 - 这个API的Uiautomator返回有点怪，稍微注意一下
|
||||
```go
|
||||
c := d.UiObject(apis.NewUiObjectQuery("resourceId", `com.taobao.taobao:id/sv_search_view`)).Sibling(apis.NewUiObjectQuery("className", "android.widget.FrameLayout"))
|
||||
```
|
||||
* Construct decendant query
|
||||
* 构造后代元素UI Object查询 - 为了和uiautomation2 python库尽可能保持API一致，我使用Child的方法名称，而不是Descendant
|
||||
```go
|
||||
c := d.UiObject(apis.NewUiObjectQuery("resourceId", `com.taobao.taobao:id/rv_main_container`)).Child(apis.NewUiObjectQuery("className", "android.widget.LinearLayout"))
|
||||
```
|
||||
* Construct indexed query
|
||||
* 构建指数UI Object查询 - 一个查询条件可能返回多个元素，当你通过Count API知道一共有多少个匹配的元素时，你可以指定小于Count值的Index（从0开始），用于指定获取哪个元素
|
||||
```go
|
||||
c := d.UiObject(
|
||||
apis.NewUiObjectQuery("className", `android.support.v7.widget.RecyclerView`)).Index(0)
|
||||
```
|
||||
|
||||
### execute ui object query
|
||||
* get first ui element
|
||||
执行UI Object查询
|
||||
* 获取第一个UI元素
|
||||
```go
|
||||
el := d.UiObject(apis.NewUiObjectQuery("resourceId", `com.taobao.taobao:id/rv_main_container`)).Get() // returns an *apis.UiElement
|
||||
el := d.UiObject(apis.NewUiObjectQuery("resourceId", `com.taobao.taobao:id/rv_main_container`)).Get() // 返回 *apis.UiElement 对象
|
||||
```
|
||||
* get element count
|
||||
* 获取匹配查询的元素数量
|
||||
```go
|
||||
count := d.UiObject(apis.NewUiObjectQuery("resourceId", `com.taobao.taobao:id/rv_main_container`)).Count()
|
||||
```
|
||||
* get the nth element - here in the example - third(with .Index(2))
|
||||
* 获取第N个元素 - 代码示例片段中 - 是第三个( .Index(2)，因为index值是从0开始的)
|
||||
```go
|
||||
el := d.UiObject(apis.NewUiObjectQuery("resourceId", `com.taobao.taobao:id/rv_main_container`)).Index(2).Get()
|
||||
```
|
||||
* wait element appear
|
||||
* 等待元素出现
|
||||
```go
|
||||
count := d.UiObject(apis.NewUiObjectQuery("resourceId", `com.taobao.taobao:id/rv_main_container`)).Wait(time.Minute)
|
||||
// if element doesn't appear in 1 minute, it returns -1
|
||||
// 返回匹配的元素的数量，如果在参数给定的时间内不出现匹配元素，返回 -1
|
||||
```
|
||||
* wait element disappear
|
||||
* 等待元素消失
|
||||
```go
|
||||
ok := d.UiObject(apis.NewUiObjectQuery("resourceId", `com.taobao.taobao:id/rv_main_container`)).WaitGone(time.Minute)
|
||||
// returns true if disappear, false otherwise
|
||||
// 如果消失，返回true，反之false
|
||||
```
|
||||
|
||||
### ui object element apis
|
||||
* Get info
|
||||
UI Object元素API
|
||||
* 获取控件的所有信息 - 包括文本和边框，注释中解释了返回值类型
|
||||
```go
|
||||
info := el.Info() // the info's type is same as xpath element's info api
|
||||
info := el.Info() // info的类型与xpath元素的同名API相同
|
||||
```
|
||||
* Get bounding rect
|
||||
* 获取 bounding rect
|
||||
|
||||
获取控件的边框，注释中解释了返回值类型
|
||||
```go
|
||||
bounds := el.Bounds() // the bounds's type is the same as xpath element's bounds api
|
||||
bounds := el.Bounds() // bounds的类型与xpath元素的同名API相同
|
||||
```
|
||||
* Get rect
|
||||
* 获取 Rect
|
||||
|
||||
也是获取控件的边框，注释中解释了返回值类型
|
||||
```go
|
||||
rect := el.Rect() // the rect's type is the same as xpath element's rects api
|
||||
rect := el.Rect() // rect的类型与xpath元素的同名API相同
|
||||
```
|
||||
* Get center position
|
||||
* 获取元素中心点位置
|
||||
```go
|
||||
x, y, ok := el.Center()
|
||||
```
|
||||
* Click
|
||||
* 点击
|
||||
```go
|
||||
ok := el.Click()
|
||||
```
|
||||
* Get text of control - the text shown in user interface
|
||||
* 获取控件在UI中显示的文本
|
||||
```go
|
||||
text := el.Text() // text is string
|
||||
text := el.Text() // text 是文本
|
||||
```
|
||||
* Swipe inside the control - if the control is a (Recycler)List
|
||||
* 在控件中滑动 - 如果控件是(Recycler)List
|
||||
```go
|
||||
dir := apis.SWIPE_DIR_LEFT
|
||||
var scale float32 = 0.8
|
||||
ok := el.SwipeInsideList(dir, scale)
|
||||
/* dir can use these 4 values:
|
||||
* SWIPE_DIR_LEFT = 1 // swipe right to left
|
||||
* SWIPE_DIR_RIGHT = 2 // swipe left to right
|
||||
* SWIPE_DIR_UP = 3 // swipe bottom to top
|
||||
* SWIPE_DIR_DOWN = 4 // swipe top to bottom
|
||||
/* 滑动方向dir 有四种值(int):
|
||||
* SWIPE_DIR_LEFT = 1 // 从右向左滑动
|
||||
* SWIPE_DIR_RIGHT = 2 // 从左向右滑动
|
||||
* SWIPE_DIR_UP = 3 // 从下到上滑动
|
||||
* SWIPE_DIR_DOWN = 4 // 从上到下滑动
|
||||
|
||||
scale is the percentage of width/height swiped
|
||||
scale 是滑动距离的比例，相对于在此滑动方向下，宽度或者高度。比如，再这个例子中，我们是从右到左横向滑动，那么scale = 0.8就意味着滑动80%的宽度距离
|
||||
*/
|
||||
```
|
||||
* Type text
|
||||
* 输入文字
|
||||
```go
|
||||
ok := el.Type("aaa")
|
||||
```
|
||||
* Screenshot - take screenshot of this control
|
||||
* 截图 - 控件截图
|
||||
```go
|
||||
img := el.Screenshot() // img is the image.Image type
|
||||
img := el.Screenshot() // img 是 image.Image 类型对象
|
||||
```
|
||||
|
||||
# If you want to support author, please donate(wechat), thanks
|
||||
|
||||
|
||||
**如果想支持作者，左边是微信打赏码；如果想和作者交朋友或者一起做好玩的编程事情，右边是微信加好友的二维码**
|
||||
|
||||
# Contact me if you'd like working with me on the computer vision & speech recognition part.
|
||||
|
||||
|
||||
|  |  |
|
||||
| -- | -- |
|
||||
739
README_en.md
Normal file
739
README_en.md
Normal file
@ -0,0 +1,739 @@
|
||||
[简体中文](./README.md) | English
|
||||
|
||||
# GO-MOBILE-AUTOMATION SDK
|
||||
|
||||
A full featured Android mobile automation sdk for golang developers
|
||||
|
||||
## The purpose
|
||||
|
||||
If you are an automation developer, you may find the python/javascript echo system provide the developers great capabilities for manipulating devices and apps.
|
||||
|
||||
For Android automation, some well known tools are:
|
||||
|
||||
1. Appium (multi-language primarily javascript/python)
|
||||
2. uiautomator2 (python)
|
||||
|
||||
Our SDK ports the uiautomation2 python library to golang.
|
||||
|
||||
Why do we do this?
|
||||
|
||||
1. Easy to deploy - deploy one executable only (at most several dlls) instead of resolving thousands of dependencies like javascript and python. The bigger scope of this family of projects is to provide a mechanism to orchestrate the automation "scripts" to run cross a bunch of platform/systems. Must have a fast & robustic app distribution and deployment mechanism.
|
||||
2. For using cloud phones (Android) - Some cloud phone providers have very poor quality adb connection. In case the automation process break because of adb connection failure, we would like the process run inside the phone
|
||||
3. Robust - apps can easily get killed by Android system, but executable not. That why despite the fact that apps like pyto-python3 provides hosting for python but we don't use it.
|
||||
|
||||
## Inspired by
|
||||
|
||||
Inspired by [OpenAtx](https://github.com/openatx) and the [Uiautomation2](https://github.com/openatx/uiautomator2) python library. We entirely use the openatx drivers, leaving the client side sdk written in golang. This benefits users, because they can use the uiautomator2 tool chain, which is super cool.
|
||||
|
||||
## Quick start
|
||||
|
||||
There are 4 steps:
|
||||
|
||||
1. Setup the Android phone
|
||||
2. Setup the development environment
|
||||
3. Start creating a golang project for mobile automation
|
||||
4. deploy&run
|
||||
|
||||
### Setup the Android phone
|
||||
|
||||
1. Install the specific version of the app you want to automate.
|
||||
```
|
||||
$ adb install [package.apk]
|
||||
```
|
||||
2. Download atx-agent from [here](https://github.com/openatx/atx-agent/releases) choose the armv7 version unless you run a x86 phone simulator
|
||||
3. Untar atx-agent and install, follow the intallation insttructions [here](https://github.com/openatx/atx-agent)
|
||||
```
|
||||
$ adb push atx-agent /data/local/tmp
|
||||
$ adb shell chmod 755 /data/local/tmp/atx-agent
|
||||
# launch atx-agent in daemon mode
|
||||
$ adb shell /data/local/tmp/atx-agent server -d
|
||||
|
||||
# stop already running atx-agent and start daemon
|
||||
$ adb shell /data/local/tmp/atx-agent server -d --stop
|
||||
```
|
||||
4. Download app-uiautomator-test.apk and app-uiautomator.apk from [here](https://github.com/openatx/android-uiautomator-server/releases) and install using adb install
|
||||
```
|
||||
$ adb install app-uiautomator-test.apk
|
||||
$ adb install app-uiautomator.apk
|
||||
```
|
||||
5. Grant all priviledges to the app "ATX"
|
||||
6. Open app "ATX" and click "启动UIAUTOMATOR", click "开启悬浮窗"
|
||||
|
||||
## Setup the development environment
|
||||
|
||||
1. Install Python3(version 3.6+) from [here](https://www.python.org/downloads/)
|
||||
2. Install [weditor](https://github.com/alibaba/web-editor)
|
||||
```
|
||||
$ pip3 install -U weditor
|
||||
```
|
||||
3. You can now open weditor as ui inspector for android applications, by typing
|
||||
```
|
||||
$ weditor
|
||||
```
|
||||
|
||||
## Create a golang automation project
|
||||
|
||||
1. Create a folder called helloworld
|
||||
2. Open terminal and type in
|
||||
```
|
||||
$ go mod init helloworld
|
||||
```
|
||||
3. Add dependency
|
||||
```
|
||||
$ go get github.com/fantonglang/go-mobile-automation
|
||||
```
|
||||
4. Add the program entry - create main.go file
|
||||
5. [Here](https://github.com/fantonglang/go-mobile-automation-examples/blob/main/douyin-luo-live/main.go) is an example of main.go file
|
||||
|
||||
## Deploy&Run
|
||||
```
|
||||
# cross build linux/arm target
|
||||
$ GOOS=linux GOARCH=arm go build
|
||||
# deploy - helloworld is the executable name, which is the same with the go module name
|
||||
$ adb push helloworld /data/local/tmp
|
||||
$ adb shell chmod 755 /data/local/tmp/helloworld
|
||||
# run
|
||||
$ adb shell /data/local/tmp/helloworld
|
||||
```
|
||||
If you start a background process, you don't need the phone to connect with your PC/macos. Note that you can transfer your linux shell knowledge to using the adb shell.
|
||||
|
||||
## Examples
|
||||
[This](https://github.com/fantonglang/go-mobile-automation-examples/blob/main/douyin-luo-live) is a working example. Read the comments in main.go carefully. This helps to resolve all dependencies and environment requirements before you start. The comments also give the commands for compilation, deployment, and execution.
|
||||
|
||||
# APIS
|
||||
|
||||
**[Connect to a device](#connect-to-a-device)**
|
||||
|
||||
**[Device APIS](#device-apis)**
|
||||
- **[Shell commands](#shell-commands)**
|
||||
- **[Retrieve the device info](#retrieve-the-device-info)**
|
||||
- **[Clipboard](#clipboard)**
|
||||
- **[Key Events](#key-events)**
|
||||
- **[Press Key](#press-key)**
|
||||
- **[New command timeout](#new-command-timeout)**
|
||||
- **[Screenshot](#screenshot)**
|
||||
- **[UI Hierarchy](#ui-hierarchy)**
|
||||
- **[Touch](#touch)**
|
||||
- **[Click](#click)**
|
||||
- **[Double Click](#double-click)**
|
||||
- **[Long Click](#long-click)**
|
||||
- **[Swipe](#swipe)**
|
||||
- **[Set Orientation](#set-orientation)**
|
||||
- **[Open Quick Settings](#open-quick-settings)**
|
||||
- **[Open Url](#open-url)**
|
||||
- **[Show float window](#show-float-window)**
|
||||
|
||||
**[Input Method](#input-method)**
|
||||
|
||||
**[XPATH](#xpath)**
|
||||
- **[Finding elements](#finding-elements)**
|
||||
- **[Xpath elements API](#xpath-elements-api)**
|
||||
|
||||
**[UI Object](#ui-object)**
|
||||
- **[construct query](#construct-query)**
|
||||
- **[execute ui object query](#execute-ui-object-query)**
|
||||
- **[ui object element apis](#ui-object-element-apis)**
|
||||
|
||||
## Connect to a device
|
||||
|
||||
There are two types of connection:
|
||||
|
||||
1. If the executable is deployed in the phone, use
|
||||
``` go
|
||||
package main
|
||||
|
||||
import (
|
||||
"log"
|
||||
"github.com/fantonglang/go-mobile-automation/apis"
|
||||
)
|
||||
...
|
||||
// you don't need to specify device id, because there is no PC connection
|
||||
d := apis.NewNativeDevice()
|
||||
```
|
||||
2. If you debug and deploy in PC/macos, use
|
||||
``` go
|
||||
package main
|
||||
|
||||
import (
|
||||
"log"
|
||||
"github.com/fantonglang/go-mobile-automation/apis"
|
||||
)
|
||||
|
||||
|
||||
//here c574dd45 is the device id, replace it with yours own
|
||||
d, err := apis.NewHostDevice("c574dd45")
|
||||
if err != nil {
|
||||
log.Println("failed connecting to device")
|
||||
return
|
||||
}
|
||||
```
|
||||
|
||||
Combine 2 code snippets. The following code enables the same piece of code working on the both deployments.
|
||||
``` go
|
||||
package main
|
||||
|
||||
import (
|
||||
"log"
|
||||
"runtime"
|
||||
"github.com/fantonglang/go-mobile-automation/apis"
|
||||
)
|
||||
|
||||
func getDevice() *apis.Device {
|
||||
if runtime.GOARCH == "arm" {
|
||||
return apis.NewNativeDevice()
|
||||
}
|
||||
//here c574dd45 is the device id, replace it with yours own
|
||||
_d, err := apis.NewHostDevice("c574dd45")
|
||||
if err != nil {
|
||||
log.Println("101: failed connecting to device")
|
||||
return nil
|
||||
}
|
||||
return _d
|
||||
}
|
||||
...
|
||||
d := getDevice()
|
||||
```
|
||||
|
||||
Take extra notice if your macos is the ARM architecture. Then you may judge based on GOOS.
|
||||
|
||||
## Device APIS
|
||||
This part showcases how to perform common device operations:
|
||||
|
||||
### Shell commands
|
||||
Example: Force stop douyin(China tiktok) app
|
||||
```go
|
||||
d.Shell(`am force-stop com.ss.android.ugc.aweme`)
|
||||
```
|
||||
Example: Start douyin app
|
||||
```go
|
||||
// You can find the app main activity by using the dumpsys command, hence I didn't implement the uiautomator2 equivalent session API for now.
|
||||
d.Shell(`am start -n "com.ss.android.ugc.aweme/.main.MainActivity"`)
|
||||
```
|
||||
|
||||
### Retrieve the device info
|
||||
Get detailed device info
|
||||
```go
|
||||
info, err := d.DeviceInfo()
|
||||
if err != nil {
|
||||
log.Println("get device info failed")
|
||||
return
|
||||
}
|
||||
bytes, err := json.Marshal(info)
|
||||
if err != nil {
|
||||
log.Println("error marshalling")
|
||||
return
|
||||
}
|
||||
fmt.Println(string(bytes))
|
||||
```
|
||||
|
||||
Below is a possible output:
|
||||
|
||||
```json
|
||||
{
|
||||
...
|
||||
"version":"11",
|
||||
"serial":"c574dd45",
|
||||
...
|
||||
"sdk":30,
|
||||
"agentVersion":"0.10.0",
|
||||
"display":{"width":1080,"height":2340}
|
||||
...
|
||||
}
|
||||
```
|
||||
|
||||
Get window size:
|
||||
|
||||
```GO
|
||||
w, h, err := d.WindowSize()
|
||||
if err != nil {
|
||||
log.Println("get window size failed")
|
||||
return
|
||||
}
|
||||
fmt.Printf("w: %d, h: %d\n", w, h)
|
||||
// device upright output example: w: 1080, h: 2340
|
||||
// device horizontal output example: w: 1080, h: 2340
|
||||
```
|
||||
|
||||
### Clipboard
|
||||
Get or set clipboard content
|
||||
|
||||
set clipboard
|
||||
```go
|
||||
err := d.SetClipboard("aaa")
|
||||
if err != nil {
|
||||
log.Println("error clipboard")
|
||||
return
|
||||
}
|
||||
```
|
||||
get clipboard: This doesn't work in Android > 9.0. Most cloud phone work on lower Android version. I don't mind.
|
||||
```go
|
||||
a, err := d.GetClipboard()
|
||||
if err != nil {
|
||||
log.Println("error clipboard")
|
||||
return
|
||||
}
|
||||
fmt.Println(a)
|
||||
```
|
||||
|
||||
### Key Events
|
||||
|
||||
* Turn on/off screen
|
||||
```go
|
||||
err := d.KeyEvent(KEYCODE_POWER) // press power key to turn on/off screen
|
||||
```
|
||||
* Home key
|
||||
```go
|
||||
err := d.KeyEvent(KEYCODE_HOME)
|
||||
```
|
||||
|
||||
d.KeyEvent is basically the Android "input keyevent " command, please refer to [this doc](https://developer.android.com/reference/android/view/KeyEvent), or if you're behind gfw, [this doc](https://blog.csdn.net/feizhixuan46789/article/details/16801429)
|
||||
|
||||
### Press Key
|
||||
Example: press Home key
|
||||
```go
|
||||
err := d.Press("home")
|
||||
```
|
||||
|
||||
supported keys are:
|
||||
```go
|
||||
VSK_HOME = "home"
|
||||
VSK_BACK = "back"
|
||||
VSK_LEFT = "left"
|
||||
VSK_RIGHT = "right"
|
||||
VSK_UP = "up"
|
||||
VSK_DOWN = "down"
|
||||
VSK_CENTER = "center"
|
||||
VSK_MENU = "menu"
|
||||
VSK_SEARCH = "search"
|
||||
VSK_ENTER = "enter"
|
||||
VSK_DELETE = "delete"
|
||||
VSK_DEL = "del"
|
||||
VSK_RECENT = "recent" //recent apps
|
||||
VSK_VOLUME_UP = "volume_up"
|
||||
VSK_VOLUME_DOWN = "volume_down"
|
||||
VSK_VOLUME_MUTE = "volume_mute"
|
||||
VSK_CAMERA = "camera"
|
||||
VSK_POWER = "power"
|
||||
```
|
||||
|
||||
### New command timeout
|
||||
How long (in seconds) will wait for a new command from the client before assuming the client quit and ending the uiautomator service
|
||||
|
||||
```go
|
||||
err := d.SetNewCommandTimeout(300) // unit is second
|
||||
```
|
||||
|
||||
### Screenshot
|
||||
* Screenshot and save - notice Android has readonly file system, this API is only available for host(PC/macos).
|
||||
```go
|
||||
err := d.ScreenshotSave("sc.png")
|
||||
```
|
||||
* Screenshot and get bytes (preferred, because opencv can accept bytes directly by "cv::imdecode" function)
|
||||
```go
|
||||
bytes, err := d.ScreenshotBytes()
|
||||
```
|
||||
* Screenshot and get image.Image object
|
||||
```go
|
||||
img, format, err := d.Screenshot() // img is the image.Image object, example of format: "jpeg"
|
||||
```
|
||||
|
||||
### UI Hierarchy
|
||||
* Get hierachy text
|
||||
```go
|
||||
content, err := d.DumpHierarchy(false, false) // content is the text
|
||||
```
|
||||
* Transform hierachy text to *xmlquery.Node object. (This is useful if you want to do xpath query based on a snapshot - this is a lot faster)
|
||||
```go
|
||||
doc, err := FormatHierachy(content) // doc is the *xmlquery.Node object
|
||||
```
|
||||
|
||||
### Touch
|
||||
Simulate "mouse press down", "mouse hold and move", "mouse up release"
|
||||
* Get touch object
|
||||
```go
|
||||
touch := d.Touch()
|
||||
```
|
||||
* Mouse press down at a position
|
||||
```go
|
||||
/* press at (relative to the top-left corner)
|
||||
* x: 50% position of the width
|
||||
* y: 60% position of the height
|
||||
*/
|
||||
err := touch.Down(0.5, 0.6)
|
||||
```
|
||||
* Mouse hold and move
|
||||
```go
|
||||
/* then move to (relative to the top-left corner)
|
||||
* x: 50% position of the width
|
||||
* y: 10% position of the height
|
||||
*/
|
||||
err := touch.Move(0.5, 0.1)
|
||||
```
|
||||
* Mouse up release
|
||||
```go
|
||||
/* then mouse up release at (relative to the top-left corner)
|
||||
* x: 50% position of the width
|
||||
* y: 10% position of the height
|
||||
*/
|
||||
err := touch.Up(0.5, 0.1)
|
||||
```
|
||||
|
||||
### Click
|
||||
Click on screen given coordinates
|
||||
|
||||
* coordinates using percentage
|
||||
```go
|
||||
/* click relative to the top-left corner
|
||||
* x: 48.1% position of the width
|
||||
* y: 24.6% position of the height
|
||||
*/
|
||||
err := d.Click(0.481, 0.246)
|
||||
```
|
||||
* coordinates using absolute pixel values
|
||||
```go
|
||||
/* click relative to the top-left corner
|
||||
* at (x: 481, y: 246)
|
||||
*/
|
||||
err := d.Click(481, 246)
|
||||
```
|
||||
|
||||
### Double Click
|
||||
```go
|
||||
err := d.DoubleClickDefault(0.481, 0.246)
|
||||
```
|
||||
|
||||
### Long Click
|
||||
Mouse click, but there is a certain time interval(0.5s) between mouse down and up
|
||||
```go
|
||||
err := d.LongClickDefault(0.481, 0.246)
|
||||
```
|
||||
|
||||
### Swipe
|
||||
* Swipe from one point (fx, fy) to another (tx, ty)
|
||||
```go
|
||||
var fx, fy, tx, ty float32 = 0.5, 0.5, 0, 0
|
||||
err := d.SwipeDefault(fx, fy, tx, ty)
|
||||
```
|
||||
* Swipe points, you can specify more than 2 points
|
||||
```go
|
||||
// swipe from (x=width*0.5, y=height*0.9) to (x=width*0.5,y=height*0.1)
|
||||
err := d.SwipePoints(0.1, apis.Point4Swipe{0.5, 0.9}, apis.Point4Swipe{0.5, 0.1})
|
||||
```
|
||||
* Drag from one point (fx, fy) to another (tx, ty)
|
||||
```go
|
||||
var fx, fy, tx, ty float32 = 0.5, 0.5, 0, 0
|
||||
err := d.DragDefault(fx, fy, tx, ty)
|
||||
```
|
||||
|
||||
### Set Orientation
|
||||
Accepts 4 orientation parameters:
|
||||
* "n" - means natural
|
||||
* "l" - means left
|
||||
* "u" - means upsidedown
|
||||
* "r" - means right
|
||||
```go
|
||||
err := d.SetOrientation("n")
|
||||
```
|
||||
|
||||
### Open Quick Settings
|
||||
```go
|
||||
err := d.OpenQuickSettings()
|
||||
```
|
||||
|
||||
### Open Url
|
||||
```go
|
||||
err := d.OpenUrl("https://bing.com")
|
||||
```
|
||||
|
||||
### Show float window
|
||||
This operation is openatx specific - open a float window to keep the automator app in the front and prevent it from getting killed
|
||||
```go
|
||||
err := d.ShowFloatWindow(true)
|
||||
```
|
||||
|
||||
|
||||
## Input Method
|
||||
Type text, (you will switch to a special input method)
|
||||
* Clear Text
|
||||
```go
|
||||
err := d.ClearText()
|
||||
```
|
||||
* Send Action
|
||||
```go
|
||||
err := ime.SendAction(SENDACTION_SEARCH)
|
||||
```
|
||||
|
||||
The following actions are supported:
|
||||
```go
|
||||
SENDACTION_GO = 2
|
||||
SENDACTION_SEARCH = 3
|
||||
SENDACTION_SEND = 4
|
||||
SENDACTION_NEXT = 5
|
||||
SENDACTION_DONE = 6
|
||||
SENDACTION_PREVIOUS = 7
|
||||
```
|
||||
|
||||
* Send Keys - Type text
|
||||
```go
|
||||
err := ime.SendKeys("aaa", true)
|
||||
```
|
||||
|
||||
## XPATH
|
||||
XPATH is the most important way of finding UI Element.
|
||||
|
||||
### Finding elements
|
||||
* Find multiple elements by xpath
|
||||
```go
|
||||
els := d.XPath(`//*[@text="your-control-text"]`).All()
|
||||
for _, el := range els {
|
||||
...
|
||||
}
|
||||
```
|
||||
* Find one element by xpath
|
||||
```go
|
||||
el := d.XPath(`//*[@text="your-control-text"]`).First()
|
||||
```
|
||||
* Check element exists by xpath
|
||||
```go
|
||||
if d.XPath(`//*[@text="your-control-text"]`).First() != nil {
|
||||
...
|
||||
}
|
||||
```
|
||||
* Wait element appear
|
||||
```go
|
||||
el := d.XPath(`//*[@text="your-control-text"]`).Wait(time.Minute)
|
||||
if el == nil {
|
||||
log.Println("element doesn't appear within 1 minute")
|
||||
return
|
||||
}
|
||||
...
|
||||
```
|
||||
* Wait element disappear
|
||||
```go
|
||||
ok := d.XPath(`//*[@text="your-control-text"]`).WaitGone(time.Minute)
|
||||
if !ok {
|
||||
log.Println("element doesn't disappear within 1 minute")
|
||||
return
|
||||
}
|
||||
```
|
||||
* If you want run xpath query based on ui hierachy snaphot,
|
||||
```go
|
||||
content, _ := d.DumpHierarchy(false, false) // content is the text
|
||||
doc, _ := FormatHierachy(content) // doc is the *xmlquery.Node object
|
||||
...
|
||||
el := d.XPath2(`//*[@text="your-control-text"]`, doc).First()
|
||||
```
|
||||
|
||||
### Xpath elements API
|
||||
* Children of Xpath element
|
||||
```go
|
||||
// el := d.XPath(`//*[@resource-id="com.taobao.taobao:id/rv_main_container"]`).First()
|
||||
children := el.Children()
|
||||
for _, c := range children {
|
||||
...
|
||||
}
|
||||
```
|
||||
* Siblings of Xpath element
|
||||
```go
|
||||
// el := d.XPath(`//*[@resource-id="com.taobao.taobao:id/rv_main_container"]/android.widget.FrameLayout[1]`).First()
|
||||
siblings := el.Siblings()
|
||||
for _, s := range siblings {
|
||||
...
|
||||
}
|
||||
```
|
||||
* Find descendants based on xpath
|
||||
```go
|
||||
// el := d.XPath(`//*[@resource-id="com.taobao.taobao:id/rv_main_container"]`).First()
|
||||
children := el.Find(`//android.support.v7.widget.RecyclerView`)
|
||||
for _, c := range children {
|
||||
fmt.Println(*c.Info())
|
||||
}
|
||||
```
|
||||
* Find bounding rect
|
||||
|
||||
This describes the bounding box surrounding this control
|
||||
```go
|
||||
bounds := el.Bounds()
|
||||
/* bounds has the type of *apis.Bounds:
|
||||
* type Bounds struct {
|
||||
* LX int // top-left-x
|
||||
* LY int // top-left-y
|
||||
* RX int // right-bottom-x
|
||||
* RY int // right-bottom-y
|
||||
* }
|
||||
*/
|
||||
```
|
||||
* Find Rect
|
||||
|
||||
This also describes the bounding box surrounding this control
|
||||
```go
|
||||
rect := el.Rect()
|
||||
/* rect has the type of *apis.Rect:
|
||||
* type Bounds struct {
|
||||
* LX int // top-left-x
|
||||
* LY int // top-left-y
|
||||
* Width int // width of control
|
||||
* Height int // height of control
|
||||
* }
|
||||
*/
|
||||
```
|
||||
* Get text of control - the text shown in user interface
|
||||
```go
|
||||
text := el.Text() // text is string
|
||||
```
|
||||
* Get control's info - which is everything, including text and bounding box
|
||||
```go
|
||||
info := el.Info()
|
||||
/* info has the type of *apis.Info
|
||||
* type Info struct {
|
||||
* Text string
|
||||
* Focusable bool
|
||||
* Enabled bool
|
||||
* Focused bool
|
||||
* Scrollable bool
|
||||
* Selected bool
|
||||
* ClassName string
|
||||
* Bounds *Bounds
|
||||
* ContentDescription string
|
||||
* LongClickable bool
|
||||
* PackageName string
|
||||
* ResourceName string
|
||||
* ResourceId string
|
||||
* ChildCount int
|
||||
* }
|
||||
*/
|
||||
```
|
||||
* Get center position
|
||||
```go
|
||||
x, y, ok := el.Center()
|
||||
```
|
||||
* Click
|
||||
```go
|
||||
ok := el.Click()
|
||||
```
|
||||
* Swipe inside the control - if the control is a (Recycler)List
|
||||
```go
|
||||
dir := apis.SWIPE_DIR_LEFT
|
||||
var scale float32 = 0.8
|
||||
ok := el.SwipeInsideList(dir, scale)
|
||||
/* dir can use these 4 values:
|
||||
* SWIPE_DIR_LEFT = 1 // swipe right to left
|
||||
* SWIPE_DIR_RIGHT = 2 // swipe left to right
|
||||
* SWIPE_DIR_UP = 3 // swipe bottom to top
|
||||
* SWIPE_DIR_DOWN = 4 // swipe top to bottom
|
||||
|
||||
scale is the percentage of width/height swiped
|
||||
*/
|
||||
```
|
||||
* Type text
|
||||
```go
|
||||
ok := el.Type("aaa")
|
||||
```
|
||||
* Screenshot - take screenshot of this control
|
||||
```go
|
||||
img := el.Screenshot() // img is the image.Image type
|
||||
```
|
||||
|
||||
## UI Object
|
||||
Find ui elements via attribute matching search. In most platforms including iOS and windows UIA, accessibility api(UI Object) is far more efficient than xpath. For example, windows UIA, to get the full xml structure takes very long time, because fetching element's info involves cross-process COM calls which takes time. But here in Android, this is not the case. Xpath is as fast as accessibility(UI Object) and far more powerful. I would prefer suggest you to use xpath.
|
||||
|
||||
### construct query
|
||||
UI object API doesn't effectively fetch any elements util you call (*UIObject).Get() *UiElement, (*UIObject).Wait(timeout time.Duration) int, or (*UIObject).WaitGone() bool
|
||||
|
||||
* Construct query based on attribute values
|
||||
```go
|
||||
uo := d.UiObject(apis.NewUiObjectQuery("resourceId", `com.taobao.taobao:id/rv_main_container`)) // this api accepts multiple NewUiObjectQuery in args, with and relationship
|
||||
```
|
||||
* Construct sibling query - I find this api's behavior is a bit awkward. Notice that.
|
||||
```go
|
||||
c := d.UiObject(apis.NewUiObjectQuery("resourceId", `com.taobao.taobao:id/sv_search_view`)).Sibling(apis.NewUiObjectQuery("className", "android.widget.FrameLayout"))
|
||||
```
|
||||
* Construct decendant query
|
||||
```go
|
||||
c := d.UiObject(apis.NewUiObjectQuery("resourceId", `com.taobao.taobao:id/rv_main_container`)).Child(apis.NewUiObjectQuery("className", "android.widget.LinearLayout"))
|
||||
```
|
||||
* Construct indexed query
|
||||
```go
|
||||
c := d.UiObject(
|
||||
apis.NewUiObjectQuery("className", `android.support.v7.widget.RecyclerView`)).Index(0)
|
||||
```
|
||||
|
||||
### execute ui object query
|
||||
* get first ui element
|
||||
```go
|
||||
el := d.UiObject(apis.NewUiObjectQuery("resourceId", `com.taobao.taobao:id/rv_main_container`)).Get() // returns an *apis.UiElement
|
||||
```
|
||||
* get element count
|
||||
```go
|
||||
count := d.UiObject(apis.NewUiObjectQuery("resourceId", `com.taobao.taobao:id/rv_main_container`)).Count()
|
||||
```
|
||||
* get the nth element - here in the example - third(with .Index(2))
|
||||
```go
|
||||
el := d.UiObject(apis.NewUiObjectQuery("resourceId", `com.taobao.taobao:id/rv_main_container`)).Index(2).Get()
|
||||
```
|
||||
* wait element appear
|
||||
```go
|
||||
count := d.UiObject(apis.NewUiObjectQuery("resourceId", `com.taobao.taobao:id/rv_main_container`)).Wait(time.Minute)
|
||||
// if element doesn't appear in 1 minute, it returns -1
|
||||
```
|
||||
* wait element disappear
|
||||
```go
|
||||
ok := d.UiObject(apis.NewUiObjectQuery("resourceId", `com.taobao.taobao:id/rv_main_container`)).WaitGone(time.Minute)
|
||||
// returns true if disappear, false otherwise
|
||||
```
|
||||
|
||||
### ui object element apis
|
||||
* Get info
|
||||
```go
|
||||
info := el.Info() // the info's type is same as xpath element's info api
|
||||
```
|
||||
* Get bounding rect
|
||||
```go
|
||||
bounds := el.Bounds() // the bounds's type is the same as xpath element's bounds api
|
||||
```
|
||||
* Get rect
|
||||
```go
|
||||
rect := el.Rect() // the rect's type is the same as xpath element's rects api
|
||||
```
|
||||
* Get center position
|
||||
```go
|
||||
x, y, ok := el.Center()
|
||||
```
|
||||
* Click
|
||||
```go
|
||||
ok := el.Click()
|
||||
```
|
||||
* Get text of control - the text shown in user interface
|
||||
```go
|
||||
text := el.Text() // text is string
|
||||
```
|
||||
* Swipe inside the control - if the control is a (Recycler)List
|
||||
```go
|
||||
dir := apis.SWIPE_DIR_LEFT
|
||||
var scale float32 = 0.8
|
||||
ok := el.SwipeInsideList(dir, scale)
|
||||
/* dir can use these 4 values:
|
||||
* SWIPE_DIR_LEFT = 1 // swipe right to left
|
||||
* SWIPE_DIR_RIGHT = 2 // swipe left to right
|
||||
* SWIPE_DIR_UP = 3 // swipe bottom to top
|
||||
* SWIPE_DIR_DOWN = 4 // swipe top to bottom
|
||||
|
||||
scale is the percentage of width/height swiped
|
||||
*/
|
||||
```
|
||||
* Type text
|
||||
```go
|
||||
ok := el.Type("aaa")
|
||||
```
|
||||
* Screenshot - take screenshot of this control
|
||||
```go
|
||||
img := el.Screenshot() // img is the image.Image type
|
||||
```
|
||||
|
||||
# If you want to support author, please donate(wechat), thanks
|
||||
|
||||
|
||||
|
||||
# Contact me if you'd like working with me on the computer vision & speech recognition part.
|
||||
|
||||
|
||||
Loading…
Reference in New Issue
Block a user