1*cfb92d14SAndroid Build Coastguard Worker# `toranj-cli` 2*cfb92d14SAndroid Build Coastguard Worker 3*cfb92d14SAndroid Build Coastguard Worker`toranj-cli` is a test framework for OpenThread using its CLI interface. 4*cfb92d14SAndroid Build Coastguard Worker 5*cfb92d14SAndroid Build Coastguard Worker`toranj` features: 6*cfb92d14SAndroid Build Coastguard Worker 7*cfb92d14SAndroid Build Coastguard Worker- It is developed in Python. 8*cfb92d14SAndroid Build Coastguard Worker- It can be used to simulate multiple nodes forming complex network topologies. 9*cfb92d14SAndroid Build Coastguard Worker- It allows testing of network interactions between many nodes. 10*cfb92d14SAndroid Build Coastguard Worker- `toranj` in CLI mode runs `ot-cli-ftd` on simulation platform (real-time). 11*cfb92d14SAndroid Build Coastguard Worker 12*cfb92d14SAndroid Build Coastguard Worker## Setup 13*cfb92d14SAndroid Build Coastguard Worker 14*cfb92d14SAndroid Build Coastguard WorkerTo build OpenThread with `toranj` configuration, the `test/toranj/build.sh` script can be used: 15*cfb92d14SAndroid Build Coastguard Worker 16*cfb92d14SAndroid Build Coastguard Worker```bash 17*cfb92d14SAndroid Build Coastguard Worker$ ./tests/toranj/build.sh all 18*cfb92d14SAndroid Build Coastguard Worker==================================================================================================== 19*cfb92d14SAndroid Build Coastguard WorkerBuilding OpenThread (NCP/CLI for FTD/MTD/RCP mode) with simulation platform using cmake 20*cfb92d14SAndroid Build Coastguard Worker==================================================================================================== 21*cfb92d14SAndroid Build Coastguard Worker-- OpenThread Source Directory: /Users/abtink/GitHub/openthread 22*cfb92d14SAndroid Build Coastguard Worker-- OpenThread CMake build type: Debug 23*cfb92d14SAndroid Build Coastguard Worker-- Package Name: OPENTHREAD 24*cfb92d14SAndroid Build Coastguard Worker... 25*cfb92d14SAndroid Build Coastguard Worker 26*cfb92d14SAndroid Build Coastguard Worker``` 27*cfb92d14SAndroid Build Coastguard Worker 28*cfb92d14SAndroid Build Coastguard WorkerThe `toranj-cli` tests are included in `tests/toranj/cli` folder. Each test-case has its own script following naming model `test-nnn-name.py` (e.g., `test-001-get-set.py`). 29*cfb92d14SAndroid Build Coastguard Worker 30*cfb92d14SAndroid Build Coastguard WorkerTo run a specific test: 31*cfb92d14SAndroid Build Coastguard Worker 32*cfb92d14SAndroid Build Coastguard Worker```bash 33*cfb92d14SAndroid Build Coastguard Worker$ cd tests/toranj/cli 34*cfb92d14SAndroid Build Coastguard Worker$ python3 test-001-get-set.py 35*cfb92d14SAndroid Build Coastguard Worker``` 36*cfb92d14SAndroid Build Coastguard Worker 37*cfb92d14SAndroid Build Coastguard WorkerTo run all CLI tests, `start` script can be used. This script will build OpenThread with proper configuration options and starts running all tests. 38*cfb92d14SAndroid Build Coastguard Worker 39*cfb92d14SAndroid Build Coastguard Worker```bash 40*cfb92d14SAndroid Build Coastguard Worker# From OpenThread repo root folder 41*cfb92d14SAndroid Build Coastguard Worker$ top_builddir=($pwd) TORANJ_CLI=1 ./tests/toranj/start.sh 42*cfb92d14SAndroid Build Coastguard Worker``` 43*cfb92d14SAndroid Build Coastguard Worker 44*cfb92d14SAndroid Build Coastguard Worker## `toranj-cli` Components 45*cfb92d14SAndroid Build Coastguard Worker 46*cfb92d14SAndroid Build Coastguard Worker`cli` python module defines the `toranj-cli` test components. 47*cfb92d14SAndroid Build Coastguard Worker 48*cfb92d14SAndroid Build Coastguard Worker### `cli.Node()` Class 49*cfb92d14SAndroid Build Coastguard Worker 50*cfb92d14SAndroid Build Coastguard Worker`cli.Node()` class creates a Thread node instance. It creates a sub-process to run `ot-cli-ftd` and provides methods to control the node and issue CLI commands. 51*cfb92d14SAndroid Build Coastguard Worker 52*cfb92d14SAndroid Build Coastguard Worker```python 53*cfb92d14SAndroid Build Coastguard Worker>>> import cli 54*cfb92d14SAndroid Build Coastguard Worker>>> node1 = cli.Node() 55*cfb92d14SAndroid Build Coastguard Worker>>> node1 56*cfb92d14SAndroid Build Coastguard WorkerNode (index=1) 57*cfb92d14SAndroid Build Coastguard Worker>>> node2 = cli.Node() 58*cfb92d14SAndroid Build Coastguard Worker>>> node2 59*cfb92d14SAndroid Build Coastguard WorkerNode (index=2) 60*cfb92d14SAndroid Build Coastguard Worker``` 61*cfb92d14SAndroid Build Coastguard Worker 62*cfb92d14SAndroid Build Coastguard WorkerNote: You may need to run as `sudo` to allow log file to be written (i.e., use `sudo python` or `sudo python3`). 63*cfb92d14SAndroid Build Coastguard Worker 64*cfb92d14SAndroid Build Coastguard Worker### `cli.Node` methods 65*cfb92d14SAndroid Build Coastguard Worker 66*cfb92d14SAndroid Build Coastguard Worker`cli.Node()` provides methods matching different CLI commands, in addition to some helper methods for common operations. 67*cfb92d14SAndroid Build Coastguard Worker 68*cfb92d14SAndroid Build Coastguard WorkerExample: 69*cfb92d14SAndroid Build Coastguard Worker 70*cfb92d14SAndroid Build Coastguard Worker```python 71*cfb92d14SAndroid Build Coastguard Worker>>> node.get_state() 72*cfb92d14SAndroid Build Coastguard Worker'disabled' 73*cfb92d14SAndroid Build Coastguard Worker>>> node.get_channel() 74*cfb92d14SAndroid Build Coastguard Worker'11' 75*cfb92d14SAndroid Build Coastguard Worker>>> node.set_channel(12) 76*cfb92d14SAndroid Build Coastguard Worker>>> node.get_channel() 77*cfb92d14SAndroid Build Coastguard Worker'12' 78*cfb92d14SAndroid Build Coastguard Worker>>> node.set_network_key('11223344556677889900aabbccddeeff') 79*cfb92d14SAndroid Build Coastguard Worker>>> node.get_network_key() 80*cfb92d14SAndroid Build Coastguard Worker'11223344556677889900aabbccddeeff' 81*cfb92d14SAndroid Build Coastguard Worker``` 82*cfb92d14SAndroid Build Coastguard Worker 83*cfb92d14SAndroid Build Coastguard WorkerCommon network operations: 84*cfb92d14SAndroid Build Coastguard Worker 85*cfb92d14SAndroid Build Coastguard Worker```python 86*cfb92d14SAndroid Build Coastguard Worker # Form a Thread network with all the given parameters. 87*cfb92d14SAndroid Build Coastguard Worker node.form(network_name=None, network_key=None, channel=None, panid=0x1234, xpanid=None): 88*cfb92d14SAndroid Build Coastguard Worker 89*cfb92d14SAndroid Build Coastguard Worker # Try to join an existing network as specified by `another_node`. 90*cfb92d14SAndroid Build Coastguard Worker # `type` can be `JOIN_TYPE_ROUTER`, `JOIN_TYPE_END_DEVICE, or `JOIN_TYPE_SLEEPY_END_DEVICE` 91*cfb92d14SAndroid Build Coastguard Worker node.join(another_node, type=JOIN_TYPE_ROUTER): 92*cfb92d14SAndroid Build Coastguard Worker``` 93*cfb92d14SAndroid Build Coastguard Worker 94*cfb92d14SAndroid Build Coastguard WorkerA direct CLI command can be issued using `node.cli(command)` with a given `command` string. 95*cfb92d14SAndroid Build Coastguard Worker 96*cfb92d14SAndroid Build Coastguard Worker```python 97*cfb92d14SAndroid Build Coastguard Worker>>> node.cli('uptime') 98*cfb92d14SAndroid Build Coastguard Worker['00:36:18.778'] 99*cfb92d14SAndroid Build Coastguard Worker``` 100*cfb92d14SAndroid Build Coastguard Worker 101*cfb92d14SAndroid Build Coastguard WorkerMethod `allowlist_node()` can be used to add a given node to the allowlist of the device and enables allowlisting: 102*cfb92d14SAndroid Build Coastguard Worker 103*cfb92d14SAndroid Build Coastguard Worker```python 104*cfb92d14SAndroid Build Coastguard Worker # `node2` is added to the allowlist of `node1` and allowlisting is enabled on `node1` 105*cfb92d14SAndroid Build Coastguard Worker node1.allowlist_node(node2) 106*cfb92d14SAndroid Build Coastguard Worker``` 107*cfb92d14SAndroid Build Coastguard Worker 108*cfb92d14SAndroid Build Coastguard Worker#### Example (simple 3-node topology) 109*cfb92d14SAndroid Build Coastguard Worker 110*cfb92d14SAndroid Build Coastguard WorkerScript below shows how to create a 3-node network topology with `node1` and `node2` being routers, and `node3` an end-device connected to `node2`: 111*cfb92d14SAndroid Build Coastguard Worker 112*cfb92d14SAndroid Build Coastguard Worker```python 113*cfb92d14SAndroid Build Coastguard Worker>>> import cli 114*cfb92d14SAndroid Build Coastguard Worker>>> node1 = cli.Node() 115*cfb92d14SAndroid Build Coastguard Worker>>> node2 = cli.Node() 116*cfb92d14SAndroid Build Coastguard Worker>>> node3 = cli.Node() 117*cfb92d14SAndroid Build Coastguard Worker 118*cfb92d14SAndroid Build Coastguard Worker>>> node1.form('test') 119*cfb92d14SAndroid Build Coastguard Worker>>> node1.get_state() 120*cfb92d14SAndroid Build Coastguard Worker'leader' 121*cfb92d14SAndroid Build Coastguard Worker 122*cfb92d14SAndroid Build Coastguard Worker>>> node1.allowlist_node(node2) 123*cfb92d14SAndroid Build Coastguard Worker>>> node1.allowlist_node(node3) 124*cfb92d14SAndroid Build Coastguard Worker 125*cfb92d14SAndroid Build Coastguard Worker>>> node2.join(node1, cli.JOIN_TYPE_ROUTER) 126*cfb92d14SAndroid Build Coastguard Worker>>> node2.get_state() 127*cfb92d14SAndroid Build Coastguard Worker'router' 128*cfb92d14SAndroid Build Coastguard Worker 129*cfb92d14SAndroid Build Coastguard Worker>>> node3.join(node1, cli.JOIN_TYPE_END_DEVICE) 130*cfb92d14SAndroid Build Coastguard Worker>>> node3.get_state() 131*cfb92d14SAndroid Build Coastguard Worker'child' 132*cfb92d14SAndroid Build Coastguard Worker 133*cfb92d14SAndroid Build Coastguard Worker>>> node1.cli('neighbor list') 134*cfb92d14SAndroid Build Coastguard Worker['0x1c01 0x0400 '] 135*cfb92d14SAndroid Build Coastguard Worker``` 136*cfb92d14SAndroid Build Coastguard Worker 137*cfb92d14SAndroid Build Coastguard Worker### Logs and Verbose mode 138*cfb92d14SAndroid Build Coastguard Worker 139*cfb92d14SAndroid Build Coastguard WorkerEvery `cli.Node()` instance will save its corresponding logs. By default the logs are saved in a file `ot-logs<node_index>.log`. 140*cfb92d14SAndroid Build Coastguard Worker 141*cfb92d14SAndroid Build Coastguard WorkerWhen `start.sh` script is used to run all test-cases, if any test fails, to help with debugging of the issue, the last 30 lines of logs of every node involved in the test-case are dumped to `stdout`. 142*cfb92d14SAndroid Build Coastguard Worker 143*cfb92d14SAndroid Build Coastguard WorkerA `cli.Node()` instance can also provide additional logs and info as the test-cases are run (verbose mode). It can be enabled for a node instance when it is created: 144*cfb92d14SAndroid Build Coastguard Worker 145*cfb92d14SAndroid Build Coastguard Worker```python 146*cfb92d14SAndroid Build Coastguard Worker>>> import cli 147*cfb92d14SAndroid Build Coastguard Worker>>> node = cli.Node(verbose=True) 148*cfb92d14SAndroid Build Coastguard Worker$ Node1.__init__() cmd: `../../../examples/apps/cli/ot-cli-ftd --time-speed=1 1` 149*cfb92d14SAndroid Build Coastguard Worker 150*cfb92d14SAndroid Build Coastguard Worker>>> node.get_state() 151*cfb92d14SAndroid Build Coastguard Worker$ Node1.cli('state') -> disabled 152*cfb92d14SAndroid Build Coastguard Worker'disabled' 153*cfb92d14SAndroid Build Coastguard Worker 154*cfb92d14SAndroid Build Coastguard Worker>>> node.form('test') 155*cfb92d14SAndroid Build Coastguard Worker$ Node1.cli('networkname test') 156*cfb92d14SAndroid Build Coastguard Worker$ Node1.cli('panid 4660') 157*cfb92d14SAndroid Build Coastguard Worker$ Node1.cli('ifconfig up') 158*cfb92d14SAndroid Build Coastguard Worker$ Node1.cli('thread start') 159*cfb92d14SAndroid Build Coastguard Worker$ Node1.cli('state') -> detached 160*cfb92d14SAndroid Build Coastguard Worker$ Node1.cli('state') -> detached 161*cfb92d14SAndroid Build Coastguard Worker... 162*cfb92d14SAndroid Build Coastguard Worker$ Node1.cli('state') -> leader 163*cfb92d14SAndroid Build Coastguard Worker``` 164*cfb92d14SAndroid Build Coastguard Worker 165*cfb92d14SAndroid Build Coastguard WorkerAlternatively, `cli.Node._VERBOSE` settings can be changed to enable verbose logging for all nodes. The default value of `cli.Node._VERBOSE` is determined from environment variable `TORANJ_VERBOSE` (verbose mode is enabled when env variable is set to any of `1`, `True`, `Yes`, `Y`, `On` (case-insensitive)), otherwise it is disabled. 166*cfb92d14SAndroid Build Coastguard Worker 167*cfb92d14SAndroid Build Coastguard Worker## `toranj-cli` and `thread-cert` test framework 168*cfb92d14SAndroid Build Coastguard Worker 169*cfb92d14SAndroid Build Coastguard Worker`toranj-cli` uses CLI commands to test the behavior of OpenThread with simulation platform. `thread-cert` scripts (in `tests/scripts/thread-cert`) also use CLI commands. However, these two test frameworks have certain differences and are intended for different situations. The `toranj` test cases run in real-time (though it is possible to run with a time speed-up factor) while the `thread-cert` scripts use virtual-time and event-based simulation model. 170*cfb92d14SAndroid Build Coastguard Worker 171*cfb92d14SAndroid Build Coastguard Worker- `toranj` test cases are useful to validate the real-time (non event-based) simulation platform implementation itself. 172*cfb92d14SAndroid Build Coastguard Worker- `toranj` test cases can be used in situations where the platform layer may not support event-based model. 173*cfb92d14SAndroid Build Coastguard Worker- `toranj` frameworks allows for more interactive testing (e.g., read–eval–print loop (REPL) model in python) and do not need a separate process to run to handle/dispatch events (which is required for the virtual-time simulation model). 174*cfb92d14SAndroid Build Coastguard Worker- `thread-cert` test cases can run quickly (due to virtual time emulation), but the test script itself needs to manage the flow and advancement of time. 175