Keywords : Android, GEM5, mouse tap, mouse move, touch point tap, touch point move , click

This post is mainly answering this question:

Is there a kind scripted , automated way, to perform mouse (touch point) movement, and mouse click, Inside Android, Running under GEM5?

Problem Description: When you are running Android under GEM5, it is very slow. This makes user interaction with Android GUI difficult. some times for example, you perform a click and at the end you don’t understand if the click operation has done or not, because, it takes a lot of time (e.g. 5 minutes or more) for the system to show you any kind of response.

Solution : We need a kind of scripted, batch mode, way to perform interaction with the GUI. For example, it would be very nice if we could write a script which contained all of the mouse movements and clicks that we want to do, and then to run this script at the suitable time.

Ready tools to perform this action: Google is providing a tool called : monkeyrunner , which is able to do this. Monkeyrunner uses the Android Debug Bridge (ADB) to connect to your Android device. The user specifies the list of desired transaction using a Python script. Monkeyrunner actually executes this Python script. (http://developer.android.com/tools/help/monkeyrunner_concepts.html)

The above solution however doesn’t work with GEM5. Because (at least as far as I know, and by the timing this writing) you can not get connected to GEM5 core running Android, through a normal ADB. In fact, GEM5 provides its own ADB.

Howto Use ADB under GEM5

Running ADB with GEM5 is easy, you need just to log into GEM5 simulation environment using m5term. So, after you started your simulation and your Android got booted, just run m5term:

./m5term 127.0.0.1 3456

Now, enable the ADB daemon: (inside m5term environment type)

adb shell

Now you can issue , a big list of different adb commands. here is the list of what you can do with adb under GEM5 Android: (just type adb on m5term prompt – don’t forget Android is fully booted and running)

root@android:/ # adb
Android Debug Bridge version 1.0.29

 -d                            – directs command to the only connected USB device
                                 returns an error if more than one USB device is present.
 -e                            – directs command to the only running emulator.
                                 returns an error if more than one emulator is running. …

(It is a long list…) As you see there are some very useful options available by adb.

Sending Key Events to Android Device

Like every Android based phone, your simulated GEM5 based Android also provides the user with a set of interface keys. You can use the input keyevent ADB command to send different key events to your Android device.

For example. when you boot Android under GEM5, since it takes usually a lot of time, your leave your computer. The Android gets booted and since there is no user interaction it goes to a kind of sleep mode. In this mode, based on my experience, you can see the user interface, you can move the big touch point circle, you can perform a touch point tap using it (and it actually shows that you have performed the tap) but you don’t see any response from the Android side. In order to wake the Android up, you can send a key sequence to it. For example, the following command sends “Enter” key event to the Android device.

adb shell input keyevent 66

A complete list of keys and also description of input command is available at:

http://stackoverflow.com/questions/7789826/adb-shell-input-events

Using getevent and sendevent ADB commands

Before you can send mouse events to your android device, you need to understand what is the event interface through which you should send your events. In order to do this we perform a simple test: while Android is running we issue the following command. This command makes the adb to show us all of the events received by the mouse/keyboard system of Android:

adb shell getevent

Now, we try to perform some touch point movements and some touch point taps inside our VNC viewer. (the one which is showing us the Android) example output for what you see:

127|root@android:/ # adb shell getevent
add device 1: /dev/input/event1
  name:     “touchkitPS/2 eGalax Touchscreen”
could not get driver version for /dev/input/mouse0, Not a typewriter
add device 2: /dev/input/event0
  name:     “AT Raw Set 2 keyboard”
could not get driver version for /dev/input/mice, Not a typewriter

/dev/input/event1: 0003 0000 00000577
/dev/input/event1: 0003 0001 000007f4
/dev/input/event1: 0000 0000 00000000
/dev/input/event1: 0003 0000 00000573
/dev/input/event1: 0003 0001 000007b1
/dev/input/event1: 0000 0000 00000000
/dev/input/event1: 0003 0000 0000056f
/dev/input/event1: 0003 0001 00000784

Now, we can learn these event sequences and produce them ourselves. This link gives a nice discussion on it:

http://softteco.blogspot.de/2011/03/android-writing-events-low-level-touch.html

The point is, based on my observation, although you may be running Android ICS on your GEM5 machine, the event interface is still the old Android 3.2 interface. So if you are interested in producing mouse movement events, look at 3.2 related code in the web site.

Here I focus on producing a mouse tap. We suppose you have moved the big circle into its correct position and now just you need to perform a good mouse click. Or another example, you have run Android with GEM5.fast in simplest possible hardware configuration , up to the point that your benchmark and its configurations are all ready and the touch point is on the “start” button of your benchmark and you have created a check point. Now you have restored in O3 detailed mode with caches and so on and GEM5 is extremely slow and you want to be sure that the click operation is getting happened. So you want to do the tap through a script:

adb shell sendevent /dev/input/event1 1 330 1
adb shell sendevent /dev/input/event1 0 0 0
adb shell sendevent /dev/input/event1 1 330 0
adb shell sendevent /dev/input/event1 0 0 0

Here we are performing one touch point tap operation. Please note that, the numbers you pass to sendevent are all decimal. the numbers that you receive through getevent are all hex values.

P.S. A good link describing sendevent and getevent

http://android.stackexchange.com/questions/26261/documentation-for-adb-shell-getevent-sendevent

Another always useful link – The Android Emulator

http://developer.android.com/tools/devices/emulator.html#console

Another nice note, on Android events

https://groups.google.com/forum/?fromgroups=#!topic/android-beginners/il2PqSKRFNI

The source code of Monkeyrunner on AndroidXREF

http://androidxref.com/4.0.4/xref/sdk/monkeyrunner/src/com/android/monkeyrunner/

Yup! Enjoy!