Myo SDK – 0.9.0
Writing Myo Scripts

While the Myo SDK provides powerful and complete facilities for writing applications that make use of the Myo armband's capabilities, some tasks can be more easily accomplished through scripting.

Myo Connect meets this need by running connectors that can handle Myo events and use them to control applications. Myo Scripts are connectors written in Lua, which is a simple scripting language that is often used for application scripting. To learn more about Lua, visit its homepage or read its documentation.

Myo Scripts are managed via the Application Manager, accessible by clicking the Application Manager entry in the Myo Connect menu. This window allows the user to add, reload and remove scripts, as well as adjust the order in which they are given access to applications. (Scripts closer to the top are given precedence.)

Full reference documentation for Myo Scripts can be found at the Myo Script Reference page.

Interfacing with Myo Connect

Myo Connect interacts with scripts through two primary mechanisms: Callback functions and API functions.

Callback functions are implemented by scripts to handle specific events or request information. When a designated event occurs, Myo Connect calls into the script, in some cases providing additional information. This includes changes in the active application, changes in the Myo armband's active pose, and so on.

API functions provide scripts with additional functionality to access the armband and system information. This includes angular values for the armband's orientation, vibrating the armband, the current time, and outputting to the debug window. It also includes commands for manipulating the system by emitting keyboard events, which can be used to control applications. Full documentation of the API can be found here.

There are a few global variables that are used to communicate between scripts and Myo Connect. Myo Connect provides the platform variable to indicate which platform it is running on. It can be either "Windows" or "MacOS" currently. Scripts are expected to set a variable called scriptId to a reverse domain name style unique identifier. It should be of the form com.example.someapplication. scriptTitle is the actual name of the script that will show up in Myo Connect, and scriptDetailsUrl should be the URL of the script in the Myo Market. It will be of the form https://market.myo.com/app/<app id>, and you can get the ID as soon as you create the draft application.

Myo Connect additionally provides the standard Lua libraries package, coroutine, table, string, bit32, and math for use by all scripts. The libraries io, os and debug are not provided. Scripts may not access the filesystem.

Each script is provided its own independent, self-contained Lua scripting environment. Variables created in one script are not available to any others.

A Simple Myo Script

To illustrate the basic flow of a Myo Script, let's look at a simple example that outputs to the Myo Connect debug console when the callbacks are invoked. Note that we leave onPeriodic() clear for now to avoid excessive output.

scriptId = 'com.thalmic.examples.outputeverything'
scriptTitle = "Output Everything"
scriptDetailsUrl = "" -- We don't have this until it's submitted to the Myo Market

function onPoseEdge(pose, edge)
    myo.debug("onPoseEdge: " .. pose .. ", " .. edge)
end

function onPeriodic()
end

function onForegroundWindowChange(app, title)
    myo.debug("onForegroundWindowChange: " .. app .. ", " .. title)
    return true
end

function activeAppName()
    return "Output Everything"
end

function onActiveChange(isActive)
    myo.debug("onActiveChange")
end

Loading this script into Myo Connect should provide some insight into how scripts are accessed.

If you are in developer mode, the debug console window will appear almost immediately (if not, toggle it Myo Connect -> Preferences). The API call myo.debug() commands Myo Connect to display the provided string in this window, raising the window if it isn't currently visible. This can be used to test and debug scripts. In our case, we are displaying events as they happen.

As the active app and window changes, calls to onForegroundWindowChange() with the names and window titles displayed will appear. The return true line indicates to Myo Connect that our script wants to control all applications unconditionally. In practice the app and title values are used to decide whether or not the active application is the one to be controlled.

onActiveChange() is called when our script becomes active or inactive based on changes in the active application and the value returned by our script in onForegroundWindowChange(). Since we always return true in the latter, this will only be called once, when our script becomes active after its first request to do so. Similarly, activeAppName() is called once upon becoming active so that Myo Connect knows which application we are controlling. This facility is provided since scripts may control different applications, and because the system names might not be very clear for users. We will illustrate further in the next example.

When poses are made using the Myo armband while running this script, output will appear relating to the pose and the edge. The pose refers to the hand configuration made, such as "fist", "waveOut", "rest" and so on. The edge refers to whether the pose is being made or released. These will typically be received in pairs, with the previous pose being released with an "off" edge and the new pose being made with an "on" edge. Usually this will entail a "rest" pose as an intermediary. The pose can also be "unknown" if the armband is not worn or properly calibrated. See the API documentation for more information.

Finally, although this first example does not employ it, onPeriodic() is called periodically to give the script an opportunity to perform time-based tasks. This could be a delayed task, doing something a certain amount of time after a trigger such as a particular pose is made, or a repeated task, such as periodically scrolling or advancing. Myo Scripts typically make use of this facility by getting the current time with myo.getTimeMilliseconds(), storing it in a variable, and comparing it with the current time in subsequent onPeriodic() calls. In the next example, we will use this to implement a periodic action.

A More Practical Example

Listed here is the PowerPoint Connector used to provide control over PowerPoint for giving presentations with the Myo armband. It implements a standard timed unlock mechanism via the Double Tap pose and allows the user to change slides forward and backward by performing the Wave In and Wave Out poses. Holding the wave poses enables a periodic "shuttle" feature so that they may quickly traverse the presentation.

Since this script is built into Myo Connect, it can be used out of the box by running and giving focus to PowerPoint. You can also download the most up to date version on the Myo Market here. More detailed discussion follows below.

scriptId = 'com.thalmic.scripts.presentation'
scriptDetailsUrl = 'https://market.myo.com/app/5474c658e4b0361138df2a9e'
scriptTitle = 'PowerPoint Connector'

function onForegroundWindowChange(app, title)
    local uppercaseApp = string.upper(app)
    return platform == "MacOS" and app == "com.microsoft.Powerpoint" or
        platform == "Windows" and (uppercaseApp == "POWERPNT.EXE" or uppercaseApp == "PPTVIEW.EXE")
end

function activeAppName()
    return "PowerPoint"
end

-- flag to de/activate shuttling feature
supportShuttle = false

-- Effects

function forward()
    myo.keyboard("down_arrow", "press")
end

function backward()
    myo.keyboard("up_arrow", "press")
end

-- Helpers

function conditionallySwapWave(pose)
    if myo.getArm() == "left" then
        if pose == "waveIn" then
            pose = "waveOut"
        elseif pose == "waveOut" then
            pose = "waveIn"
        end
    end
    return pose
end

-- Shuttle

function shuttleBurst()
    if shuttleDirection == "forward" then
        forward()
    elseif shuttleDirection == "backward" then
        backward()
    end
end

-- Triggers

function onPoseEdge(pose, edge)
    -- Forward/backward and shuttle
    if pose == "waveIn" or pose == "waveOut" then
        local now = myo.getTimeMilliseconds()

        if edge == "on" then
            -- Deal with direction and arm
            pose = conditionallySwapWave(pose)

            if pose == "waveIn" then
                shuttleDirection = "backward"
            else
                shuttleDirection = "forward"
            end

            -- Extend unlock and notify user
            myo.unlock("hold")
            myo.notifyUserAction()

            -- Initial burst
            shuttleBurst()
            shuttleSince = now
            shuttleTimeout = SHUTTLE_CONTINUOUS_TIMEOUT
        end
        if edge == "off" then
            myo.unlock("timed")
            shuttleTimeout = nil
        end
    end
end

-- All timeouts in milliseconds
SHUTTLE_CONTINUOUS_TIMEOUT = 600
SHUTTLE_CONTINUOUS_PERIOD = 300

function onPeriodic()
    local now = myo.getTimeMilliseconds()
    if supportShuttle and shuttleTimeout then
        if (now - shuttleSince) > shuttleTimeout then
            shuttleBurst()
            shuttleTimeout = SHUTTLE_CONTINUOUS_PERIOD
            shuttleSince = now
        end
    end
end

Keyboard Commands

The script controls Powerpoint and Keynote by sending keyboard events to the system while these applications are in the foreground. Myo Connect provides an API for sending keyboard events through the myo.keyboard() function, documented in detail here.

function forward()
    myo.keyboard("down_arrow", "press")
end

function backward()
    myo.keyboard("up_arrow", "press")
end

The above snippet shows a pair of helper functions for sending the relevant key presses to move to the next and previous slides using simple presses of the up and down arrow keys.

-- Burst forward or backward depending on the value of shuttleDirection.
function shuttleBurst()
    if shuttleDirection == "forward" then
        forward()
    elseif shuttleDirection == "backward" then
        backward()
    end
end

This snippet shows another helper function that calls the previous ones based on the state of the shuttleDirection variable. This allows the script to determine which direction to move when the pose is first detected in onPoseEdge(), but to separately send the key events based on time events from onPeriodic().

Lock / Unlock Behaviour

To prevent undesired accidental commands while wearing the Myo armband, by default it remains in a locked state until the standard unlock pose, Double Tap, is detected. This starts letting poses through to your application. After a few seconds, the Myo armband locks itself again automatically. You may want to keep the Myo armband unlocked longer, as with the shuttle feature of our script. Obviously, having the armband lock while in the middle of skipping slides is not ideal, so we call myo.unlock("hold") to keep it unlocked until the user releases the pose, where we set it back to the default myo.unlock("timed").

function onPoseEdge(pose, edge)
    -- Forward/backward and shuttle
    if pose == "waveIn" or pose == "waveOut" then
        local now = myo.getTimeMilliseconds()

        if edge == "on" then
            -- Deal with direction and arm
            pose = conditionallySwapWave(pose)

            if pose == "waveIn" then
                shuttleDirection = "backward"
            else
                shuttleDirection = "forward"
            end

            -- Extend unlock and notify user
            myo.unlock("hold")
            myo.notifyUserAction()

            -- Initial burst
            shuttleBurst()
            shuttleSince = now
            shuttleTimeout = SHUTTLE_CONTINUOUS_TIMEOUT
        end
        if edge == "off" then
            myo.unlock("timed")
            shuttleTimeout = nil
        end
    end
end

This is the recommended way to handle locking and unlocking in your script, and is what users will expect. If your script should be unlocked all or most of the time (if you are controlling a game, for example) you can disable the standard locking behaviour completely by calling myo.setLockingPolicy("none"). Turn standard locking back on with myo.setLockingPolicy("standard").

Dealing With Directional Poses

Since the Myo armband may be worn on both left and right arms, the Wave In and Wave Out poses do not by default indicate the correct left and right direction. We handle this by swapping them when the armband is detected to be on the left arm, as determined by myo.getArm(). We can subsequently treat Wave Out as to the right and Wave In as to the left.

-- Makes use of myo.getArm() to swap wave out and wave in when the Myo armband is being worn on
-- the left arm. This allows us to treat wave out as wave right and wave in as wave
-- left for consistent direction. The function has no effect on other poses.
function conditionallySwapWave(pose)
    if myo.getArm() == "left" then
        if pose == "waveIn" then
            pose = "waveOut"
        elseif pose == "waveOut" then
            pose = "waveIn"
        end
    end
    return pose
end

Handling Pose Input

As mentioned, changes in poses detected by the Myo armband are communicated to the script by calling onPoseEdge() with the pose changing state and the new state ("edge"). The section of onPoseEdge() below looks for the Wave In and Wave Out poses in order to determine shuttle direction and perform the initial slide change.

function onPoseEdge(pose, edge)
    -- Forward/backward and shuttle
    if pose == "waveIn" or pose == "waveOut" then
        local now = myo.getTimeMilliseconds()

        if edge == "on" then
            -- Deal with direction and arm
            pose = conditionallySwapWave(pose)

            if pose == "waveIn" then
                shuttleDirection = "backward"
            else
                shuttleDirection = "forward"
            end

            -- Extend unlock and notify user
            myo.unlock("hold")
            myo.notifyUserAction()

            -- Initial burst
            shuttleBurst()
            shuttleSince = now
            shuttleTimeout = SHUTTLE_CONTINUOUS_TIMEOUT
        end
        if edge == "off" then
            myo.unlock("timed")
            shuttleTimeout = nil
        end
    end
end

Much of the above code is setting states up for the timed logic in onPeriodic().

Timed Behaviour

The key to time-based behaviour is the onPeriodic() callback, which is called every 10 milliseconds while the script is active, and myo.getTimeMilliseconds() which returns the current time. This value can be used to measure intervals so that effects can be triggered after specified durations.

In the snippet below, we keep track of when the most recent shuttle command was issued with shuttleSince as well as the current shuttle timeout with shuttleTimeout. Whenever we cross the timeout threshold we emit a shuttle burst, changing the slide in the determined direction, and reset shuttleSince to the current time. shuttleTimeout is initially set to the longer value when shuttle behaviour is enabled, providing the initial delay, then reset to the shorter one once it has been triggered, providing the faster repeat rate.

-- All timeouts in milliseconds

-- Delay when holding wave left/right before switching to shuttle behaviour
SHUTTLE_CONTINUOUS_TIMEOUT = 600

-- How often to trigger shuttle behaviour
SHUTTLE_CONTINUOUS_PERIOD = 300

function onPeriodic()
    local now = myo.getTimeMilliseconds()

    -- Shuttle behaviour
    if shuttleTimeout then

        -- If we haven't done a shuttle burst since the timeout, do one now
        if (now - shuttleSince) > shuttleTimeout then
            --  Perform a shuttle burst
            shuttleBurst()

            -- Update the timeout. (The first time it will be the longer delay.)
            shuttleTimeout = SHUTTLE_CONTINUOUS_PERIOD

            -- Update when we did the last shuttle burst
            shuttleSince = now
        end
    end
end

Controlling The Right Application

Our script is intended for presentations, but how do you determine if a presentation application is in focus, and how can you tell Myo Connect that we want to control it? This is the purpose of the onForegroundWindowChange() callback, which is invoked for all scripts whenever the foreground application or window changes. The script may use the values provided - the names of the current application and title of its active window - to determine whether or not to request control of the application. This is done by the return value, which should be true if the script wants control and false otherwise.

function onForegroundWindowChange(app, title)
    local uppercaseApp = string.upper(app)
    return platform == "MacOS" and app == "com.microsoft.Powerpoint" or
        platform == "Windows" and (uppercaseApp == "POWERPNT.EXE" or uppercaseApp == "PPTVIEW.EXE")
end

The script controls PowerPoint on both the Windows and Mac OS platforms. On Windows for app you will get the name of the actual executable, and on Mac it will give you the package name, so the script makes use of the global variable platform to determine which it should match. On Windows you should be able to recognize app regardless of the case, since the file system is not case sensitive.

The title variable will be the name of the window, which can be useful in, for example, controlling specific webpages in a browser. It can be less reliable than using app, so we recommend you use app when available.

When control is yielded to (or taken from) the script, its onActiveChange() callback is called in preparation. This is your opportunity to do any setup or cleanup required. If you are doing separate down and up events with the keyboard rather than using press, this is a good place make sure everything is up. Most scripts will probably not need to implement it, however.

function onActiveChange(isActive)
    -- Our script doesn't need to implement this
end

Only the active script receives onPoseChanged() and onPeriodic() callbacks, so it's important to correctly request activation not only so that the script will run but so that others will not be prevented from doing so themselves.