Introduction

Overview

The Ohmni WebSDK provides an easy, sandboxed way for you to develop rich, interactive web applications that run on the robot.

The SDK gives you the ability to use any web application as the UI shown on the robot with full control over the display and audio. You can stream videos or music with HTML5, play YouTube videos, or run networked web games, etc.

Also, your web application can make special calls that trigger the physical interfaces of the robot - moving around, looking up and down with the neck, controlling the lights, as well as using text to speech and speech interfaces.

There are two modes of operation: Standalone and In-call.

Standalone mode is a way to load and run your web app directly on the display of the robot when no user is connected. Essentially, your URL is loaded into a sandboxed WebView with special hooks so that you can call lower level functions.

In-call mode gives you access to the same suite of calls and controls, but the HTML5/CSS/JS logic runs in the calling user's browser, during the telepresence call. This gives you massive flexibility to add backend integrations, additional UI and interactive HTML5/CSS/JS, etc. all from within the call. The API commands are then RPC'ed over the telepresence connection and run on the bot.

Standalone mode

Write any standard web page in whatever framework you choose. It can be static pages, Rails, Express, etc. Make it as interactive as you like - it will appear on the robot's screen and people with the robot can click on buttons, etc.

Use WebAPI calls to control the motion and behavior of the robot. It's as easy as that! You can trigger them on a javascript timer, based on button presses, or based on some complex WebRequests - whatever you want.

Now, from the Ohmni web app (https://app.ohmnilabs.com), log in and click the gear icon to bring up additional settings for the robot that you want to run the code on. It expands the fields as shown below.

Simply enter the URL of the target page of your web app into the "Load URL" box and click the "Load" button:

You should see the page load on your Ohmni’ screen. If you are in a call, the page will overwrite the video screen on the robot. To clear this page, clear “Clear” button. You may choose to set this URL persistent on Ohmni's screen by checking the "Set URL as home screen". Whenever Ohmni starts up or is idle, it will show this screen.

Add ohmni-standalone.js to your app

Now, to get started with robot control calls in your web application, include Ohmni.js in the header of your web app:

<script src="https://api.ohmnilabs.com/ohmni-api/Ohmni-standalone.js"/>

This will allow you to use calls in the Ohmni.x namespace wherever you want to in your web application. For example, you could bind a button onclick handler to call the function as follows:

<button onclick="tiltdown()">Tilt neck down</button>
<script>
function tiltdown() {
    Ohmni.setNeckPosition(650, 100);
}
</script>

Sample Code

https://api.ohmnilabs.com/ohmni-api/test-standalone.html

In-call mode

The in-call mode is very useful when you want to extend a telepresence session to include additional UI, information, interaction, etc. To spark your imagination, here are some cool applications people have built with it:

• Robot arm teleoperation
  • OhmniLabs R&D has built some cool extensions to use Vive VR controllers with a Chrome browser extension to pipe the data over the call and teleoperate an arm added to Ohmni.
• Patient data and caregiver survey
  • Some health care partners add in backend data integrations that allow the remote caregiver to log data about the patient during their visit, which goes straight to their own backend.

Version Requirements

Update to the latest Ohmni app version from the robot’s setting menu. This API is only available on Gen12 version hardware and above. To check: 1. Log into the ‘Your Bots” page - http://app.ohmnilabs.com 2. Hover over the robot name and click on the gear icon to the right of the name 3. Version must be 4.0 or higher

Getting Started

Add ohmni.js to your web app

<script src="https://api.ohmnilabs.com/ohmni-api/Ohmni-incall.js"></script>

For any clickable DOM element, add data-external="true" property to avoid conflict with underlying video stream DOM.

Set your app as an overlay

Host the app and set the url in Ohmni settings panel.

NOTE: If you do not see the “Overlay HTML” field. Contact OhmniLabs Help Desk for the capability to be enabled.

Click “Save” and refresh the page for this to take effect. Connect to the robot and the app should will be overlayed on the video stream.

Demo app

Paste the URL below into the “Overlay HTML” field (as above) to see a simple example. Feel free to look at the source to see how things work.

https://api.ohmnilabs.com/ohmni-api/test-incall.html