IoT (tentative title)

About

IoT is a universal environment - not universal in the sense that it runs on everything, universal in the sense that you can choreograph all aspects of your life with it. Think of it as a system that turns the world around you into your personal assistant. Siri may be able to take simple minded instructions, and Google Now might be able to spy on your appointments, but they're stuck in your phone.

In less abstract terms, you can think of it as a mashup of Siri, Google Now, a programming language, and some home automation software, if that helps.

Why?

There are lots of existing solutions for programming things, if you’re a programmer by trade. Unfortunately, software tends to fall into one of two categories - strict, over-engineered solutions that are designed to work nicely for long periods, and hacky scripts that are designed to do one thing really quickly before they’re discarded. The former is somewhat difficult for non-programmers to play with, because it generally requires understanding a complicated project and all its technical documentation, and the latter is somewhat annoying even for programmers to use, because the majority of the time spent writing each script is reading documentation and configuring basic things over and over again. Even after the script is written, you then have to worry about things like deployment and monitoring, which aren’t particularly fun.

Even for programmers, certain tasks aren’t easy. If I told a non-programmer to send an email whenever I get a message over Facebook Messenger and sat them at my desk, they wouldn’t really have to do anything - Facebook and Mail.app are probably already open, and they’d just click a couple buttons when they notice a new message come in. If I gave the same task to a programmer, they’d probably spend most of the time reading Facebook’s documentation, followed by learning how to set up SMTP, and only a couple minutes actually writing the code to connect the two.

IoT is designed to be both super powerful (in what it lets you do), and super easy (in the way you do it). For an average user, this means that you can write one-line programs, like “if I walk in the front door, turn on the living room lights” with ease.

For the non-programmers, there are several awesome features:

Even for programmers, there are several notable features that make it super useful:

Out of the box, it even comes with a bunch of modules to do certain things:

For example, we could use it to do any of the following, with a few lines of code:

With regards to hardware, multiple vendors’ products are generally supported for things like lighting. If a specific device isn’t supported, it generally only takes a couple days to add support for it, given access to the hardware in question. Other modules can be written to handle more abstract things fairly easily - the current list of modules is mostly just based on what I wanted to exist at the time.

I want to emphasize, however, that just because something CAN be supported, doesn’t mean it SHOULD be supported. As an example, using Insteon for lighting is almost instantly responsive, whereas Hue takes a few seconds to report new data. If the alternative is between being able to control your lights and not being able to control your lights because they aren’t compatible with Insteon, then by all means, use Hue. However, if you’re trying to retrofit something with poor support (be it lighting, a messaging platform, or something else like an old thermostat), you might find it easier to just switch to something with nicer technical guts.

Aesthetics

‘What does it look like’ is a difficult question to answer - in short, it can look like anything you want - even nothing at all. IoT is simply a program that tracks actions and lets you generate reactions in response. That means you can use it to write rules that don’t have any aesthetic component whatsoever. For example, if you used it to turn a light in a closet on and off when the door is opened and closed, there’s no control panel or GUI necessary - the only hint that it exists is the fact that the lights turn on and off, and the few lines of code you wrote to make that happen. However, there might be some things worth displaying - maybe you want a nice control panel by the door, or something on your desk, with hundreds of buttons you can press. That’s where Panel comes in.

Panel is a user interface for IoT - it isn’t the only interface, or the best interface, or the officially supported one - rather, it’s the one that I happened to write that seemed useful for me. It’s written in React.js / CSS / HTML, with native components for Android and Windows - a fancy way of saying “web browser with elevated permissions”. If you don’t want to use it, and wish to write your own UI, feel free to ignore the rest of this section - however, you may be interested in the native components for whatever interface you come up with, if you choose to simply write a web interface. The native components simply display a web page on screen, but allow for additional features like remotely adjusting the brightness of the display (so when you sleep it won’t blind you), enable the ability to play background sounds in a browser (something Android prevents by default), or automatically restarting the web browser in the event of a crash. Panel has no special permissions within IoT that would otherwise prevent you from writing a separate frontend - it’s just a web page that uses the /api/ feed, that happens to be served out by IoT’s webserver.

Here's an example of what it can look like:

A screenshot from the wall-mounted kitchen tablet.

Tapping anywhere on screen brings up a control panel for that room, with additional rooms selectable on the side.

You can deploy new versions of Panel by either changing the value of VERSION in the Device table in the database, sending the word ‘deploy’ to the server (if you have things listening for text/voice commands), or just pressing refresh in your browser. The number for VERSION is meaningless, except that higher numbers cause any device viewing the page to reload. Make sure that the copy of Panel you deploy actually works first, otherwise you’ll lose the ability to deploy more versions without manually resetting devices or pressing refresh. You can bypass this by using native components (which will keep restarting the browser if they receive a bad version), but that only works if you have those set up on each device.

Panel is made up of many components - they’re located in app/assets/javascript/components/. Each component is only responsible for displaying data - there’s no decision making logic inside Panel. Components are either used to put pages of stuff on screen (e.g. a page of icons you can tap), or are sub-components that are being put on screen by a different component (e.g. each individual icon). You will definitely need to customize the panels that are included by default, because they’re designed for my exact condominium. However, you should use them as a guide, as they demonstrate how to use React to display icons, lists of things, a media player, a clock, and more. Changing parts of a panel usually just involves copying and pasting other lines.

IoT has no mechanisms for automatically generating panels based on adding or removing hardware, so you’ll have to copy and paste lines if and when you want buttons to do things. A feature does not need to exist on a panel to exist in the rest of the software (i.e. you might not want a button to turn on the stove that someone can accidentally press, even though the software might still have the ability to do that).

Panel was designed with a few goals in mind - my goals might not necessarily apply to you, but I feel they’re worth mentioning compared to most other home automation UIs:

Not all of these options require dedicated hardware. For another option, consider this screenshot of my desktop:

Clicking anywhere on the desktop brings up the same UI. The desktop version gets access to other panels that I've restricted on other devices. It also doesn't have a clock in the bottom corner, because I've turned it off.

There are other interfaces worth looking at, depending on the time and place you want to use them. Most of the time, for example, I use IoT’s natural language parser to talk to it, using my normal messaging client, Riot. It looks something like this:

Here's a simple conversation in Riot with IoT.

The chat window supports text, file attachments, videos, and images. Here's an example in which I told IoT to display new photos that my friends post on Instagram, as well as control some speakers. You can also use it as a debugger or a console, if that's your thing.

Here's an example of conversation threading - IoT knows that "turn off the kitchen lights" isn't a response to the question about the email address, but rather something else. When I then respond with 'no one', it figures out that I was referring to the question it asked earlier.