iOS MIDI Over Bluetooth

It’s finally under development!  The ability to send and receive MIDI messages to a small wireless Bluetooth box with standard MIDI In/Out ports using an iOS device.  I can’t put a timeframe on the process because I have so much on my plate, but I’ll show below what I’ve got so far and where I plan on going with it.

Overview

Below is a sort of overall diagram for the system.  The iDevice sends and receives MIDI messages with a small external piece of hardware via Bluetooth Low Energy (BLE).  This new type of Bluetooth is responsible for making this whole project possible, and is only supported on iDevices beginning with the iPhone 4S and newer.  This includes the iPhone 5, iPad 3 and 4, newest iPod Touch, and iPad Mini.  The small piece of hardware contains a MIDI In and Out port and communicates these messages with the iDevice.  MIDI Out messages can be used to drive, for example, a DAW or hardware synth, and the MIDI In port can receive and forward controller messages to the iDevice.

Hardware

Here is a picture of my prototype piece of hardware that would send/receive the MIDI messages with the iDevice and the external MIDI device.  The two boards pictured are identical, only one is needed.  The board has a MIDI In and Out port and an RS-232 serial port that is just used to load new firmware to the Bluetooth chip.  Eventually, this would be removed and a USB Mini port put in its place for updating the firmware (if necessary).  Currently, the MIni USB port is just used to power the board.

Functionality

In addition to the hardware, I have written the firmware to be run on the Bluetooth microprocessor and the appropriate MIDI to BLE iOS classes for sending and receiving messages.  By implementing these classes into my work-in-progress ROTOSynth app (see tab, a fully functioning controller and synthesizer iPad app), I am able to both control an external synth using my app’s controller by sending MIDI messages to my hardware (and thus, out the MIDI Out port), or play my app’s synthesizer using an external MIDI controller (connected to the MIDI In on my hardware).  Although already outdated, here’s a video of when I first got one side of the communication going (iPad controller to MIDI Out).

Latency

Now, the most important question everyone will ask is, “Ok, cool, but how does it stack up against the existing ability to send/receive MIDI messages over Wi-fi on iDevices?”.  In short, it beats it – by a lot.

Below are two graphs which show the results of my latency testing of various performance scenarios.  To test, I used two microphones connected to my PC through a Presonus Firebox recording simultaneously in my DAW.  One mic was placed right next to the appropriate controller (a key on the MIDI controller or the screen on my iPad, depending on the test), and the other was placed at the sound source (either the studio monitor or iPad speaker, depending).  For each test, I would begin recording on the PC, then initiate 10 Note On events on the appropriate controller.  I struck the controller assertively so as to capture the touch event on the controller microphone.  Then, by manual analysis in the DAW, I was able to see the latency between the controller touch event and the corresponding sound.  Below is the average latency over 10 trials for each scenario.

Each schencario is explained a little more in depth here:

  • Axiom to Logic w/ MIDI Cable – Essentially the control.  A hardwired MIDI controller (M-Audio Axiom 49) into my USB Focusrite interface feeding into Logic.
  • iPad to Logic w/ BLE – ROTOSynth acting as a BLE controller.  External BLE device connected to MIDI In on Focusrite interface and feeding Logic
  • Axiom to iPad w/ BLE – ROTOSynth acting as the synthesizer.  Axiom controller connected to MIDI In on BLE device and controlling the Core Audio synth in my app.
  • iPad All Local – ROTOSynth acting as both controller and synthesizer.  Tested to get idea of baseline for iDevice synthesis app latency (using Audio Units)
  • SoundPrism to Logic w/ Ad-Hoc – Very popular iPad controller app that implements the Core MIDI framework.  Used for baseline of Core MIDI latency over 802.1 Ad-hoc network
  • SoundPrism to Logic w/ Wi-Fi – Same as above but using an 802.1n local Wi-Fi network rather than ad-hoc network.

As can be seen, the latency with BLE is about 30 ms better than with the best case scenario using Core MIDI’s implementation over Wi-fi.  Even more, look at the graph below, which shows the Standard Deviation (SD) of the 10 latency tests for each scenario.  For those not familiar, the standard deviation gives a measure of how much the data varies around the average value.  In this context, it tells us how much the latency varies from one note to the next.  A high SD means the latency is all over the place, a low SD means the latency is very consistent.

The SD using BLE for MIDI is remarkably lower than with MIDI over Wi-fi.  This means that the latency is more “learnable” so that musicians can adapt and not be left guessing at when the note they press will actually be played.

Versatility

Now, the fastest way for users to get up and running with MIDI over BLE is for me to release the hardware along with a graphically basic app that supports Virtual MIDI.  By doing so, my app could receive and transmit MIDI messages between other iOS apps and the external piece of MIDI hardware connected.  However, that’s the most basic application.  Multiple BLE devices can be connected at once – and each with their own input/output data stream, so ideally, you could control, or be controlled, by several external MIDI devices.  Even further, a MIDI to DMX (lighting) box would be totally possible, so iOS apps could implement full control over live venue lighting!  Beyond the basic implementation to get MIDI over BLE performing the functions of MIDI over Wi-fi, there’s countless more possibilites!

Interested?

If this sounds like something you’d like to use, either as an end-user or developer wanting to make cool, multi-MIDI device apps, comment below to let me know!  With enough interest, I would certainly pursue this with a kickstarter project for the hardware and a simple iOS app for interfacing with Virtual MIDI apps.  If developers want to use multiple devices to start implementing cool multi-device control, I’ll consider releasing an SDK.  You guys determine it, so just let me know!

Progress Update:

See here for an update on progress.

24 thoughts on “iOS MIDI Over Bluetooth

  1. Point me to the kickstarter page, man. This is brilliant and would open up new realms of live jamming with hardware.

  2. This is a bit of a holy grail for iOS devices. I’ve talked to a few people about something like this but the bottle neck has always been how to get the iPad to recognize the device as a Core MIDI device over wifi. If you can make this happen I think there will be tremendous interest once people realize what they can do with it. Please keep me posted and I will keep my readers posted over on MATRIXSYNTH. If you do pop this up on Kickstarter, let me know. I put an initial post up here: http://m.matrixsynth.com/2012/10/wireless-midi-bluetooth-box-finally.html

    cheers,
    matrix

  3. I also tested the latency of BLE with MIDI in to BLE 4.0 module to iPhone, about 30ms with about 5ms deviation using our own soft sounds. I found that the latecy can be decreased by 10ms by adjusting the Bluetooth connection parameters. Hope this can help you.

  4. Great idea Jay!!! Just one sugestion: use batteries instead of the USB port to power it (I’m looking forward to attach it to the back of my keytar controller to have complete freedom…). Waiting in line for the Kickstarter page!!!!!!!!!

  5. Count me in as an end user that would use it and would join the kickstarter. Could you increase the price of the hardware so that the app could be free and keep the simple app as open source to make sure we have access to the app in the future even if we have to compile it ourselves with a developer account? I have seen developers and apps disappear on the App Store.

  6. Hi Michael,

    Yup, I’ve already got the connection interval to the minimum. Actually very consistently 18-18.7 ms.

  7. I’ve learn some just right stuff here. Definitely price bookmarking for revisiting. I wonder how much attempt you put to create this sort of wonderful informative website.

  8. Awesome work man! U’ve found yourself another backer!

    Have you given much thought to supporting OSC too?

  9. Hey, great work! This is exactly what I’ve been waiting for in order to build my next controller. Please keep me informed!

  10. hi, that project is totally what i’m after!
    any chance you would release the schematics ?
    if you kickstart it i’ll totally follow/help !
    web developper/mac addict/musician/french

  11. Hi, sorry but I won’t be releasing the schematic. The schematic itself wouldn’t do much good anyway without the firmware I’ve written for the Bluetooth microprocessor, which is where all the hard work is. Thanks for the support!

  12. I can’t wait until your project comes to fruition! I hope every MIDI controller and software manufacturer beats your door down to licence your technology!