Categories
Engineer gadgets webrtc

Hacking together a WebRTC Pi in the sky – keevio eye

WebRTC on a drone

Team ipcortex put together the keevio eye hack for the TADHack London mini hackathon at Idea London on 11-12th April. The idea was to develop a proof of concept for WebRTC running headless on small embedded devices and talking to our keevio video chat interface. Hardly mission critical but TADHack is a load of fun, and a good way of trying stuff out that pushes the technology envelope a bit which inevitably ends up feeding back useful ideas and techniques into our core platforms. There was also a lot riding on this after our success with RTCEmergency at TADHack last year. Matt Preskett is one of our lead developers and the guy behind the hack, and this is his write up of the experience of developing the app.

On the Tuesday evening before TADHack we hadn’t had time to think about possible hacks. We’d been busy with other events and the process of progressing our core keevio platform towards release. A few months ago we were playing with the idea of a WebRTC Raspberry Pi security camera for our bike shed, so as I walked out of the office I suggested, perhaps to my detriment, that it might be a fun idea to use our JavaScript API, running on a Raspberry Pi strapped to the bottom of a quad copter feeding live video via WebRTC…

I did a bit of research on Tuesday evening, but decided with the timescales involved and some of the parts/equipment needed that perhaps we were biting off more than we could chew. Also I wasn’t really sure how I was going to run our API on a headless Raspberry Pi 50ft in the air. Even if that could be overcome I wasn’t sure the ARM processor would be up to the task of decrypting and encrypting the streams.

Wednesday morning I had all but written off the idea. At the time I was working on load testing our UC platform, which required running our API on a headless server. I set about looking into running headless Chromium, and, by the end of Wednesday with the help of Xvfb I had our API running and automatically accepting video chat from keevio.

Thursday was a busy day we didn’t really have an opportunity to discuss the hack. Rob as perhaps a sign of desperation speculatively ordered a Pi and Pi camera.

Friday morning and we still hadn’t concluded what we’d be doing for the hack, after a quick meeting we gave Pi copter (Pi in the sky?) the go ahead. We had just over 48 hours to put all the pieces together. I started off with Raspbian; I don’t really like the extra gumpf that comes with this distribution but I didn’t have time to piece a fresh instance of Debian together. Raspbian only offers Chromium 22 in its repositories; this was when WebRTC was in its infancy. I looked at compiling the latest Chromium, but this would require either a cross compile environment or compiling on the Pi, neither of which I had time for. I looked around again for an alternative distribution and settled on Arch after checking that they offered an up to date version of Chromium for ARM. It’s a bit bleeding edge but more than sufficient for our requirements.

After getting the Pi installed the first thing was to get Chromium to recognise the camera. Chromium talks to video devices through the V4L component of linux.

I inserted the following lines to /boot/config.txt to enable the camera:

gpu_mem=128

start_file=start_x.elf

fixup_file=fixup_x.dat

Then I added the camera module to /etc/modules-load.d/raspberrypi.conf:

bcm2835-v4l2

After rebooting the Pi, udev created a /dev/video0 device, so it was looking good. The next step was to install Chromium, Xvfb and lighttpd. I setup lighttpd to listen on loopback as I was going to be hard coding the username and password into the webpage: not nice but necessary.

This is the JavaScript I wrote for the hack, due to using our API I could keep it short and sweet.


var keevioShare = (
  function(username, password) {

    function avCB(av) {

      console.log('INFO: avCB with', av);

      if ( av.get('existing') )
        return;

      av.hook(
        function() {
          if ( av.get('status') != 'acknowledged' )
            return;
          getUserMedia(
            {
            video: {mandatory: {maxWidth: 640, maxHeight: 480}},
            audio: false
            },
            function(stream) {
              av.accept(stream);
              console.log('INFO: Accepted request with', stream);
            },
            function(e) {
              console.log(e);
            }
          );
          console.log('INFO: Getting user media.');
        }
      );
    }

    function authCB(authenticated) {
      if ( authenticated ) {
        IPCortex.PBX.startPoll(
          function() {
            if ( ! IPCortex.PBX.enableFeature('av', avCB, ['chat']) )
              console.log('ERROR: av not enabled!');
            /* Set myself online */
            IPCortex.PBX.enableChat(function() { });
          },

          function(number, description) {
            console.log('ERROR: API reports ' + description + '!');
          }
        );
        console.log('INFO: Authenticated.');
      } else
        console.log('ERROR: Failed to authenticate!');
    }
    onAPILoadReady = (
      function() {
        IPCortex.PBX.Auth.login(username, password, null, authCB);
      }
    );
  }
);

Next I needed Chromium to start automatically on boot, I cheated a little bit by using cron. I’m not overly familiar with systemd so writing a startup script didn’t seem a priority with the time scale involved. I added the following to crontab:

@reboot /usr/bin/xvfb-run –wait=15 /usr/bin/chromium –use-fake-ui-for-media-stream –disable-default-apps –remote-debugging-port=9222 –user-data-dir=remote-profile http://

localhost &

Chromium required a few switches to allow it to run headless:

1) To stop Chromium asking for permission to access the camera:

–use-fake-ui-for-media-stream

 

2) To stop Chromium asking to be set as default:

–disable-default-apps

 

3) For remote debugging (it only listens on loopback):

–remote-debugging-port=9222

 

4) Place the users Chromium profile in a defined location:

–user-data-dir

At this point I started running into trouble with the camera. Every time I started up Chromium I could only get a maximum resolution of 16×16 no matter what v4l2-ctl commands I ran, which wasn’t going to be a good experience. After quite a lot of searching I found the solution and added the following to /etc/modprobe.d/bcm2835.conf:

options bcm2835-v4l2 gst_v4l2src_is_broken=1

 

We needed to serve everything over https as Rob was going to be in London and I would be back in Buckinghamshire flying the quad. That caused me another headache as you can’t load secure and insecure content in the same page. I setup lighttpd to serve pages via https using a self-signed certificate for localhost. Due to Chromium running headless I couldn’t accept the certificate security warnings; I needed access to the Xvfb instance. Installing x11vnc enabled access to the X display. I started the service using the following command on the Pi:

# x11vnc -localhost -display :99

By default xvfb-run starts on display 99. I port forwarded VNC via SSH:

# ssh [email protected](hostname) -L 5900:127.0.0.1:5900

Then I connected using vncviewer to localhost; this allowed me to import the localhost certificate into Chromium’s certificate authority to stop the security warnings.

I settled on netctl to setup the wireless network as this was quick and easy, after having a bit of a nightmare with an access point I borrowed from work I ended up using an old Sky router I had lying around.

keevio eye - the Pi in the sky
keevio eye mk II: no zip ties in sight!
keevio eye - the Pi in the sky
Special lightweight case and minimal gubbins inside due to payload limitations

Finally I put everything together. Feeding power from the balanced charging port of the LiPo battery to a 5V UBEC into the Pi’s GPIO interface. In the process, I managed to accidentally reverse the polarity into the GPIO… which felt like game over as it was now midday Saturday. Luckily something in the supply saved me and it was OK. Attaching the Pi to the quad was an engineering challenge in itself but inventive use of zip ties and self adhesive pads worked out. After a quick test run we got clean video up to 150M and still received video up to 300M.

Here’s a quick video of keevio eye in action!

Previous posts from the ipcortex WebRTC week:

Wormholes, WebRTC and the implications of algorithmical analysis

Matrix.org: Defragmenting today’s communications

WebRTC – where are the real world applications?

Welcome to ipcortex WebRTC week on trefor.net

Check out all our WebRTC posts here

Categories
Apps Engineer video voip

Thunderbirds are Go for Raspberry Pi

Pal Dean Elwood sent me this video. He set up embedded VoIP app developer Voxygen which is going from strength to strength. They developed the call control APIs for Telefonica’s Blue Via platform.

The video shows a mockup Thunderbird video screen just like they had in real life. The app is powered by Voxygen software running on a Raspberry Pi and talks to Blue Via for the call control and conferencing.

State information from Blue Via feeds through to the flashing eyes to say when a call is coming in or has been answered. The RaspberryPi manages another LED to say who is speaking at any given point in time.

When they recorded the Thunderbirds series the technology was of course all purely imaginary. Now it is pure simplicity.

Hardware was simply

  • Raspberry Pi
  • The ‘slice of PI/O’ board, to interface to 10 LEDs and 5 switches
  • 5 push-buttons with integrated LED
  • 5 pairs of White-LED ‘eyes’
  • A small amplifier and speaker for sound effects
  • Power supply and a few ancillary electronic components

Software was

  • Standard Pi Debian wheezy operating system
  • Application code, written in PHP 5
  • Some interfacing code in C that allows switching of LEDs and sensing of switches

Marvellous innit? Now watch the vid. Thunderbirds are go!

Categories
Engineer gadgets

Raspberry Pi – the real McCoy

say hi to RaspberryPiI kinda like winding down on a Friday afternoon Raspberry Pi fresh out of the boxalthough this week has been so hectic it feels as if I need a couple of extra days to squeeze it all in.

Fridays are usually when I put up the trefor.net megaprize competition. It isn’t too late to have one this week but first I need to talk about Raspberry Pi, for yes, I have been playing with one.

The amazing thing about Raspberry Pi is that it just works out of the box. They have done a great job. The processor is a bit slower than you are used to with graphics but hey – it’s twenty five quid for petes sake.raspberry pi screenshot

The pics in this post say it all really.

The first is just the PCB. It’s almost like a mobile phone without the screen and battery. Then I have a few screens shots with the obligatory picture of the blog in it to show the browser in action.

Second screenshot zooms in to show the menu options and then I’ve shown the development environment in a window.raspberry pi dev environment You can be up and running straight away although you will need to learn some basic programming languages, which is the whole point of the device.

The last photo shows the desktop setup wit ha Raspberry Pi plugged in to an Ethernet cable. Obviously there is only one video card so only one of the screens are used.

It might almost be conceivable to use two processor cards to have two screens. I can certainly see Raspberry Pi being used in many applications around the home and office.Raspberry Pi desktop setup

Obviously the cables would have to be tidied up a bit for every day use and I expect Health and Safety would insist on a box for the processor card.

The video at the bottom of this post shows a screensaver in action. Thanks to Gareth Bryan for letting me have a play. I didn’t get in particularly early with my registration.