Categories
Business fun stuff webrtc

WebRTC: hacking apps for mental health services

WebRTC hacks for social benefit

Last week I explained how we at IPCortex were working with a social enterprise called Founders and Coders to use WebRTC to help solve some social challenges.

The plan was to introduce the ambitious FAC team (16 trainee Javascript developers), to WebRTC via a week long workshop. We’d then support them in using the IPCortex API to quickly put together proof of concept applications for other social enterprises. Afterwards, we’d demo it together at TADHack, the go-to event for devs pushing the boundaries of WebRTC.

TADHack was last weekend and I’m proud to be able to share more about the application we developed, called Confidant, and what we learned during the process.

Developing an idea

The idea we selected comes from a real life requirement brought to us by a charity and an NHS Trust. Their aim was to enhance the provision of youth mental health counselling services remotely: an idea that demonstrates the feasibility of using WebRTC to provide better access to support services. They’d use a community of volunteers on related university courses to provide supervised mentoring services – with the mentors receiving credit for professional experience gained by volunteering their time.

The original intention was to split the development team up and do several different smaller scale hacks. However, the use case for Confidant was very tangible and so well thought out that it immediately caught the team’s imagination. They were excited about making a real difference and decided they wanted to work as one team to deliver the best possible proof of concept hack in the time available.

From zero to demo in 7 days flat

By the time we got to talking about the hack we’d been working with the Founders and Coders students in the WebRTC workshop for a couple of days. I’d seen them working individually or in small groups on some basic WebRTC practical exercises, but wasn’t sure how a huge project with 16 student developers, all working to deliver one application, was going to work. To add to the challenge, they mostly work in React, a technology about which I knew nearly nothing before this week. It looked like I would be learning a lot too.

What came next was a big surprise. Just to recap, this was a team of 16 trainee developers who are about 12 weeks in on an intensive Javascript course. This was their first taste of the real time web and telecoms APIs, and I think also the first time they had worked on a project of this magnitude in a large team. They pretty much immediately, and with no externally obvious single point of leadership, organised themselves into a couple of sub-teams to analyse the requirements and map user journeys for the mentor and client respectively.

The sub teams then presented their results back and a working priority feature list and realistic plan of what was feasible in a couple of days development was quickly produced. A git repository and wiki were there from the start to share information and track issues from the requirements analysis stage. This was by far the most professional hack development process I have ever seen!

Three incredibly intense days later they presented Confidant together at TADHack (video at the bottom of this post). I had the opportunity to present with them about the whole process at the WebRTC Global Summit on Monday, and the positive feedback was overwhelming. You can read a bit more in the Prezi I created for the session.

The next step is to present the application back to the charity customer, and hopefully find some buy-in and resources to start work on taking it a minimum viable product, so that it can be deployed as a pilot to see how it works in real life.

Overall it’s been a fascinating process. We’re delighted to have had the opportunity to use WebRTC for social good and hopefully do our bit to help improve mental health services for young people.

Rob Pickering is CEO of communications company IPCortex and is a good friend of this blog.

Loads of other WebRTC posts here.

Categories
End User webrtc

Hacking WebRTC for Social Benefit

Social Enterprise WebRTC

A few weeks ago, I came across a social enterprise called Founders and Coders. They provide free Javascript development courses in East London via peer learning, mentoring and exposure to projects brought to them by other social enterprises and corporate clients. It’s an innovative response to the skills gap that most of us in the industry are acutely aware of.

I had a chat with the current cohort on the kind of the capabilities that can be introduced into applications using WebRTC. They were fascinated and we quickly hatched a plan for us to run a training workshop with them and following on from that, a development project where we invite third sector organisations to present ideas that Founders and Coders can take forwards into proof of concept hacks.

Yesterday was the first day of the workshop, and we were able to quickly get them up to speed with how WebRTC works and how they can use the IPCortex API to make phone calls and initiate video chat.

We are finishing the workshop today and start work on a project with a really interesting real world social use case first thing tomorrow morning. There is lots to do, but the intention is to take the idea to TADHack London which is conveniently happening over this weekend to work on it a bit further.

We’ve already selected the project from a health charity that we will develop, and I’ll talk a bit more about the it as it starts to unfold, but it is ambitious! None of the FAC students participating will have been exposed to implementing real time communications before the workshop which started yesterday, and by the end of the week they will have hopefully developed a real application from scratch. I don’t think we have ever done anything like this on this kind of timescale before but it is going to be great fun. We just might also generate something that has a lasting impact using communication for social good.

Rob Pickering is CEO of communications company IPCortex and is a good friend of this blog.

Loads of other WebRTC posts here.

Categories
4g Business Mobile ofcom UC webrtc

WebRTC and the mobile reseller opportunity

The WebRTC opportunity for mobile sales dealers

So far in the ipcortex WebRTC week we’ve talked a lot about the impact that WebRTC will have on how we might communicate, as well as exploring some of the technical aspects of the technology. One thing that we’ve not really touched upon is the way that WebRTC will change the commercial comms ecosystem and, being browser based technology, how it will come to affect the mobile business market.

We invited Dave Stephens,  ‎Sales Manager at major O2 dealer Aerial Telephones to share his views on the current challenges in the business mobile market, diversification into unified communications and how WebRTC will impact the delivery of solutions that marry the two.

A changing market

mobile conversationThe business mobile market is in a difficult space right now. Monthly prices are falling whilst handset costs are rising dramatically; a situation made worse in the UK where by and large we still expect to be able to get a free handset with a new contract. Of course we all know the handset is not really free, rather subsidised by the selected tariff, but the result is that many mobile providers only seeing a profit in month 18 onwards.

This differs from  most other countries, where the norm is to select a tariff and then have to purchase the handset separately. While this alternative is beginning to creep into the UK market it’s proving to be a very difficult shift from the “free handset” culture that’s become so ingrained over the last fifteen years.

The business mobile world has also taken a few other hits recently. Non traditional mobile players are making real plans to infringe on the space. WhatsApp are now offering phone calls over 3G, 4G and Wi-Fi, and Google have confirmed their intention to act as an MVNO (in the US at first). Their Project Fi will introduce pay-for-what-you-use data plans, where unused data allowance is credited at the end of the billing cycle. Add to this that within the last few months, Ofcom have proposed a dramatic cap on the price of mobile phone calls between different networks. This will reduce another revenue stream for most UK mobile providers.

For business mobile resellers, there is additional pressure in that many of them have seen their base being attacked by traditional IT or unified comms resellers. It is true that it is far easier for IT or UC resellers to move into the business mobile market than it for a mobile reseller to go the other way, which would take significant investment and upskilling.

Adapt or perish

ChameleonThis all contributes to an environment where companies in the mobile space must adapt or perish. This isn’t limited to resellers, either. It can even be seen at a mobile network operator level where even the big players are beginning to move into some very untraditional services such as hosted telephony, landline services and even hosted IT products.

For the opportunistic and imaginative reseller, however, moving into other areas of business comms like these can present significant benefits and is a challenge worth attempting. “Mobility” is a growing concern within the IT and Telecoms industry right now with many businesses striving to adopt a “work anywhere” approach. We are seeing a clear push to give employees the tools they need to be effective wherever they are. This is ideal for the savvy mobile reseller that has always had this as their core remit.

There are of course issues when looking after a truly mobile unified communications platform. Primarily this is related to the fact that there are 3 core mobile operating systems which are constantly being upgraded, not to mention the 1000s of different handsets that users can choose from, each with their own quirks and nuances. Standard native mobile apps delivered by PBXs produce all kinds of headaches for engineering teams. This is where the development of WebRTC is really exciting as it may negate the need to install, upgrade and manage these difficult situations.

That’s a long way off – not every mobile OS supports WebRTC – but we are watching the progression of the standard with a keen eye.

Previous posts from the ipcortex WebRTC week:

Real Time Campaigning: How will WebRTC and other tech impact elections in 10 years’ time?

Hacking together a WebRTC Pi in the sky – keevio eye

Wormholes, WebRTC and the implications of algorithmical analysis

Matrix.org: Defragmenting today’s communications

WebRTC – where are the real world applications?

Welcome to ipcortex WebRTC week on trefor.net

Check out all our WebRTC posts here

Categories
End User social networking surveillance & privacy webrtc

Real Time Campaigning: How will WebRTC and other tech impact elections in 10 years’ time?

What might a WebRTC enabled democracy & election process look like in 10 years’ time? (Or, technically, 12)

There’s a lot of pre-election stuff that’s the same every year. The campaigning, the squabbles, the gaffes and the villains: they’re all regular plot lines in Britain’s most depressing pantomime. As we go to the polling stations tomorrow, however, we can reflect on 2015 as the year that something did change – the first year that the parties appear keen, rather than reluctant, to embrace technology. We’re seeing as many memes and mashups as we are manifestos; not surprising really as this is, afterall, what many of the traditional media outlets have dubbed “the social media election”.

It’s true that there’s been far more activity on the social media battlefield than ever before (even if they’ve not quite got it right) and it seems that parties are even beginning to use big data – although they’ve a long way to go to replicate the success that Obama had with data in his 2012 campaign. But what role could or should technology play in the elections of the future? What might, say, the 2027 election look like? How might WebRTC play a part in that? Here’s what I imagine might happen…

Every campaign sits on a foundation of micro targeting

TargetIf there’s a question worth asking, in 2027 there’ll be some data that supports the answer. Parties will dedicate greater spend to using big data as the foundation of each campaign – whether that’s in the capture and curation of data relevant to them or analysing it.
This will allow focus of specific campaign messages on certain groups, or even at an individual level. They’ll focus on swing voters, and those within swing constituencies, targeting them with whichever marketing method suits that opportunity, at that time. Meaningful, one-to-one engagement with individual voters will be commonplace, made easier with social media. In addition, these engagements will be more memorable because they’ll use video and other real time comms via WebRTC.
Shaping campaigns in this way has obvious benefits for the parties, but could this type of targeting backfire? Will voters get creeped out and perceive the relevant party in a negative way? Will the long heralded privacy backlash make it too difficult to capture the right data in the first place? Do we rely too much on the integrity of the people to whom we give our data?

Predicting outcomes and campaign agility

With so much data available, much of it collected from social media engagements, will it be easier to predict results?

In the 2012 election in the US, analyst Nate Silver created a model that accurately predicted the winner in every state. Was his success simply due to the fact that Nate was ahead of the curve with the system he was using, and no one had time to react? In 2027, prediction models will have become even more sophisticated and we will see a greater emphasis on doing this in real time. That will then have an effect on parties’ activities and focus throughout the campaign. Each party will need to be agile and have the means to react quickly to changing predictions. Technology like WebRTC could provide another way to communicate with party members, on the ground campaigners or even swing voters in a really quick and effective way.

Real democracy in real time

Electronic systems could allow the public to vote on issues before or as decisions are taken in Parliament. The government paid lip service to using technology to help represent the public’ views with e-petitions, but will they ever be brave enough to open up decision making to registered voters on a regular, or even real time basis? Technology like WebRTC, with its low barrier to delivering enriched comms universally, could potentially be used to allow voters to watch a live debate and then vote at the end. This vote could then shape Members’ opinions or, even, make the decision outright. Would Parliament ever be that bold, and would MP’s accept their role being changed from being a voting representative of a constituency to its steward?

Some governments have already trialled this kind of approach, albeit to shape decision making in advance of its debate. DemocracyOS is an example of this: an open source solution that seeks to provide voters with the means to inform, debate and vote on bills before they are passed. According to them, it’s already been used by the Government of Mexico, the Congress of Buenos Aires, and by some congressmen in the US amongst others. Adopting this kind of approach would be an interesting way to reduce the effectiveness of large companies’ lobbying, and ensuring that airtime in front of MPs isn’t just a question of money and power.

I easily can imagine that forward-thinking councils in the UK, or even individual MPs could use this kind of democratic technology to debate local issues, gaining traction by social media sharing. It would be a welcome alternative to local, “public” consultations that are conducted so discreetly that the public are not properly represented.

Even if government, councils and elected representatives don’t themselves adopt that approach, there are other organisations that seek to make government more democratic from the outside. US startup Placeavote has an interesting model, where site members vote on bills on any range of topics and Placeavote’s candidates will represent the majority of voters. It has failed to gain much traction so far but could prove disruptive given the chance, and I imagine that by 2027 we could have seen someone try a similar approach in the UK.

Reducing expenses, humanising politics and customer service 101

keevio webrtc interfaceIn 2027, MPs will find it much easier to balance their Parliamentary duties with those in their constituency. Technology like WebRTC will mean there’s little excuse to not participate in a debate or vote because they will be able to do so remotely, and there would no longer be the possibility for bills to be passed due to poor scheduling and low turnour. Furthermore, MPs won’t need a second home in London and can spend more time in their constituency.
Internet connectivity will be ubiquitous, as will devices to access it. This means that they can use tech like WebRTC to engage with their constituents in a different way with memorable, multimedia enriched conversations with the same universal reach of the phone systems of the past. For example, elected MPs and their representatives could use this to make their “MP surgeries” more accessible for their constituents by negating the need to travel. They could even adopt a real time “ask me anything” approach during pre-election campaigns.
By 2027, local MPs will have learned lessons from the way that businesses use technology to improve their customer service. Communicating with your MP will be more efficient and timely and, as a result, people will engage with them more than ever before.

The voting process itself

DecisionAn obvious area where technology could improve elections is in the voting process itself. For example, how backwards and archaic is it that we should turn up to a physical location with just a polling card and no verification of identity, yet we already need an online government gateway ID to get a passport? And how secure is it really to leave counts of paper ballots to volunteers? Technology like WebRTC could reduce the technical barrier of providing biomechanical verification in the process.

In addition, increasing the number of people who are registered to vote, and those who actually do place a vote is an ongoing challenge. Technology could make the process of registering and voting more convenient in the hope of increasing participation. To this end, the Political and Constitutional Reform Committee has already proposed that all electors should have the choice to vote online in the UK by 2020. Electronic voting has already been trialled in some countries and so some level of e-voting in the UK by 2027 is not unimaginable – although the experience in Estonia hasn’t actually increased turnout in itself so its effect on this could be in question. Furthermore, whilst paper counting by humans may have its drawbacks, it is very open, auditable and therefore resilient against high level, systematic abuse. Will we ever have the same level of assurance with an electronic vote?

Whatever happens, it’s pretty safe to say that the stage has been set for much wider use of technology during the election process. The challenges will be cultural and institutional – and we’ll be interested to see which parties will be first to adopt real time technologies to make a real difference to the voting public.

Previous posts from the ipcortex WebRTC week:

Hacking together a WebRTC Pi in the sky – keevio eye

Wormholes, WebRTC and the implications of algorithmical analysis

Matrix.org: Defragmenting today’s communications

WebRTC – where are the real world applications?

Welcome to ipcortex WebRTC week on trefor.net

Check out all our WebRTC posts here

Categories
Engineer gadgets webrtc

Hacking together a WebRTC Pi in the sky – keevio eye

WebRTC on a drone

Team ipcortex put together the keevio eye hack for the TADHack London mini hackathon at Idea London on 11-12th April. The idea was to develop a proof of concept for WebRTC running headless on small embedded devices and talking to our keevio video chat interface. Hardly mission critical but TADHack is a load of fun, and a good way of trying stuff out that pushes the technology envelope a bit which inevitably ends up feeding back useful ideas and techniques into our core platforms. There was also a lot riding on this after our success with RTCEmergency at TADHack last year. Matt Preskett is one of our lead developers and the guy behind the hack, and this is his write up of the experience of developing the app.

On the Tuesday evening before TADHack we hadn’t had time to think about possible hacks. We’d been busy with other events and the process of progressing our core keevio platform towards release. A few months ago we were playing with the idea of a WebRTC Raspberry Pi security camera for our bike shed, so as I walked out of the office I suggested, perhaps to my detriment, that it might be a fun idea to use our JavaScript API, running on a Raspberry Pi strapped to the bottom of a quad copter feeding live video via WebRTC…

I did a bit of research on Tuesday evening, but decided with the timescales involved and some of the parts/equipment needed that perhaps we were biting off more than we could chew. Also I wasn’t really sure how I was going to run our API on a headless Raspberry Pi 50ft in the air. Even if that could be overcome I wasn’t sure the ARM processor would be up to the task of decrypting and encrypting the streams.

Wednesday morning I had all but written off the idea. At the time I was working on load testing our UC platform, which required running our API on a headless server. I set about looking into running headless Chromium, and, by the end of Wednesday with the help of Xvfb I had our API running and automatically accepting video chat from keevio.

Thursday was a busy day we didn’t really have an opportunity to discuss the hack. Rob as perhaps a sign of desperation speculatively ordered a Pi and Pi camera.

Friday morning and we still hadn’t concluded what we’d be doing for the hack, after a quick meeting we gave Pi copter (Pi in the sky?) the go ahead. We had just over 48 hours to put all the pieces together. I started off with Raspbian; I don’t really like the extra gumpf that comes with this distribution but I didn’t have time to piece a fresh instance of Debian together. Raspbian only offers Chromium 22 in its repositories; this was when WebRTC was in its infancy. I looked at compiling the latest Chromium, but this would require either a cross compile environment or compiling on the Pi, neither of which I had time for. I looked around again for an alternative distribution and settled on Arch after checking that they offered an up to date version of Chromium for ARM. It’s a bit bleeding edge but more than sufficient for our requirements.

After getting the Pi installed the first thing was to get Chromium to recognise the camera. Chromium talks to video devices through the V4L component of linux.

I inserted the following lines to /boot/config.txt to enable the camera:

gpu_mem=128

start_file=start_x.elf

fixup_file=fixup_x.dat

Then I added the camera module to /etc/modules-load.d/raspberrypi.conf:

bcm2835-v4l2

After rebooting the Pi, udev created a /dev/video0 device, so it was looking good. The next step was to install Chromium, Xvfb and lighttpd. I setup lighttpd to listen on loopback as I was going to be hard coding the username and password into the webpage: not nice but necessary.

This is the JavaScript I wrote for the hack, due to using our API I could keep it short and sweet.


var keevioShare = (
  function(username, password) {

    function avCB(av) {

      console.log('INFO: avCB with', av);

      if ( av.get('existing') )
        return;

      av.hook(
        function() {
          if ( av.get('status') != 'acknowledged' )
            return;
          getUserMedia(
            {
            video: {mandatory: {maxWidth: 640, maxHeight: 480}},
            audio: false
            },
            function(stream) {
              av.accept(stream);
              console.log('INFO: Accepted request with', stream);
            },
            function(e) {
              console.log(e);
            }
          );
          console.log('INFO: Getting user media.');
        }
      );
    }

    function authCB(authenticated) {
      if ( authenticated ) {
        IPCortex.PBX.startPoll(
          function() {
            if ( ! IPCortex.PBX.enableFeature('av', avCB, ['chat']) )
              console.log('ERROR: av not enabled!');
            /* Set myself online */
            IPCortex.PBX.enableChat(function() { });
          },

          function(number, description) {
            console.log('ERROR: API reports ' + description + '!');
          }
        );
        console.log('INFO: Authenticated.');
      } else
        console.log('ERROR: Failed to authenticate!');
    }
    onAPILoadReady = (
      function() {
        IPCortex.PBX.Auth.login(username, password, null, authCB);
      }
    );
  }
);

Next I needed Chromium to start automatically on boot, I cheated a little bit by using cron. I’m not overly familiar with systemd so writing a startup script didn’t seem a priority with the time scale involved. I added the following to crontab:

@reboot /usr/bin/xvfb-run –wait=15 /usr/bin/chromium –use-fake-ui-for-media-stream –disable-default-apps –remote-debugging-port=9222 –user-data-dir=remote-profile http://

localhost &

Chromium required a few switches to allow it to run headless:

1) To stop Chromium asking for permission to access the camera:

–use-fake-ui-for-media-stream

 

2) To stop Chromium asking to be set as default:

–disable-default-apps

 

3) For remote debugging (it only listens on loopback):

–remote-debugging-port=9222

 

4) Place the users Chromium profile in a defined location:

–user-data-dir

At this point I started running into trouble with the camera. Every time I started up Chromium I could only get a maximum resolution of 16×16 no matter what v4l2-ctl commands I ran, which wasn’t going to be a good experience. After quite a lot of searching I found the solution and added the following to /etc/modprobe.d/bcm2835.conf:

options bcm2835-v4l2 gst_v4l2src_is_broken=1

 

We needed to serve everything over https as Rob was going to be in London and I would be back in Buckinghamshire flying the quad. That caused me another headache as you can’t load secure and insecure content in the same page. I setup lighttpd to serve pages via https using a self-signed certificate for localhost. Due to Chromium running headless I couldn’t accept the certificate security warnings; I needed access to the Xvfb instance. Installing x11vnc enabled access to the X display. I started the service using the following command on the Pi:

# x11vnc -localhost -display :99

By default xvfb-run starts on display 99. I port forwarded VNC via SSH:

# ssh root@(hostname) -L 5900:127.0.0.1:5900

Then I connected using vncviewer to localhost; this allowed me to import the localhost certificate into Chromium’s certificate authority to stop the security warnings.

I settled on netctl to setup the wireless network as this was quick and easy, after having a bit of a nightmare with an access point I borrowed from work I ended up using an old Sky router I had lying around.

keevio eye - the Pi in the sky
keevio eye mk II: no zip ties in sight!
keevio eye - the Pi in the sky
Special lightweight case and minimal gubbins inside due to payload limitations

Finally I put everything together. Feeding power from the balanced charging port of the LiPo battery to a 5V UBEC into the Pi’s GPIO interface. In the process, I managed to accidentally reverse the polarity into the GPIO… which felt like game over as it was now midday Saturday. Luckily something in the supply saved me and it was OK. Attaching the Pi to the quad was an engineering challenge in itself but inventive use of zip ties and self adhesive pads worked out. After a quick test run we got clean video up to 150M and still received video up to 300M.

Here’s a quick video of keevio eye in action!

Previous posts from the ipcortex WebRTC week:

Wormholes, WebRTC and the implications of algorithmical analysis

Matrix.org: Defragmenting today’s communications

WebRTC – where are the real world applications?

Welcome to ipcortex WebRTC week on trefor.net

Check out all our WebRTC posts here

Categories
Engineer surveillance & privacy video webrtc

Wormholes, WebRTC and the implications of algorithmical analysis

James Batchelor is Founder and Chief Executive at Alertacall, an organisation which uses neat technology to deliver services which increase human contact with people at risk and are used to improve the lives of many thousands of vulnerable people. Prior to that he was involved in the creation several ventures in the internet service provision, internet retail, telecoms, recruitment and telecare sectors. James has been an ipcortex customer since some of our earliest days and is one of those people who, every time I have the pleasure of chatting to him, I always walk away with a valuable bundle of unique insight. I posed the question to James about the technology impact of WebRTC, and this is what he came back with…

WebRTC meets wormholes

On a long-haul flight in 2001, with the occasionally pungent aroma of reconditioned air in my nostrils and the drone of Rolls Royce engines through my headphones I was transported for a few hours not only to USA – but in to an alternative future. I had the immense pleasure of having time and little else to do but read a novel and a science fiction one too.

The story I read, “The Light of Other Days”, is centred on the discovery of wormhole technology which can be used to pass information instantaneously between points in the space-time continuum. The technology is commercialised by a global media company and used to create the “wormcam” which allows for anything anywhere to be viewed with profound implications for privacy.

As I ponder the applications and implications of WebRTC, and explore its own wormhole like qualities, I wonder whether there are similar impacts for humanity and how the absolute digitisation of our communications streams – coupled with the massive computing power now at our fingertips – could impact upon our own privacy in novel and unexpected ways.

My own company Alertacall is particularly interested in understanding how patterns in the way people communicate with us can indicate a change in their “need”. This is with the positive goal of helping our older customers get the help they need before a situation escalates and becomes materially more difficult to manage. And, as our future products and services start to use WebRTC and other similar communications technologies I wonder what additional data we’ll have at our disposal.

Real-time analysis

I’ve long hypothesised that computers should be able to detect from cameras and other input devices subtle things about human physiology that the human eye cannot, but only had clear evidence of it after stumbling across the fascinating TED talk See invisible motion hear silent sounds.

This talk demonstrates the possibility of detecting heart rate with nothing more than video, by analysing the microscopic movements in our skin caused by pulsating arteries. I wonder how long it is before a methodology to determine skin temperature is devised, or what can be inferred by knowing how quickly someone breathes, blinks or swallows?

In 2012 the mathematician Mr Max Little announced that Parkinson’s symptoms can be detected by using algorithms that analyse voice data. There is also Voice Stress Analysis, which can indicate a range of emotional states including the detection of whether someone is potentially lying. What else could be inferred from a “call”?

But what specifically has this got to do with WebRTC and similar stacks? I suggest that the incredible proximity of these communications streams to silicon provides an unprecedented opportunity to develop applications that exploit all of these methods for causes good and bad. For example: imagine if calls to emergency services were prioritised using real-time analysis of video and voice, where the person most likely to be having a heart attack is answered first.

Also, imagine a world, in which the person or organisation you are in a call with has installed one of the dozens of analysis applications that are likely to emerge – and can infer huge amounts about your physiology. “Mum, I’m absolutely fine” the daughter says to her mother, but moments later the concerned mother’s machine tells her it’s simply not true with a simple Chrome plugin.

We’re tremendously excited about the applications we can build with WebRTC to connect with our customers and to connect our customers to each other – but live in constant wonder about what opportunities will emerge.

 

Previous posts from the ipcortex WebRTC week:

Matrix.org: Defragmenting today’s communications

WebRTC – where are the real world applications?

Welcome to ipcortex WebRTC week on trefor.net

Check out all our WebRTC posts here

Categories
Engineer webrtc

WebRTC – where are the real world applications?

I’ve been working with WebRTC for a few years now and from time to time talk in public about the technology and its potential. A pretty popular question goes along the lines of “that’s all very well, but where is the revenue in it for me?”

Around the time we did our first end-user demo of WebRTC technology making phone calls in 2012, I wrote up an article where I predicted a range of applications for the technology. Thankfully none of the fairly scary scenarios that I painted in that post have come to pass. Standardisation and browser support have been a bit slower than some of us would have liked but spite of this WebRTC has been quietly making inroads into both the traditional communications space and being used to deliver novel new applications. With the Microsoft announcement about WebRTC support in the next version of their browser, and Google’s massive strides in reliability and interoperability in the core WebRTC project, the future of the technology now looks certain.

At ipcortex we’ve worked on a number of WebRTC developments including RTCEmergency, a weekend hack last year to re-imagine the way we do emergency services calls which won a Google prize for innovation at TADHack Madrid and, with feet closer to the ground, our first major commercial WebRTC end-user application keevio which provides a full range of business Unified Communication services into any device with a WebRTC capable browser.

Amazon Mayday and other apps, do they or don’t they?

An interesting property of WebRTC is that in a really well implemented application, a user need not know or care if it is using WebRTC. It is of course really easy to tell if a web-page prompts to use the device camera or microphone in Chrome or Firefox and then delivers in-page audio and video without a 10-minute plugin dance, but for a dedicated mobile app with no web interface, it is impossible to know for sure without resorting to examining the source code or tracing interactions on the wire.

That is how the world found out that the Amazon Mayday service was using WebRTC to provide real time video chat as part of its live support service.

Other consumer communication applications that have more or less publicly adopted WebRTC to deliver real time communications over the past couple of years include:

  • Comcast – streaming personal video between set-top boxes and handhelds using WebRTC
  • AT&T – allowing calls to/from own mobile number via browser & API
  • Google Hangouts – Google are the major force behind WebRTC development and it was seen as a bit of a coming of age for the technology when they publicly announced that their flagship hangouts product was now using it half way through 2014.
  • Facebook Messenger – head over to www.messenger.com using Chrome and start an audio of video call with a Facebook friend. Your conversation just used WebRTC. That is a huge user base.

Now to some extent many of these don’t need to use WebRTC to deliver what they do; all of these companies have sufficient muscle that they could have developed dedicated applications or plugins to achieve the same functionality – albeit in a less usable way. Indeed most of the examples above still do use plugins that have been developed if you access them from a browser that doesn’t include native WebRTC support, so WebRTC is just a way of streamlining certain kinds of access.

That’s the point really, WebRTC isn’t something users care about – it should be invisible. It is the applications you create with it that have the user-visible value.

The value is in the application not the network

Whilst the simple messaging use cases for WebRTC have been early adopters, and nobody could claim that Facebook, or WhatApp are commercially insignificant, their existence has probably closed the door on making vast amounts of cash out of building a simple consumer messaging application with a bit of WebRTC voice and video thrown in.

If that is bad news for a would-be application developer, the good news is that the universal end-to-end capability that WebRTC delivers means that smart applications can still emerge which generate value by streamlining some aspect of communication.

Metcalfe’s Law (smart guy, even if he did have to eat his own words after predicting the Internet would collapse by 1997) says that the value of a telecommunication network is proportional to the square of the number of participants. This was later tweaked for social networks to be closer to n  log(n) for the number of participants, but you get the point – it is a hockey stick curve where biggest network creates vast value and smaller networks have a very low comparative value. It explains why for example Fring sold a couple of years ago for a reported $50m and the WhatsApp acquisition closed out at close to $22bn, it also explains RCS’s commercial failure. It is really hard to build networks that acquire enough scale quickly enough to have significant value.

WebRTC is a bit different as, once browser and device support is complete, it builds a ready rich communication network of “everything on the Internet”. That isn’t by the way just “everything on the Internet with a screen”; we’ve put full implementations of WebRTC applications on a Raspberry Pi and strapped it under a quadcopter running off the flying machine battery (more on that later this week!).

A really important feature of building an application with WebRTC is that you get a huge potential Metcalfe’s Law advantage before you write a line of code (but so does everyone else).

Contextual vs Free Communication

So if there is vast amount of potential derived from intrinsic network size, and one class of basic social communication applications are already stitched up, where will the next killer communication applications, perhaps using WebRTC, come from?

Most new applications succeed because they are either some large factor better than what currently exists (10 times is an oft quoted number), or they solve a universally felt pain point.

Thankfully there are lots of pain points in communication, and it is relatively easy to deliver 10 times the value of a 3KHz phone call. Unlike the personal realm, where some quite good messaging tools not only exist but now dominate, business in many areas still relies pretty heavily on basic communication mechanisms.

Look at how phone calls work. Users sit working on a processing task and in the middle of this a loud intrusive ringing sound comes from a plastic box on their desk. They have just a few seconds to decide whether to respond, and the only choice they have is to ignore it and lose the conversation (or even worse commit to a longer interruption to pick up a voicemail later), or pick it up and be immediately dropped into a high bandwidth synchronous communication with no context. The only information they may have about the reason it is ringing will be the name or number of the caller. We must respond immediately, context switching away from what we are doing or not at all. Depending on the job that you do, just the interruption itself, never mind the actual cost of dealing with the communication has probably cost 5-10 mins of productive work. You really wouldn’t invent a system like that from scratch today, and indeed much of the value in existing business phone systems relate to applying workarounds for these fundamental drawbacks (call queues to make interactions asynchronous for the recipient, screen popping/click to dial to give agent context etc).

Phone calls are then initiated outside of any particular context, and once started are synchronous, demanding the undivided attention of both participants.

It is far easier to initiate communication without moving away from a task flow, and with the benefit of additional context. In this way attention flows naturally and productively between communicating and processing. This is one of the important ways that WebRTC will deliver contextual communication from within other task based systems – dealing with customer support communications within the context of a support application etc. Done properly, because it is web based, this will be entirely seamless and the user will just view communication as another task experience.

Synchronous to Asynchronous

Many folks of my age are conditioned and therefore still obsessed with calling each other, but fast forward to the next generation and they pretty much exclusively run their lives far more effectively on asynchronous messaging, only escalating to realtime (usually group) voice/video when they really want to give some high bandwidth communication their undivided attention. This is way more efficient and allows interleaving and prioritising of communications and processing.

Asynchronous, Synchronous, Free and Contextual communication - a quadrant diagram

Not only will the next business communication apps be primarily contextual, if they want to remove pain points, they will also offer asynchronous communications as the norm with a simple escalation path to high bandwidth, rich synchronous communications like video and screensharing with voice.

So in summary what does all this mean to me if I’m thinking of deploying my first WebRTC based service or application?

  1. Don’t think of it as a “WebRTC service”. That shouldn’t be visible to your users if you do your job properly.
  2. A personal multimedia messaging application for free communication among your own customers is fine, but won’t set the world on fire – you will be competing with WhatsApp, Facebook, appear.in, Google etc and Metcalfe’s law is on their side (unless federation ever happens and don’t bet on that).
  3. If you are looking for a USP, think of integration with a key business process to either massively streamline communication or remove a pain point.
  4. Find a bunch of 16-25 year olds and test your application with them. If it has the key attributes of contextual and asynchronous then it will probably pass muster with them. If it doesn’t, they will wonder what planet you come from.
  5. Don’t think about it for too long – just get on and do it! The time for producing WebRTC toolkits, APIs, test applications and pilots was 2013, it is about delivering polished applications and services ahead of the competition now.

 

Previous posts from the ipcortex WebRTC week:

Welcome to ipcortex WebRTC week on trefor.net

 

Categories
chromebook Engineer media video webrtc

Bandwidth use for Google Hangouts #WebRTC

Was on a WebRTC conference call this morning. I was calling from the Chrome browser in my Chromebook. Volume could have been slightly louder but the quality of the call was terrific. All I did was click on a link and hey presto. I’ll tell you more about it in due course.

We chatted for over half an hour. It wasn’t video as the other participants were using standard SIP phones. We were hooked up through a WebRTC gateway in the (good ole) US of A.

One on the subjects that came up was bandwidth use of video streams when making WebRTC calls. Using a gateway minimises the amount of processing that you have to do locally and also cuts down on the internet bandwidth you need.

Google Hangouts apparently use your laptop/local device to do the video mixing and thus you need more i/o bandwidth. Google tells us that for person to person video hangout the min bandwidth required is 256kbps/512kbps (up/down) and ideally for the best experience 1Mbps/2.5Mbps).

For calls with more than 2 persons the ideal scenario changes to 900kbps/2Mbps. This means that many people living with poor quality ADSL connections will not be able to properly experience the power of Google Hangouts.

It also explains why calls at weekends (that’s when we hangout) to my daughter at Durham University are also poor quality. It has been known for four of us kids to be on the hangout – one in Durham and three in separate rooms in the house in Lincoln (me and the two lads still at home).  We have 7Mbps up in our house but in Durham it is an ADSL connection shared between four in a student house.

Shame really. For the want of a few quid more on the broadband line it could be much better. Students however are always skint and conserve the cash and we should recognise that they are representative of many people in the UK.

With time everyone will be on a faster broadband connection but for the moment, and I know I’m quite likely to get noises of agreement (or maybe just the occasional assenting nod) from readers in rural areas, many still have to live with limitations of their internet connection.

Mind you I’m all right Jack:)

That’s all.

Categories
Engineer webrtc

ITSPA WebRTC workshop at Google Campus

itspa logoChaired the ITSPA WebRTC workshop at Google Campus yesterday. It had a great turnout so there is obviously an appetite to find out a bit more about WebRTC.

I’ve written about WebRTC several times before including here. The workshop comprised of presentations about the technology from Rob Pickering of IPCortex and Peter Dunkley of Crocodile Rich Communications Systems (they need an acronym methinks 🙂 ) followed by some demos (IPCortex, Crocodile and Drum).

In one sense these  demos are not very interesting. They are just showing video calling – no different to skype or google talk et al. The biggest difference is that with WebRTC the “client” is embedded in the web page that you are visiting. No need to download anything to run on your PC or phone. In theory therefore WebRTC could make for far more ubiquitous online real time communications.

WebRTC should facilitate communities of interest. For example if I have a WebRTC service in this blog then it would be easy to set up conference calls and discussions around specific posts – a big enhancement on the current commenting system. It should even be possible to record such discussions and embed them for others to listen to later.

You always hear about the next big thing that is going to kill off the good (?) old fashioned PSTN and WebRTC was mentioned as a contributor to this yesterday. The PSTN is eventually going to die but not for a long time yet. The WebRTC model, like the original Skype is not about minutes.

In any case, the PSTN is slowly moving away from a minutes based model to a fixed price all you can eat one which makes the death or otherwise of the PSTN a moot point.

WebRTC is potentially very interesting though but there is still a lot of work to be done. The standards are far from complete, even to the point of discussion as to which video codec to use. Half the industry wants to use H264 which is an existing and well bedded in codec. Unfortunately this half is also the half that owns all the patents for H264.

The other half supports Google’s push to use its own VP8 codec which it is making available royalty free.  Of course the H264 camp doesn’t like this and Nokia has apparently said that it owns some codecs that are applicable to VP8 in an attempt to stop it being “free”. You could take the view that Nokia won’t be around for much longer but you can’t base the codec decision on that and in any event someone would probably maintain ownership the patents.

For the moment most of us will have to get on making real money with existing products. SIP trunk anyone?

Categories
Engineer voip webrtc

That Alexander Graham Bell moment and WebRTC @IPCortex

I gave a talk at IPCortex’s 10th birthday party bash yesterday. Was really impressed with Rob Pickering’s WebRTC demo. He made a call from a Yealink VoIP phone hanging off the IPCortex PBX to a browser based client, also registered with the PBX.

Then I called the browser client’s DDI from my mobile – surely one of the first PSTN to WebRTC calls (ok ok I know someone else will have probably already done this but it did feel like an “Alexander Graham Bell moment”).

WebRTC is an open source (ie free) project that enables web browsers with Real-Time Communications (RTC) capabilities via simple Javascript APIs. It is supported by Google who have been buying companies with key patents so that they can be made available free of charge to the community.

It is the future of communications. IPCortex are at the front edge of this work and the video below is part of Rob’s demo. We are all having a bit of fun and it was only a very rough and ready implementation but it shows what can be done. Although it is still in the early  stages of evolution expect lots of applications to use the  WebRTC API in time.

Other WebRTC posts you might want to read:

Uber cool WebRTC video conferencing service appear.in

ITSPA WebRTC Workshop at Google Campus