WebRTC Conference

Version 27 (Dan Pascu, 07/07/2017 07:45 pm)

1 18 Adrian Georgescu
h1. SylkServer WebRTC Video Conference
2 2 Adrian Georgescu
3 20 Adrian Georgescu
https://webrtc-test.sipthor.net
4 20 Adrian Georgescu
5 4 Adrian Georgescu
6 2 Adrian Georgescu
h2. Design
7 2 Adrian Georgescu
8 12 Dan Pascu
Two types of conferences are being supported: ad-hoc conferences and moderated conferences.
9 12 Dan Pascu
10 12 Dan Pascu
h3. Ad-hoc conferences
11 12 Dan Pascu
12 12 Dan Pascu
An ad-hoc conference is a conference where all participants have the same status and no one is controlling what other are participants are doing. The participants are rendered in a matrix or up to 3x3 depending of how many participants are in the room. The layout switches automatically for everybody as participants join or leave.
13 12 Dan Pascu
14 12 Dan Pascu
The conference room has a fixed total bitrate configured by the server, that can be specified per room or globally with the max_bitrate setting in webrtcgateway.ini (see below). This bitrate is shared by all participants in the room, meaning that the more participants are in the room, the less bitrate each participant will use for the video stream they send, keeping the total room usage constant to the value configured by max_bitrate. The bitrate adjustment per participant is done automatically by sylkserver as participants join or leave the room, by diving the available bitrate among the number of participants. The end result of this is that each participant will send a fraction of max_bitrate (which is determined by the number of participants in the room) and will always receive a total combined of max_bitrate from all the participants in the room, no matter how many participants are in the room. The formula to compute the bitrate per participant is shown below:
15 12 Dan Pascu
16 12 Dan Pascu
<pre>
17 12 Dan Pascu
participant_send_bitrate = max_bitrate / max(number_of_participants - 1, 1)
18 12 Dan Pascu
</pre>
19 12 Dan Pascu
20 12 Dan Pascu
Using this formula we can make sure that each participant always receives max_bitrate traffic in incoming video streams, independent of the number of participants. The traffic send/received by each party can be expressed like (considering N to be the number of participants and N>1):
21 12 Dan Pascu
22 12 Dan Pascu
<pre>
23 12 Dan Pascu
participant_sent_traffic     = max_bitrate / (N - 1)
24 12 Dan Pascu
participant_received_traffic = max_bitrate
25 12 Dan Pascu
26 23 Dan Pascu
sylkserver_sent_traffic      = max_bitrate * N            (participant_received_traffic * N)
27 23 Dan Pascu
sylkserver_received_traffic  = max_bitrate * N / (N - 1)  (participant_sent_traffic     * N)
28 12 Dan Pascu
</pre>
29 12 Dan Pascu
30 14 Dan Pascu
h3. Moderated conferences
31 14 Dan Pascu
32 14 Dan Pascu
A moderated conference is a conference where a moderator can decide the flow of the conference. The moderator is the first participant to join the conference. The moderator has the ability to see a list with all the participants, can select 1 or 2 of them to be the active speakers and also has the ability to mute other participants (audio and/or video). The moderator can also change the active speakers at any time.
33 12 Dan Pascu
34 16 Dan Pascu
The other participants will see the selected active speakers in full-sized video and the other participants as thumbnails. They will not be able to choose which other participant to watch, the conference view in their browser will be controlled by the moderator that decides who is the active speaker that everybody else sees on their screen in full-sized video.
35 16 Dan Pascu
36 16 Dan Pascu
The active speakers selected by the moderator will have their bitrate set to either max_bitrate (for 1 active speaker) or max_bitrate/2 (for 2 active speakers), while everybody else will have their bitrate set to a low value (64kb/s), just enough to have them represented in small thumbnails on other participant's screens.
37 15 Dan Pascu
38 2 Adrian Georgescu
h2. Features
39 2 Adrian Georgescu
40 17 Dan Pascu
h3. Ad-hoc conferences
41 1 Adrian Georgescu
42 17 Dan Pascu
Ad-hoc conferences are best suited for conversations with family/friends, since bandwidth/bitrate is managed automatically and does not involve a dedicated person to control the flow of the conference. However they can also be used for any other video conferences that imply a free-flowing type of discussion where any participant can jump into the conversation at any time.
43 11 Adrian Georgescu
44 17 Dan Pascu
h3. Moderated conferences
45 11 Adrian Georgescu
46 17 Dan Pascu
Moderate conferences are best suited for a business environment, where participants have to make some sort of presentation in front of the other participants and a moderator is assigned to control the flow of the conference and give the microphone to the appropriate participant, while the others are just watching the active speaker. They can also be used for a conference with 2 active participants that are having a public debate on a subject, while every other participant is just watching it and eventually asking questions.
47 7 Adrian Georgescu
48 2 Adrian Georgescu
h2. Configuration
49 2 Adrian Georgescu
50 10 Dan Pascu
Sylkserver allows the maximum bitrate and video codec to be configured, globally or per room with the following settings in webrtcgateway.ini file:
51 10 Dan Pascu
52 10 Dan Pascu
<pre>
53 10 Dan Pascu
; Maximum video bitrate allowed per sender in a room in bits/s. This value is
54 10 Dan Pascu
; applied to any room that doesn't define its own. The value is any integer
55 10 Dan Pascu
; number between 64000 and 4194304. Default value is 2016000 (~2Mb/s).
56 10 Dan Pascu
; max_bitrate = 2016000
57 10 Dan Pascu
58 10 Dan Pascu
; The video codec to be used by all participants in a room. This value is
59 10 Dan Pascu
; applied to any room that doesn't define its own.
60 10 Dan Pascu
; Possible values are: h264, vp8 and vp9. Default is vp9.
61 10 Dan Pascu
; video_codec = vp9
62 10 Dan Pascu
</pre>
63 10 Dan Pascu
64 10 Dan Pascu
65 1 Adrian Georgescu
h2. Client support
66 1 Adrian Georgescu
67 25 Dan Pascu
h2. Things that were explored
68 25 Dan Pascu
69 1 Adrian Georgescu
h2. Measurements
70 1 Adrian Georgescu
71 21 Adrian Georgescu
These load measurements were done on a Macbook Pro 15" with a 2.3GHz Intel Core I7 CPU, while having 7 participants in the room with each using 336Kb/s. The measurement shows the CPU usage in Firefox web browser with the aforementioned conditions, for the specified video codecs and resolutions which are used by all participants:
72 1 Adrian Georgescu
73 10 Dan Pascu
<pre>
74 10 Dan Pascu
 * H264/VGA - 150% CPU
75 10 Dan Pascu
 * H264/HD  - 250% CPU
76 10 Dan Pascu
 * VP9/VGA  - 220% CPU
77 10 Dan Pascu
 * VP9/HD   - 350% CPU
78 10 Dan Pascu
</pre>
79 6 Adrian Georgescu
80 6 Adrian Georgescu
81 6 Adrian Georgescu
h2. Remaining tasks
82 6 Adrian Georgescu
83 6 Adrian Georgescu
 * sylkserver: control interface for moderator
84 22 Dan Pascu
 * janus: patch to request full frames when a paused video is resumed
85 24 Dan Pascu
 * Rebuild mobile version
86 24 Dan Pascu
87 22 Dan Pascu
h2. Conclusions
88 22 Dan Pascu
89 22 Dan Pascu
h2. Software that was modified
90 26 Dan Pascu
91 26 Dan Pascu
In order to implement the bandwidth management and CPU load optimizations the following software was modified:
92 26 Dan Pascu
93 26 Dan Pascu
# sylkserver https://github.com/AGProjects/sylkserver
94 26 Dan Pascu
# sylk-webrtc https://github.com/AGProjects/sylk-webrtc
95 1 Adrian Georgescu
# sylkrtc.js https://github.com/AGProjects/sylkrtc.js
96 27 Dan Pascu
# python-application https://github.com/AGProjects/python-application
97 27 Dan Pascu
# python-sipsimple https://github.com/AGProjects/python-sipsimple