DesignVideo

Version 12 (Anonymous, 05/07/2010 04:34 pm)

1 2 Adrian Georgescu
[[TOC(Design*, depth=1)]]
2 2 Adrian Georgescu
3 2 Adrian Georgescu
= Video blueprint =
4 3 Adrian Georgescu
5 3 Adrian Georgescu
Design and implement Ticket [ticket:18]
6 4 Saúl Ibarra Corretgé
7 4 Saúl Ibarra Corretgé
== Goals ==
8 4 Saúl Ibarra Corretgé
9 11 Adrian Georgescu
The goal is to implement the support for video session is the SIPSIMPLE middleware. For the video encoding/decoding VLC library shall be used, which supports H.264 codec (first codec to be supported).
10 1 Adrian Georgescu
11 11 Adrian Georgescu
The video stream should behave like the audio stream at the transport level: SRTP and ICE should be usable. It must comply with the IMediaStream interface, the same way AudioStream does.
12 11 Adrian Georgescu
13 11 Adrian Georgescu
14 11 Adrian Georgescu
== Integration ==
15 11 Adrian Georgescu
16 11 Adrian Georgescu
In the picture below the integration of the video support into SIPSIMPLE is shown divided into it's different components:
17 11 Adrian Georgescu
18 11 Adrian Georgescu
(IMAGE)
19 11 Adrian Georgescu
20 11 Adrian Georgescu
 * PJSIP: a new transport is needed. Instead of creating a whole new transport, which would require to implement SRTP and ICE again, a transport adapter will be implemented. A transport adapter sits between a real transport and a pjmedia_stream. To the stream, this adapter will look like a media transport, and to existing media transport, this adapter will look like a stream. The benefit of this approach is we can use the same adapter for both kind of media transports, that is the UDP and ICE media transport. This is exactly the approach that was taken for SRTP. This transport adapter will be responsible for encoding/decoding the video information.
21 11 Adrian Georgescu
22 11 Adrian Georgescu
 * Middleware:
23 12 Adrian Georgescu
24 11 Adrian Georgescu
  * VideoTransport: the VideoTransport will use RTPTransport to carry the video data and will be responsible for building the SDP for the video stream. A new option will be added to RTPTransport so that it starts the video transport adapter instead of the regular transport when needed.
25 12 Adrian Georgescu
26 11 Adrian Georgescu
  * VideoStream: implements IMediaStream interface. Will export a plugable mechanism so that the application layer can access the video data and display it in a window for example, similar to ExternalVNCViewer on MSRP streams.
27 11 Adrian Georgescu
28 11 Adrian Georgescu
 * Application: the application will receive the video data from the stream and 'paint' it on a window.
29 11 Adrian Georgescu
30 11 Adrian Georgescu
== Video acquisition ==
31 4 Saúl Ibarra Corretgé
32 12 Adrian Georgescu
First approach will be to create pjmedia_videostream object which will do the video acquisition at low level and pass it to transport_video.
33 12 Adrian Georgescu
34 1 Adrian Georgescu
== Roadmap ==
35 1 Adrian Georgescu
36 12 Adrian Georgescu
This milestones should be achieved in order to get video working:
37 7 Saúl Ibarra Corretgé
38 12 Adrian Georgescu
 * SDP negotiation: make SIPSIMPLE able to generate and negotiate a valid video SDP.
39 12 Adrian Georgescu
 * Null video: SIPSIMPLE will generate a valid RTP payload with dummy data that will be exchanged after a successful SDP negotiation.
40 12 Adrian Georgescu
 * Video reception an still image sending: add the ability to receive and display the remote video stream and generate a valid video stream from an still image.
41 12 Adrian Georgescu
 * Video acquisition: send real video data.
42 9 Saúl Ibarra Corretgé
43 12 Adrian Georgescu
== Tools ==
44 9 Saúl Ibarra Corretgé
45 12 Adrian Georgescu
libVLC will be used for encoding and decoding the video data. It has ctypes based Python bindings, that should be used at the application level to display remote video. Acquisition will most likely be done in C.
46 1 Adrian Georgescu
47 12 Adrian Georgescu
A libVLC shared library will need to be built with all necessary module statically linked: h264 encoder/decoder, core modules, etc.