History

« Previous - Version 14/18 (diff) - Next » - Current version
Anonymous, 05/07/2010 04:37 pm

<acronym title="Design*, depth=1">TOC</acronym>

= Video blueprint =

Design and implement Ticket [ticket:18]

Goals

The goal is to implement the support for video session is the SIPSIMPLE middleware. For the video encoding/decoding VLC library shall be used, which supports H.264 codec (first codec to be supported).

The video stream should behave like the audio stream at the transport level: SRTP and ICE should be usable. It must comply with the IMediaStream interface, the same way AudioStream does.

Integration

In the picture below the integration of the video support into SIPSIMPLE is shown divided into it's different components:

Image(video_design.png)

  • PJSIP: a new transport is needed. Instead of creating a whole new transport, which would require to implement SRTP and ICE again, a transport adapter will be implemented. A transport adapter sits between a real transport and a pjmedia_stream. To the stream, this adapter will look like a media transport, and to existing media transport, this adapter will look like a stream. The benefit of this approach is we can use the same adapter for both kind of media transports, that is the UDP and ICE media transport. This is exactly the approach that was taken for SRTP. This transport adapter will be responsible for encoding/decoding the video information.
  • Middleware:
    • VideoTransport: the VideoTransport will use RTPTransport to carry the video data and will be responsible for building the SDP for the video stream. A new option will be added to RTPTransport so that it starts the video transport adapter instead of the regular transport when needed.
    • VideoStream: implements IMediaStream interface. Will export a plugable mechanism so that the application layer can access the video data and display it in a window for example, similar to ExternalVNCViewer on MSRP streams.
  • Application: the application will receive the video data from the stream and 'paint' it on a window.
Video acquisition

First approach will be to create pjmedia_videostream object which will do the video acquisition at low level and pass it to transport_video.

Roadmap

This milestones should be achieved in order to get video working:

  • SDP negotiation: make SIPSIMPLE able to generate and negotiate a valid video SDP.
  • Null video: SIPSIMPLE will generate a valid RTP payload with dummy data that will be exchanged after a successful SDP negotiation.
  • Video reception an still image sending: add the ability to receive and display the remote video stream and generate a valid video stream from an still image.
  • Video acquisition: send real video data.
Tools

libVLC will be used for encoding and decoding the video data. It has ctypes based Python bindings, that should be used at the application level to display remote video. Acquisition will most likely be done in C.

A libVLC shared library will need to be built with all necessary module statically linked: h264 encoder/decoder, core modules, etc.

videostream-components.png (13.1 kB) Klaus Darilion, 09/01/2009 11:40 pm

sip_video_session.py (1.3 kB) Klaus Darilion, 09/04/2009 03:23 pm

vlcw.py (683 Bytes) Klaus Darilion, 09/04/2009 03:23 pm

videostream.py (1.2 kB) Klaus Darilion, 09/04/2009 03:36 pm

videostream-classes.png (17.9 kB) Klaus Darilion, 09/04/2009 03:37 pm

video_design.png (56.4 kB) , 05/07/2010 04:34 pm