Re: Synchronized annotations in a collaborative drawing session

Hi,
We have developed a application in C# using leadtools Multimedia toolkit ver.16. It connects to a live stream and captures videos and images from it.For capturing them ltmmConvertCtrl is used and to view them ltmmPlayCtrl is used.The user can select video or image from captured media files and share it with other clients in the collaboration session.
We want to implement the annotations functionality using which the user can draw on the image/video displayed in ltmmplayer in synchronization with other clients.i.e the annotations drawn by one client will be reflected on the other clients PC at the same time and vice a versa.
We want to know :
1. Whether it is possible to implement such functionality using leadtools annotation library?
2.If not,can we draw on ltmmplayer using some other method.


Thanks & Regards,
Utkarsh.

Hello,

We do not have annotations support integrated into our Multimedia toolkit, but this would be possible with our Document Imaging toolkit and the RasterImageViewer. For the player control to be able to display annotations, the annotations would need to be burned into the image data before the video is rendered. With the RasterImageViewer, the video can be displayed, and then annotations laid overtop of the RasterImageViewer in an annotations container.

Based on your other recent posting (http://support.leadtools.com/CS/forums/29203/ShowPost.aspx), it appears that you may be using an MPEG2-TS for this implementation. With this, you may be able to send the annotation information in the KLV data to every user. One each users machine, you would then extract the annotation data from the KLV, and display the new content.

This will be fairly complex, depending on where the video is coming from, and how you are going to administrate the updating of the annotations. Every user will not be able to modify the annotations being viewed simulataneously. If two people tried modifying annotations at the same time, you are going to run into a lost update problem. If they modify the same annotation (or even different annotations), whomever gets their update in last will be the one that is show. For this reason, you'll need to have one host/driver (implement some type of locking mechanism) so that one person can modify the annotations as he/she chooses, and then every else sees the changes.

I am posting a small sample project that uses LEADTOOLS Multimedia 16 and the LEADTOOLS Document Imaging 16.5 to illustrate how you can hook a player control in with a RasterImageViewer for display with annotations. It uses our annotation automation with the RasterImageViewer. I get the image data from the player control using the callback filter. For additional help on getting working with annotations, please see the help file topic "Programming With LEADTOOLS Annotations". For more information about how to use KLV, please check out the help file topics "ILMKlvBuilder Interface" and "ILMKlvParser Interface".

Hi,
Actually we want to share and update annotations drawn by participants in a drawing collaboration session via server not by transmitting directly to other users.
I mean when one user draws some annotation object it will be passed to other users PC via server and that object is drawn on their viewer.

Is this method feasible or
can u suggest some example/method to do so?

Thanks,
Utkarsh.

Hello Utkarsh,

The only way this would work is to have a single server that is streaming the video to all of the users, and to pass the annotations data in a metadata stream like KLV.

You will need to manage which end-client is allow to update the annotations streaming from the server. If you use annotation automation, you could set the automation object to run-mode for all users who are not editting any of the viewable annotations at the time. The one user who is given the ability to modify these annotations will have the automation in edit mode. For an example of how this would work, I'd recommend checking out the main annotations demo in the Document Imaging SDK. We have a ClickOnce demo here that should also illustrate this:
http://support.leadtools.com/supportportal/clickoncedemos/V16/CSAutomationDemo/CSAutomationDemo.htm

The KLV data is a one-way stream. It will only work from the server to each of the clients. You will need to use some other protocol for updating the annotation information that is getting transmitted from the server. You can use the AnnCodecs Load and Save methods on the client and server for loading the annotation information from a file or stream.

So to summarize, the server will stream MPEG2-TS with KLV data. In among the KLV data will be annotation information. On each client, the video will need to be displayed in a RasterImageViewer like the project I have posted. You'll need to get the annotation information from the KLV data, and update the annotation container as the annotations change on the server.
A single user will be given the ability to update the server at any one time. Although you could allow the users to move the annotations and modify them, only one should have the ability to update the server at any one given time. This way, you won't run into any updating issues. Once the user saves the annotation changes, and transmits them to the server, the server can then update all of the other users.

Regarding KLV, you can read more about the ILMKlvBuilder and ILMKlvParser interfaces in our online documentation here:
http://www.leadtools.com/help/LEADTOOLS/v16/Multimedia/Filters/Interfaces/ILMKlvBuilder.htm
http://www.leadtools.com/help/LEADTOOLS/v16/Multimedia/Filters/Interfaces/MPEG2Demux_ILMKlvParserInterface.htm
http://www.leadtools.com/help/LEADTOOLS/v16/Multimedia/Filters/Interfaces/MPEGDemux_KLVKeyInformation.htm