In This Topic ▼

Aborting Loading from the RTSP, ONVIF, and UDP Source Filters

Description of the problem:

The LEAD RTSP Source, LEAD ONVIF Source, and LEAD MPEG2 Transport UDP Source filters will listen for long periods of time to incoming data when a stream is loaded or run. By default, the app will not be processing messages during this time and will be pretty much frozen. Depending on the filter, the app will be frozen when you try to load or run the stream, as follows:

The UDP Source blocks listening for data during the load.
The RTSP Source and ONVIF Source filters block listening for data when you run the stream. But if you load either the RTSP or ONVIF Source filter using the LEADTOOLS Multimedia toolkit and the AutoRun property is set to True, the freeze for the source filter will occur during the load (because the load will automatically run the stream before the load operation finishes).

The freezing will occur if the data does not arrive (for example, if the server you are connecting to is down or is unable to respond in a timely manner). If the data arrives, there is no problem and the load/run process finishes quickly. To avoid blocking the main user interface thread for long periods of time, you have two choices:

Perform the Load/Run operation in another thread
Use the callback mechanism implemented by these filters to receive notifications when no data is arriving. In these notifications, you can abort the wait or process windows messages that unfreeze the app. If you process windows messages, you need to be careful to avoid re-entrancy problems.

The UDP source will listen for 16 seconds for incoming data when you try to load the stream. If nothing comes in during this time, the load will fail. You can change the value of the timeout using the "Timeout" parameter described in UDP Source URL Syntax.

The RTSP Source and ONVIF Source will listen for 16-25 seconds for incoming audio and video data when you run the stream. The most frequent cause for the long wait is waiting for UDP packets that will not make it through when you play from an RTSP server that is outside the local network.

If you use the Play and Convert modules of LEADTOOLS Multimedia, "loading the stream" means setting the SourceFile property. By the time the call to set the SourceFile property returns, the load will either succeed (if data is received) or fail (if data is not received).

If you use the Play and Convert modules of LEADTOOLS Multimedia, "running the stream" means calling IltmmPlay::Run, IltmmConvert::RunConvert or IltmmConvert::StartConvert (or their .NET equivalents, Leadtools.Multimedia.PlayCtrl.Run, Leadtools.Multimedia.ConvertCtrl.RunConvert or Leadtools.Multimedia.ConvertCtrl.StartConvert).

Description of the solution:

There are two ways of solving this problem: a high-level way and a low-level way:

High-level Solution: Listen to the ltmmEC_LOADSTATUS event with the ltmmAM_LOADSTATUS_WAITING_FOR_DATA notification
Low-level Solution: Implement an object that exposes the ILMSrcCallback interface and registers it to the filters using the ILMUDPSrc, ILMONVIFSrc or ILMRTSPSrc interface. The LEADTOOLS Multimedia toolkit automatically does this for you whenever you load an RTSP or UDP stream. The filters will periodically call the ILMSrcCallback interfaces if they have not received any video and audio data. Abort the wait by returning an error from the callback interface.

Use the low-level solution and implement ILMSrcCallback notifications only in the following situations:

If you use DirectShow to build the Play or Convert graph manually
If you create the UDP source , RTSP source or ONVIF source and add it as the source object using the IltmmPlay::put_SourceObject or the IltmmConvert::put_SourceObject.

If you use the Play or Convert modules and load the stream using IltmmPlay::put_SourceFile or IltmmConvert::put_SourceFile, you can just use the high-level solution and do not need to use the interfaces. Whenever you load a URL from the LEAD UDP, RTSP or ONVIF Source filter, LEADTOOLS automatically creates an internal object that implements the ILMSrcCallback interface and registers it with the source filter by setting the ILMUDPSrc::SourceObject, ILMRTSPSrc::SourceObject, or ILMONVIFSrc::SourceObject property to this internal object. Whenever the internal object receives an EVENTTYPE_NODATAAVAILABLE notification, it sends an ltmmPlay_Notify_MediaEvent event to the notification window (this is a window your application creates). The event contains the ltmmEC_LOADSTATUS event and lParam1 gets set to ltmmAM_LOADSTATUS_WAITING_FOR_DATA. lParam2 gets set to the amount of time (in milliseconds) that has elapsed since the UDP, RTSP or ONVIF source started waiting for data.

Note that to avoid re-entrancy problems, only the following methods should be called while you process this ltmmPlay_Notify_MediaEvent event (the other methods may not work as expected if they are called while processing this event notification):

IltmmPlay::ResetSource or IltmmConvert::ResetSource
IltmmPlay::Stop or IltmmConvert::StopConvert (for the RTSP and ONVIF Source if the event is received while processing a call to run the graph).

The pseudo-code for this scenario will look like this for the Play object (let's call it m_player):

Create a window that will listen to notifications (hwndNotify). Let us assume the window proc for this window is CMainFrame::OnPlayerNotify below
Tell the Player to use this notification window: m_player->SetNotifyWindow((long) m_hWnd, WM_PLAYERNOTIFY);
Start the load (for example, with a 30 sec timeout): m_player->put_SourceFile(L"udp://127.0.0.1:9005?Timeout=30");
The load will either succeed (if UDP Source receives data) or fail (if data is not received within 30 seconds or if you cancel the wait by calling m_player->ResetSource)

If the UDP Source is not receiving any data, the notification window will be called every second while step 3 is executing and will call CMainFrame::OnPlayerNotify. The implementation below shows you how to cancel the load after 20 seconds:

LRESULT CMainFrame::OnPlayerNotify(WPARAM wParam, LPARAM lParam) 
{ 
   if(wParam == ltmmPlay_Notify_MediaEvent) 
   { 
      ltmmMediaEvent *pEventParams = (ltmmMediaEvent *)lParam; 
      if(pEventParams && pEventParams->lEventCode == ltmmEC_LOADSTATUS) 
         if(pEventParams->lParam1 == ltmmAM_LOADSTATUS_WAITING_FOR_DATA) 
      { 
         if(pEventParams->lParam2 >= 20000) 
            m_player->ResetSource();   /* abort the load if it takes more than 20 seconds */ 
      } 
   } 
   return 0; 
}

This might not seem straightforward but here is why you need to do it this way: the UDP source does not get created and added to the graph until you call put_SourceFile. So you cannot get the UDP source filter interface before calling put_SourceFile. And you cannot get notifications after put_SourceFile returns because by then it is too late to receive UDP source notifications. There are two situations:

If the UDP source receives data, the graph is built, put_SourceFile succeeds and returns. You can now obtain the UDP source interface, but it is too late to receive notifications (since the load process ended).
If the UDP source is not receiving any data, it will send EVENTTYPE_NODATAAVAILABLE notifications to the ILMSrcCallback. But you cannot get access to the UDP source to set the callback interface since put_SourceFile has not returned yet. You could create another thread while put_SourceFile is running, get the source object, query it for ILMUDPSrc interface, set the callback, etc. But why go to all this trouble, when you can simply listen to the ltmmEC_LOADSTATUS event?

The situation is slightly different for the RTSP Source filter, because the biggest wait occurs when you try to run the graph instead of during the load. So you could load the stream, get the ILMRTSPSrc interface and set the ILMSrcCallback before running the graph. But, still, it is easier to listen to the ltmmEC_LOADSTATUS event.

You can see an example of a window notification implementation in the C++ Player and MPEG-2 Transport Demos. Note that the demo listens for these events, but ignores them. You can just search in the source code for ltmmAM_LOADSTATUS_WAITING_FOR_DATA and add your own cancel code in there to quickly see how this works.