Programming with the MPEG-2 Transport Demultiplexer

The following topics describe how to program with the MPEG-W Transport Demultiplexer:

• Reducing the Streaming Start Tme
• Reducing the Playback Latency
• Updating the First and Last Stream Positions in the Background
• Improving Reliability when Playing Back Corrupted or Noisy H.264 Video Streams

Reducing the Streaming Start Time

Before playback can start, the MPEG-2 Transport Demultiplexer needs to receive Program Map packets describing the elementary streams present in the MPEG-2 stream. The MPEG-2 Transport Demultiplexer will then wait until it finds packets of all the streams that it deems to be important.

The playback will not start until all the important streams are identified or until a certain amount of data has been received and no packets have been encountered for an elementary stream that has not been properly identified. This can cause undesired delays in starting playback because of elementary streams which contain packets at rare intervals.

The Demux provides a mechanism through which you can override this default behavior and inform the Demux which streams are not important enough to delay the playback for. You can make your application listen for the EC_LOADSTATUS (0x43) DirectShow event, which is sent by the Demux when it detects a Program Map packet containing the number and type of elementary streams.

At this point, you can enumerate all the elementary streams using the StreamCount property and the GetStreamInfo method. GetStreamInfo gives you the StreamPID, the StreamType, and the descriptors for each elementary stream.

In general, stream types of 0x02 indicate an MPEG-2 compressed video stream, 0x10 indicate an MPEG-4 Part 2 compressed video stream and 0x1B indicate an MPEG-2 Part 10 (H.264) compressed video stream. But the video stream can be indicated by other StreamType values as well. Refer to the MPEG-2 documentation (ISO/IEC 13818-1) for a complete list of the possible StreamType values.

You can speed up the start of the playback by setting the StreamFlags property to 0 for all the streams you are not interested in. For example, to start the video playback as soon as possible, you can set the StreamFlags to 0 for all the streams that are not video. But keep in mind that if you do this, then the streams that are deemed unimportant might not be rendered. For example, you might not get any audio if the demux does not receive any audio packets until the packets containing information about the video characteristics (width, height) are received and processed.

The EC_LOADSTATUS (0x43) event is sent to the graph with lParam1 set to AM_LOADSTATUS_PROGRAM_PACKET_RECEIVED (0x100). EC_LOADSTATUS is a standard DirectShow event. If you do not have it defined in your headers, you can just define it as 0x43.

Use the IltmmPlay::SetNotifyWindow / IltmmConvert::SetNotifyWindow methods to select which window should receive notification messages. Process the notification event and look for wParam to be set to ltmmPlay_Notify_MediaEvent / ltmmConvert_Notify_MediaEvent. Then cast lParam to a pointer to a ltmmMediaEvent structure. You have received this event if ltmmMediaEvent.lEventCode is set to EC_LOADSTATUS and ltmmMediaEvent.lParam1 is set to AM_LOADSTATUS_PROGRAM_PACKET_RECEIVED.

C++ example:

#define WM_PLAYERNOTIFY (WM_USER + 1000) 
/* Set the notification window and pick the message */ 
m_player->SetNotifyWindow((long) m_hWnd, WM_PLAYERNOTIFY); 
/* Make OnPlayerNotify handle the notification message */ 
ON_MESSAGE(WM_PLAYERNOTIFY, OnPlayerNotify) 
/* Wait for ltmmPlay_Notify_MediaEvent event and see if I have the right one */ 
LRESULT CMainFrame::OnPlayerNotify(WPARAM wParam, LPARAM lParam) 
{ 
   switch(wParam) 
   { 
   case ltmmPlay_Notify_MediaEvent: 
      { 
         ltmmMediaEvent *pEventParams = (ltmmMediaEvent *)lParam; 
         if(pEventParams && pEventParams->lEventCode == EC_LOADSTATUS && pEventParams->Param1 == AM_LOADSTATUS_PROGRAM_PACKET_RECEIVED) 
         { 
            ILMMpgDmx *pMpgDmx = GetMPEG2DemuxInterface(); 
            if(pMpgDmx) 
            { 
               MarkNonVideoStreamsUnimportant(pMpgDmx); 
               pMpgDmx->Release(); 
            } 
         } 
      } 
   } 
   return 0; 
} 
HRESULT MarkNonVideoStreamsUnimportant(ILMMpgDmx *pMpgDmx) 
{ 
   HRESULT hr; 
   char s[200]; 
   long nStreamCount; 
   hr = pMpgDmx->get_StreamCount(&nStreamCount); 
   if(SUCCEEDED(hr)) 
      for(int i = 0; i < nStreamCount; i++) 
   { 
      long nStreamPID, nStreamType; 
      VARIANT descriptor; 
      hr = pMpgDmx->GetStreamInfo(i, &nStreamPID, &nStreamType, &descriptor); 
      if(SUCCEEDED(hr)) 
         if(nStreamType != 0x02 && nStreamType != 0x10 && nStreamType != 0x1B) 
      { 
         /* mark this stream as unimportant, because I want to start streaming as soon as possible */ 
         hr = pMpgDmx->put_StreamFlags(nStreamPID, 0); 
      } 
   } 
   return hr; 
} 

Reducing Playback Latency

If your application needs to be as close to Live as possible, set the AutoLive property to VARIANT_TRUE. Set the AutoLiveTolerance property to the maximum latency (amount of time in seconds) by which you can be behind Live. But keep in mind that setting a low tolerance can cause jerky playback if the demux has to constantly jump ahead to keep up with the live stream. Jerky playback can also occur if the variation between the time the video packets are supposed to be displayed and the time they are received exceeds this maximum latency.

You can ensure smooth playback by setting a minimum latency using the MinBufferingAmount property. The MinBufferingAmount property is the minimum amount of data (in seconds) buffered at the demux level during playback. If the amount of data buffered falls below MinBufferingAmount, the stream is automatically slowed down to reduce the chance of the playback being jerky.

The initial latency is set by seeking or by setting the CurrentStreamTime, CurrentStreamPTS or CurrentStreamOffset property to a negative value. For example, if the CurrentStreamTime is set to -0.5, the initial latency is 0.5s. This initial latency can be increased if the demux detects that sometimes the latency falls below the MinBufferingAmount.

The relationship between the latency values should be as follows: Maximum latency (AutoLiveTolerance) > Initial Latency > Minimum latency (MinBufferingAmount). The latency calculation (comparison to Live) is done at regular intervals set determined by the DurationCheckInterval property. See the discussion about background updating of first and last stream positions, below.

Updating the First and Last Stream Positions in the Background

When running in DVR mode, the demux is recalculating the first and last positions of the current stream using a background thread. The frequency of these checks is indicated by the DurationCheckInterval property. Every DurationCheckInterval seconds, the demux will read the first data in the stream and update the internal first stream position if necessary. After DurationCheckInterval more seconds, it will update the internal last stream position in. DurationCheckInterval seconds later, it will update the internal first stream position again, then the last position and so on. But keep in mind it updates only the internal first and last stream positions, not also the values of the FirstStreamTime/FirstStreamPTS/FirstStreamOffset properties. The values returned by those properties are updated from these internal values only when the RefreshPosition method is called. The reason for that is to make sure that all stream position properties are synchronized and are taken at the same point in time (when RefreshPosition is called). If you never call RefreshPosition, the stream position properties will have outdated values.

There is some penalty hit whenever this thread checks the first and last positions, so do not set the DurationCheckInterval to a value that is too low, (otherwise the playback can become jerky). If you are not interested in these properties and want to disable this background thread, just set DurationCheckInterval to a really high value. This background thread should run at regular intervals when you want the auto live feature to work. The auto live feature works by checking the last stream position every time it is updated by the background thread. But the check for the last position is expensive (it can take a long time), hence sometimes the demux gives up trying to update it. It may do that when it cannot find the last position in a certain amount of bytes and will try again next when it is time to update the last stream position. This is the reason why sometimes the last stream position may stay unchanged for a while and then jump ahead more than the usual. The jump in this situation is done to maintain optimal playback performance and to minimize any interference with the video playback.

Improving Reliability when Playing Back Corrupted or Noisy H.264 Video Streams

By default, the H264 decoder is running in the optimal performance mode. Streams with excessive noise or corruption can cause various problems in this setting. If you have problems playing such bad files or streams, then you should update ILMH264Decoder::DecodingOptions to contain the H264DecodingFlag_DecodeCorruptedStream flag. Optionally, you can set the H264DecodingFlag_DisplayGrayOnError flag as well, or enable the H264 Decoder properties in the settings dialog box as shown in Figure 1.

Figure 1. LEAD H264 Decoder (3.0) Settings Dialog Box

Other topics

ILMMpgDmx Interface
ILMH264Decoder Interface