LEADTOOLS Filters Help
| LEADTOOLS Filters Help > Filters, Codecs and Interfaces > Multiplexers and Demultiplexers > Demultiplexers and Splitters > MPEG Demultiplexers > Programming with the MPEG-2 Transport Demultiplexer |
Reduce the streaming start time
Background update of first and last stream positions
Improving reliability when playing corrupted or noisy H.264 video streams
Before playback can start, the MPEG-2 Transport Demultiplexer needs to receive a Program Map packets describing the elementary streams present in the MPEG-2 stream. The MPEG-2 Transport Demultiplexer will then wait until it finds packets of all the streams that it deems to be important. The playback will not start until all the important streams are identified or until a certain amount of data has been received and no packets have been encountered for an elementary stream that has not been properly identified. This can cause undesired delays in starting playback caused by elementary streams which contain packets at rare intervals.
The Demux provides a mechanism through which you can override this default behavior and inform the Demux which streams you do not consider to be important enough to delay the playback for. You may make your application listen for the EC_LOADSTATUS (0x43) DirectShow event, which is sent by the Demux when it detects a Program Map packet containing the number and type of elementary streams. At this point, you can enumerate all the elementary streams using the StreamCount property and the GetStreamInfo method. GetStreamInfo gives you the StreamPID, the StreamType and the descriptors for each elementary stream. In general, stream types of 0x02 indicate an MPEG-2 compressed video stream, 0x10 indicate an MPEG-4 Part 2 compressed video stream and 0x1B indicate an MPEG-2 Part 10 (H.264) compressed video stream. But the video stream can be indicated by other StreamType values as well. Refer to the MPEG-2 documentation (ISO/IEC 13818-1) for a complete list of the possible StreamType values.
You can speed up the start of the playback by setting the StreamFlags property to 0 for all the streams you are not interested in. For example, to start the video playback as soon as possible, you can set the StreamFlags to 0 for all the streams that are not video. But, if you do this, then the streams that are deemed unimportant might not be rendered. For example, you might not get any audio if the demux does not receive any audio packets until the packets containing information about the video characteristics (width, height) are received and processed.
The EC_LOADSTATUS (0x43) event is sent to the graph with lParam1 set to AM_LOADSTATUS_PROGRAM_PACKET_RECEIVED (0x100). EC_LOADSTATUS is a standard DirectShow event. If you do not have it defined in your headers, you can just define it as 0x43.
Use the IltmmPlay::SetNotifyWindow / IltmmConvert::SetNotifyWindow methods to select which window should receive notification messages. Process the notification event and look for wParam to be set to ltmmPlay_Notify_MediaEvent / ltmmConvert_Notify_MediaEvent. You should then cast lParam to a pointer to a ltmmMediaEvent structure. You have received this event if ltmmMediaEvent.lEventCode is set to EC_LOADSTATUS and ltmmMediaEvent.lParam1 is set to AM_LOADSTATUS_PROGRAM_PACKET_RECEIVED.
#define WM_PLAYERNOTIFY (WM_USER + 1000)
/* Set the notification window and pick the message */
m_player->SetNotifyWindow((long) m_hWnd, WM_PLAYERNOTIFY);
/* Make OnPlayerNotify handle the notification message */
ON_MESSAGE(WM_PLAYERNOTIFY, OnPlayerNotify)
/* Wait for ltmmPlay_Notify_MediaEvent event and see if I have the right one */
LRESULT CMainFrame::OnPlayerNotify(WPARAM wParam, LPARAM lParam)
{
switch(wParam)
{
case ltmmPlay_Notify_MediaEvent:
{
ltmmMediaEvent *pEventParams = (ltmmMediaEvent *)lParam;
if(pEventParams && pEventParams->lEventCode == EC_LOADSTATUS && pEventParams->Param1 == AM_LOADSTATUS_PROGRAM_PACKET_RECEIVED)
{
ILMMpgDmx *pMpgDmx = GetMPEG2DemuxInterface();
if(pMpgDmx)
{
MarkNonVideoStreamsUnimportant(pMpgDmx);
pMpgDmx->Release();
}
}
}
}
return 0;
}
HRESULT MarkNonVideoStreamsUnimportant(ILMMpgDmx *pMpgDmx)
{
HRESULT hr;
char s[200];
long nStreamCount;
hr = pMpgDmx->get_StreamCount(&nStreamCount);
if(SUCCEEDED(hr))
for(int i = 0; i < nStreamCount; i++)
{
long nStreamPID, nStreamType;
VARIANT descriptor;
hr = pMpgDmx->GetStreamInfo(i, &nStreamPID, &nStreamType, &descriptor);
if(SUCCEEDED(hr))
if(nStreamType != 0x02 && nStreamType != 0x10 && nStreamType != 0x1B)
{
/* mark this stream as unimportant, because I want to start streaming as soon as possible */
hr = pMpgDmx->put_StreamFlags(nStreamPID, 0);
}
}
return hr;
}
If your application needs to be as close to Live as possible, set the AutoLive property to VARIANT_TRUE. Set the AutoLiveTolerance property to the maximum amount of time (in seconds) and you can be Live behind. But keep in mind that setting a low tolerance might cause jerky playback if the demux has to constantly jump ahead to keep up with the live stream. The comparison to Live is done at regular intervals set determined by the DurationCheckInterval property. See the discussion about background updating of first and last stream positions below.
When running in DVR mode, the demux is recalculating the first and last positions of the current stream using a background thread. The frequency of these checks is indicated by the DurationCheckInterval property. Every DurationCheckInterval seconds, the demux will read the first data in the stream and update the internal first stream position if necessary. After DurationCheckInterval more seconds, it will update the internal last stream position in. DurationCheckInterval seconds later, it will update the internal first stream position again, then the last position and so on. But keep in mind it updates only the internal first and last stream positions, not also the values of the FirstStreamTime/FirstStreamPTS/FirstStreamOffset properties. The values returned by those properties are updated from these internal values only when you call the RefreshPosition method. The reason for that is to make sure the all stream position properties are synchronized and are taken at the same point in time (when RefreshPosition is called). If you never call RefreshPosition, the stream position properties will have outdated values.
There is some penalty hit whenever this thread checks the first and last positions, so you do not want to set DurationCheckInterval to a value that is too low, otherwise the playback might become jerky. If you are not interested in these properties and want to disable this background thread, just set DurationCheckInterval to a really high value.
This background thread should run at regular intervals when you want the auto live feature to work. The auto live feature works by checking the last stream position every time it is updated by the background thread. But the check for the last position is expensive (it can take a long time), hence sometimes the demux gives up trying to update it. It may do that when it cannot find the last position in a certain amount of bytes and will try again next when it is time to update the last stream position. This is the reason why sometimes the last stream position may stay unchanged for a while and then jump ahead more than the usual. The jump in this situation is done to maintain optimal playback performance and to minimize any interference with the video playback.
By default, the H264 decoder is running in the optimal performance mode. Streams with excessive noise or corruption can cause various problems in this setting. If you have problems playing such bad files or streams, then you should update ILMH264Decoder::DecodingOptions to contain the H264DecodingFlag_DecodeCorruptedStream flag. Optionally, you can set the H264DecodingFlag_DisplayGrayOnError flag as well, or enable the H264 Decoder properties in the settings dialog box as shown in Figure 1.
Figure 1. LEAD H264 Decoder (3.0) settings dialog box
