Creating a LEADTOOLS Multimedia Extract Video Frames Application using Sample Targets

Perform the following steps to create and run a multimedia convert application to extract video frames using SampleTargets with the LEADTOOLS Multimedia ConvertCtrl control.

Start Visual Studio.
Choose File->New->Project... from the menu.
In the New Project dialog box, choose either "Visual C# Projects" or "VB Projects" in the Projects Type List, and choose "Windows Application " in the Templates List.
Type the project name as "Multimedia Frame Extractor" in the Project Name field, and then click OK. If desired, type a new location for your project or select a directory using the Browse button, and then click OK.
In the "Solution Explorer" window, right-click the "References" folder, and select "Add Reference..." from the context menu. In the "Add Reference" dialog box, select the ".NET" tab and select Leadtools.Multimedia and click OK. If this DLL does not appear under the .NET tab, you will need select the Browse tab and add it from the "<LEADTOOLS_INSTALLDIR>\Bin\Dotnet4\Win32 " folder, by selecting the following DLL:
- Leadtools.Multimedia.dll
After browsing, click the Select button and then press the OK button to add the above DLL to the application.
Next, we need to add the using/import directive for this DLL to our Form's source file. Switch Form1 to code view (right-click Form1 in the Solution Explorer then choose View Code) and add the following lines to the beginning of the file:
```
vb 
Imports Leadtools.Multimedia 
```
C#
```
using Leadtools.Multimedia; 
```

While still in code view, declare the following private variables for use later:

vb 
Private _sourceFile As String 
Private _targetPath As String

private string _sourceFile; 
private string _targetPath;

Next, we need to add an instance of the ConvertCtrl to the form. Make sure Form1 is in design view. Go to the toolbox (View->Toolbox) and drag a ConvertCtrl control on the form. NOTE: If you do not have ConvertCtrl in your toolbox, select Tools->Choose Toolbox Items from the menu. Click Browse and then select Leadtools.Multimedia.dll from "<LEADTOOLS_INSTALLDIR>\Bin\Dotnet4\Win32" and then click Open and then click OK. After adding it to the form, set the following properties on the convert control:

Property	Value
Name	_convertctrl
Anchor	Top, Bottom, Left, Right
BackColor	Black

Go to the toolbox (View->Toolbox) and drag a ProgressBar control on the form (below the convert control) and set the following properties:

Property	Value
Name	_progress
Anchor	Bottom, Left, Right
Step	1

Next, we will add a text control and browse button for the source video file selection. Go to the toolbox (View->Toolbox) and drag a Text control and a Button control to the form (below the progress bar) and set the following properties:

Property	Value
Name	_textSrcFile
Anchor	Bottom, Left, Right
ReadOnly	True
BackColor	ControlLightLight
Name	_buttonBrowseSrc
Text	Source...
Anchor	Bottom, Right

Next, we will add another text control and browse button to select the output folder for extracted frames. Go to the toolbox (View->Toolbox) and drag a Text control and a Button control to the bottom of the form and set the following properties:

Property	Value
Name	_textTargetPath
Anchor	Bottom, Left, Right
ReadOnly	True
BackColor	ControlLightLight
Name	_buttonBrowseTarget
Text	...
Anchor	Bottom, Right

Now, we need one final button to start the conversion/frames extraction work. Go to the toolbox (View->Toolbox) and drag a Button control to the bottom of the form and set the following properties:

Property	Value
Name	_buttonConvert
Text	Extract
Anchor	Bottom, Right

While still in design view, select the properties for the form and add an event handler to the Form1 Load event. Code this event handler as follows:

vb 
Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load 
   _sourceFile = "<LEADTOOLS_INSTALLDIR>\Media\DaDa_DVD_MPEG2.mpg" 
   _targetPath = Path.Combine(Directory.GetCurrentDirectory(), "Frames") 
   If (Not Directory.Exists(_targetPath)) Then 
      Directory.CreateDirectory(_targetPath) 
   End If 
   _textSrcFile.Text = _sourceFile 
   _textTargetPath.Text = _targetPath 
End Sub

private void Form1_Load(object sender, System.EventArgs e) 
{ 
   _sourceFile = @"<LEADTOOLS_INSTALLDIR>\Media\DaDa_DVD_MPEG2.mpg"; 
   _targetPath = Path.Combine(Directory.GetCurrentDirectory(), "Frames"); 
   if (!Directory.Exists(_targetPath)) 
      Directory.CreateDirectory(_targetPath); 
   _textSrcFile.Text = _sourceFile; 
   _textTargetPath.Text = _targetPath; 
}

Also, add an event handler to the _convertctrl Progress event and code it as follows:

vb 
Private Sub _convertctrl_Progress(ByVal sender As System.Object, ByVal e As ProgressEventArgs) Handles _convertctrl.Progress 
   _progress.Value = CInt((_progress.Maximum * e.percent / 100)) 
End Sub

private void _convertctrl_Progress(object sender, ProgressEventArgs e) 
{ 
   _progress.Value = (int)(_progress.Maximum * e.percent / 100); 
}

Now, add an event handler to the _convertctrl Complete event to let us know when the conversion/extraction process is completed. Code this event handler as follows:

vb 
Private Sub _convertctrl_Complete(ByVal sender As System.Object, ByVal e As EventArgs) Handles _convertctrl.Complete 
   MessageBox.Show("Frame Extraction Complete") 
   _buttonBrowseSrc.Enabled = True 
   _buttonBrowseTarget.Enabled = True 
   _buttonConvert.Enabled = True 
End Sub

private void _convertctrl_Complete(object sender, EventArgs e) 
{ 
   MessageBox.Show("Frame Extraction Complete"); 
   _buttonBrowseSrc.Enabled = true; 
   _buttonBrowseTarget.Enabled = true; 
   _buttonConvert.Enabled = true; 
}

Add an event handler to the _buttonBrowseSrc Click event to select the source video file for processing. Code this event handler as follows:

vb 
Private Sub _buttonBrowseSrc_Click(ByVal sender As Object, ByVal e As EventArgs) Handles _buttonBrowseSrc.Click 
   Dim ofn As OpenFileDialog = New OpenFileDialog() 
   ofn.FileName = _sourceFile 
   ofn.FilterIndex = 0 
                 
   If _textSrcFile.Text <> String.Empty Then 
      ofn.InitialDirectory = Path.GetDirectoryName(_textSrcFile.Text) 
   Else 
      ofn.InitialDirectory = Directory.GetCurrentDirectory() 
   End If 
                 
   ofn.Multiselect = False 
   ofn.Title = "Select Source Video" 
   ofn.CheckFileExists = True 
                 
   ofn.Filter = "MPEG2 Video Files (*.mpg)|*.mpg |   All files (*.*)|*.*" 
                 
   If ofn.ShowDialog() = DialogResult.OK Then 
      _textSrcFile.Text = ofn.FileName 
      _sourceFile = _textSrcFile.Text 
   End If 
End Sub

private void _buttonBrowseSrc_Click(object sender, EventArgs e) 
{ 
   OpenFileDialog ofn = new OpenFileDialog(); 
   ofn.FileName = _sourceFile; 
   ofn.FilterIndex = 0; 
                 
   if (_textSrcFile.Text != string.Empty) 
      ofn.InitialDirectory = Path.GetDirectoryName(_textSrcFile.Text); 
   else 
      ofn.InitialDirectory = Directory.GetCurrentDirectory(); 
                 
   ofn.Multiselect = false; 
   ofn.Title = "Select Source Video"; 
   ofn.CheckFileExists = true; 
                 
   ofn.Filter = "MPEG2 Video Files (*.mpg)|*.mpg |   All files (*.*)|*.*"; 
                 
   if (ofn.ShowDialog() == DialogResult.OK) 
   { 
      _textSrcFile.Text = ofn.FileName; 
      _sourceFile = _textSrcFile.Text; 
   } 
}

Next, add an event handler to the _buttonBrowseTarget Click event to handle browsing for the output path where the extracted video frames will be saved. Code this event handler as follows:

vb 
Private Sub _buttonBrowseTarget_Click(ByVal sender As Object, ByVal e As EventArgs) Handles _buttonBrowseTarget.Click 
   Dim fbd As FolderBrowserDialog = New FolderBrowserDialog() 
   fbd.SelectedPath = _targetPath 
   fbd.ShowNewFolderButton = True 
   fbd.Description = "Frames Output Folder" 
   fbd.RootFolder = Environment.SpecialFolder.MyComputer 
                 
   If fbd.ShowDialog() = DialogResult.OK Then 
      _targetPath = fbd.SelectedPath 
      _textTargetPath.Text = _targetPath 
   End If 
End Sub

private void _buttonBrowseTarget_Click(object sender, EventArgs e) 
{ 
   FolderBrowserDialog fbd = new FolderBrowserDialog(); 
   fbd.SelectedPath = _targetPath; 
   fbd.ShowNewFolderButton = true; 
   fbd.Description = "Frames Output Folder"; 
   fbd.RootFolder = Environment.SpecialFolder.MyComputer; 
                 
   if (fbd.ShowDialog() == DialogResult.OK) 
   { 
      _targetPath = fbd.SelectedPath; 
      _textTargetPath.Text = _targetPath; 
   } 
}

Finally, add an event handler to the _buttonConvert Click event to perform the conversion/extraction. Code this event handler as follows:

vb 
Private Sub _buttonConvert_Click(ByVal sender As Object, ByVal e As EventArgs) Handles _buttonConvert.Click 
   Try 
      ExtractFrames() 
      _buttonBrowseSrc.Enabled = False 
      _buttonBrowseTarget.Enabled = False 
      _buttonConvert.Enabled = False 
   Catch ex As Exception 
      MessageBox.Show(Me, ex.Message) 
   End Try 
End Sub

private void _buttonConvert_Click(object sender, EventArgs e) 
{ 
   try 
   { 
      ExtractFrames(); 
      _buttonBrowseSrc.Enabled = false; 
      _buttonBrowseTarget.Enabled = false; 
      _buttonConvert.Enabled = false; 
   } 
   catch (Exception ex) 
   { 
      MessageBox.Show(this, ex.Message); 
   } 
}

Now, with all the forms, elements, and events setup out of the way, we can write the code behind the ExtractFrames method. This is where the real work of starting the conversion, looping through all the sample frames and extracting them to bitmap files is done. Add the following methods and code as follows:

vb 
Public Sub ExtractFrames() 
   ' input file and output files 
   Dim sampleFileName As String 
                 
   ' This example demonstrates how to use SampleTarget and MediaSample 
   ' objects to directly access sample frames from a conversion graph. 
                 
   Try 
      ' set the source and target files 
      _convertctrl.SourceFile = _sourceFile 
                 
      ' set the preview 
      _convertctrl.Preview = True 
                 
      ' create a new sample target 
      Dim st As SampleTarget = New SampleTarget() 
                 
      ' set the target media type for the video stream 
      Dim mtTarget As MediaType = New MediaType() 
      mtTarget.Type = Leadtools.Multimedia.Constants.MEDIATYPE_Video 
      mtTarget.SubType = Leadtools.Multimedia.Constants.MEDIASUBTYPE_RGB24 
                 
      ' set the sample target's accepted media type 
      st.SetAcceptedMediaType(mtTarget) 
                 
      ' assign the sample target to the capture control 
      _convertctrl.TargetObject = st 
      _convertctrl.TargetFormat = TargetFormatType.StillImage 
                 
      ' run the convert 
      _convertctrl.StartConvert() 
                 
      Dim msFrame As MediaSample = st.GetSample(1000) 
      Dim sampleStart, sampleStop As Long 
                 
      Do While Not msFrame Is Nothing 
         ' get the sample target's connected media type 
         Dim mtSample As MediaType = st.GetConnectedMediaType() 
                 
         ' get the sample times 
         msFrame.GetTime(sampleStart, sampleStop) 
                 
         ' create the file name for this frame bitmap 
         sampleFileName = String.Format("{0}\Frame_{1}-{2}.bmp", _targetPath, sampleStart, sampleStop) 
                 
         ' create the bitmap for this frame 
         WriteSampleBitmap(sampleFileName, msFrame, mtSample) ' mtCopy); 
                 
         Try 
            ' create a media sample using the captured sample from above  
            msFrame = st.GetSample(1000) 
         Catch cex As COMException 
            ' if we have reached the end of stream we are finished 
            If cex.ErrorCode = CInt(ErrorCode.VFW_E_SAMPLE_REJECTED_EOS) OrElse cex.ErrorCode = CInt(ErrorCode.VFW_E_WRONG_STATE) Then 
               Exit Do 
            ElseIf cex.ErrorCode = CInt(ErrorCode.VFW_E_TIMEOUT) Then 
               Continue Do 
            Else 
               Throw cex 
            End If 
         End Try 
      Loop 
   Catch e1 As Exception 
      ' fail! 
   End Try 
End Sub 
                 
Private Function WriteSampleBitmap(ByVal outFile As String, ByVal ms As MediaSample, ByVal mt As MediaType) As Bitmap 
   ' get the video information  
   Dim vih As VideoInfoHeader = mt.GetVideoFormatData() 
                 
   ' create a bitmap to hold the sample and copy it 
   Dim bmp As Bitmap = New Bitmap(vih.bmiHeader.biWidth, vih.bmiHeader.biHeight, FormatFromBitCount(vih.bmiHeader.biBitCount)) 
   Dim bmpData As BitmapData = bmp.LockBits(New Rectangle(0, 0, bmp.Width, bmp.Height), ImageLockMode.ReadWrite, bmp.PixelFormat) 
   Marshal.Copy(ms.Buffer, 0, bmpData.Scan0, GetBitmapSize(bmp, vih.bmiHeader.biBitCount)) 
   bmp.UnlockBits(bmpData) 
                 
   ' flip the upside down buffer 
   bmp.RotateFlip(RotateFlipType.Rotate180FlipX) 
                 
   ' save the image 
   bmp.Save(outFile, ImageFormat.Bmp) 
   Return bmp 
End Function 
                 
Private Function FormatFromBitCount(ByVal bitCount As Integer) As PixelFormat 
   Select Case bitCount 
      Case 8 
         Return PixelFormat.Format8bppIndexed 
      Case 16 
         Return PixelFormat.Format16bppRgb555 
      Case 32 
         Return PixelFormat.Format32bppRgb 
      Case 48 
         Return PixelFormat.Format48bppRgb 
      Case 24 
         Return PixelFormat.Format24bppRgb 
   End Select 
   Throw New Exception("Unrecognized bit count") 
End Function 
                 
Private Function GetBitmapSize(ByVal bmp As Bitmap, ByVal bitCount As Integer) As Integer 
   Dim pixelSize As Integer = CInt(Math.Log(CDbl(bitCount))) 
   Return (bmp.Width * pixelSize + pixelSize And (Not 3)) * bmp.Height 
End Function 
                 
Private Function GetBitmapScanRowSize(ByVal bmpSize As Integer, ByVal stride As Integer, ByVal width As Integer) As Integer 
   Return CInt(bmpSize / (stride / width)) 
End Function

public void ExtractFrames() 
{ 
   // input file and output files 
   string sampleFileName; 
                 
   // This example demonstrates how to use SampleTarget and MediaSample 
   // objects to directly access sample frames from a conversion graph. 
                 
   try 
   { 
      // set the source and target files 
      _convertctrl.SourceFile = _sourceFile; 
                 
      // set the preview 
      _convertctrl.Preview = true; 
                 
      // create a new sample target 
      SampleTarget st = new SampleTarget(); 
                 
      // set the target media type for the video stream 
      MediaType mtTarget = new MediaType(); 
      mtTarget.Type = Constants.MEDIATYPE_Video; 
      mtTarget.SubType = Constants.MEDIASUBTYPE_RGB24; 
                 
      // set the sample target's accepted media type 
      st.SetAcceptedMediaType(mtTarget); 
                 
      // assign the sample target to the capture control 
      _convertctrl.TargetObject = st; 
      _convertctrl.TargetFormat = TargetFormatType.StillImage; 
                 
      // run the convert 
      _convertctrl.StartConvert(); 
                 
      MediaSample msFrame = st.GetSample(1000); 
      long sampleStart, sampleStop; 
                 
      while (msFrame != null) 
      { 
         // get the sample target's connected media type 
         MediaType mtSample = st.GetConnectedMediaType(); 
                 
         // get the sample times 
         msFrame.GetTime(out sampleStart, out sampleStop); 
                 
         // create the file name for this frame bitmap 
         sampleFileName = string.Format("{0}\\Frame_{1}-{2}.bmp", _targetPath, sampleStart, sampleStop); 
                 
         // create the bitmap for this frame 
         WriteSampleBitmap(sampleFileName, msFrame, mtSample); // mtCopy); 
                 
         try 
         { 
            // create a media sample using the captured sample from above  
            msFrame = st.GetSample(1000); 
         } 
         catch (COMException cex) 
         { 
            // if we have reached the end of stream we are finished 
            if (cex.ErrorCode == (int)ErrorCode.VFW_E_SAMPLE_REJECTED_EOS 
 
               ||   cex.ErrorCode == (int)ErrorCode.VFW_E_WRONG_STATE) 
               break; 
            else if (cex.ErrorCode == (int)ErrorCode.VFW_E_TIMEOUT) 
               continue; 
            else 
               throw cex; 
         } 
      } 
   } 
   catch (Exception) 
   { 
      // fail! 
   } 
} 
                 
private Bitmap WriteSampleBitmap(string outFile, MediaSample ms, MediaType mt) 
{ 
   // get the video information  
   VideoInfoHeader vih = mt.GetVideoFormatData(); 
                 
   // create a bitmap to hold the sample and copy it 
   Bitmap bmp = new Bitmap(vih.bmiHeader.biWidth, vih.bmiHeader.biHeight, FormatFromBitCount(vih.bmiHeader.biBitCount)); 
   BitmapData bmpData = bmp.LockBits(new Rectangle(0, 0, bmp.Width, bmp.Height), ImageLockMode.ReadWrite, bmp.PixelFormat); 
   Marshal.Copy(ms.Buffer, 0, bmpData.Scan0, GetBitmapSize(bmp, vih.bmiHeader.biBitCount)); 
   bmp.UnlockBits(bmpData); 
                 
   // flip the upside down buffer 
   bmp.RotateFlip(RotateFlipType.Rotate180FlipX); 
                 
   // save the image 
   bmp.Save(outFile, ImageFormat.Bmp); 
   return bmp; 
} 
                 
private PixelFormat FormatFromBitCount(int bitCount) 
{ 
   switch (bitCount) 
   { 
      case 8: 
         return PixelFormat.Format8bppIndexed; 
      case 16: 
         return PixelFormat.Format16bppRgb555; 
      case 32: 
         return PixelFormat.Format32bppRgb; 
      case 48: 
         return PixelFormat.Format48bppRgb; 
      case 24: 
         return PixelFormat.Format24bppRgb; 
   } 
   throw new Exception("Unrecognized bit count"); 
} 
                 
private int GetBitmapSize(Bitmap bmp, int bitCount) 
{ 
   int pixelSize = (int)Math.Log((double)bitCount); 
   return (bmp.Width * pixelSize + pixelSize & ~3) * bmp.Height; 
} 
                 
private int GetBitmapScanRowSize(int bmpSize, int stride, int width) 
{ 
   return bmpSize / (stride / width); 
}

Now you are ready to Build, and Run the program to test it. Use the Source... button to browse for the source video file and then select the target output folder for frames by using the [...] browse button. Next click the Extract button to perform the conversion/extraction and then open the output folder to view the extracted bitmap files.