Creating a LEADTOOLS Multimedia Extract Video Frames Application using Sample Targets

Perform the following steps to create and run a multimedia convert application to extract video frames using SampleTargets with the LEADTOOLS Multimedia ConvertCtrl control.

1. Start Visual Studio .NET.

2. Choose File->New->Project... from the menu.

3. In the New Project dialog box, choose either "Visual C# Projects" or "VB Projects" in the Projects Type List, and choose "Windows Application " in the Templates List.

4. Type the project name as "Multimedia Frame Extractor" in the Project Name field, and then click OK. If desired, type a new location for your project or select a directory using the Browse button, and then click OK.

5. In the "Solution Explorer" window, right-click the "References" folder, and select "Add Reference..." from the context menu. In the "Add Reference" dialog box, select the ".NET" tab and select Leadtools.Multimedia and click OK. If this DLL does not appear under the .NET tab, you will need select the Browse tab and add it from the "<LEADTOOLS_INSTALLDIR>\Bin\Dotnet\Win32 " folder, by selecting the following DLL:

   • Leadtools.Multimedia.dll

   After browsing, click the Select button and then press the OK button to add the above DLL to the application.

6. Next, we need to add the using/import directive for this DLL to our Form's source file. Switch Form1 to code view (right-click Form1 in the Solution Explorer then choose View Code) and add the following lines to the beginning of the file:

   VB

   Imports Leadtools.Multimedia 
                     
        

   C#

   using Leadtools.Multimedia; 
                     
        

7. While still in code view, declare the following private variables for use later:

   VB

   Private _sourceFile As String 
   Private _targetPath As String 
                     
        

   C#

   private string _sourceFile; 
   private string _targetPath; 
                     
        

8. Next, we need to add an instance of the ConvertCtrl to the form. Make sure Form1 is in design view. Go to the toolbox (View->Toolbox) and drag a ConvertCtrl control on the form. NOTE: If you do not have ConvertCtrl in your toolbox, select Tools->Choose Toolbox Items from the menu. Click Browse and then select Leadtools.Multimedia.dll from "<LEADTOOLS_INSTALLDIR>\Bin\Dotnet\Win32" and then click Open and then click OK. After adding it to the form, set the following properties on the convert control:

   Property	Value
   Name	_convertctrl
   Anchor	Top, Bottom, Left, Right
   BackColor	Black

9. Go to the toolbox (View->Toolbox) and drag a ProgressBar control on the form (below the convert control) and set the following properties:

   Property	Value
   Name	_progress
   Anchor	Bottom, Left, Right
   Step	1

10. Next, we will add a text control and browse button for the source video file selection. Go to the toolbox (View->Toolbox) and drag a Text control and a Button control to the form (below the progress bar) and set the following properties:

    Property	Value
    Name	_textSrcFile
    Anchor	Bottom, Left, Right
    ReadOnly	True
    BackColor	ControlLightLight
    Name	_buttonBrowseSrc
    Text	Source...
    Anchor	Bottom, Right

11. Next, we will add another text control and browse button to select the output folder for extracted frames. Go to the toolbox (View->Toolbox) and drag a Text control and a Button control to the bottom of the form and set the following properties:

    Property	Value
    Name	_textTargetPath
    Anchor	Bottom, Left, Right
    ReadOnly	True
    BackColor	ControlLightLight
    Name	_buttonBrowseTarget
    Text	...
    Anchor	Bottom, Right

12. Now, we need one final button to start the conversion/frames extraction work. Go to the toolbox (View->Toolbox) and drag a Button control to the bottom of the form and set the following properties:

    Property	Value
    Name	_buttonConvert
    Text	Extract
    Anchor	Bottom, Right

13. While still in design view, select the properties for the form and add an event handler to the Form1 Load event. Code this event handler as follows:

    VB

    Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load 
       _sourceFile = "<LEADTOOLS_INSTALLDIR>\Media\DaDa_DVD_MPEG2.mpg" 
       _targetPath = Path.Combine(Directory.GetCurrentDirectory(), "Frames") 
       If (Not Directory.Exists(_targetPath)) Then 
          Directory.CreateDirectory(_targetPath) 
       End If 
       _textSrcFile.Text = _sourceFile 
       _textTargetPath.Text = _targetPath 
    End Sub 
                      
         

    C#

    private void Form1_Load(object sender, System.EventArgs e) 
    { 
       _sourceFile = @"<LEADTOOLS_INSTALLDIR>\Media\DaDa_DVD_MPEG2.mpg"; 
       _targetPath = Path.Combine(Directory.GetCurrentDirectory(), "Frames"); 
       if (!Directory.Exists(_targetPath)) 
          Directory.CreateDirectory(_targetPath); 
       _textSrcFile.Text = _sourceFile; 
       _textTargetPath.Text = _targetPath; 
    } 
                      
         

14. Also, add an event handler to the _convertctrl Progress event and code it as follows:

    VB

    Private Sub _convertctrl_Progress(ByVal sender As System.Object, ByVal e As ProgressEventArgs) Handles _convertctrl.Progress 
       _progress.Value = CInt((_progress.Maximum * e.percent / 100)) 
    End Sub 
                      
         

    C#

    private void _convertctrl_Progress(object sender, ProgressEventArgs e) 
    { 
       _progress.Value = (int)(_progress.Maximum * e.percent / 100); 
    } 
                      
         

15. Now, add an event handler to the _convertctrl Complete event to let us know when the conversion/extraction process is completed. Code this event handler as follows:

    VB

    Private Sub _convertctrl_Complete(ByVal sender As System.Object, ByVal e As EventArgs) Handles _convertctrl.Complete 
       MessageBox.Show("Frame Extraction Complete") 
       _buttonBrowseSrc.Enabled = True 
       _buttonBrowseTarget.Enabled = True 
       _buttonConvert.Enabled = True 
    End Sub 
                      
         

    C#

    private void _convertctrl_Complete(object sender, EventArgs e) 
    { 
       MessageBox.Show("Frame Extraction Complete"); 
       _buttonBrowseSrc.Enabled = true; 
       _buttonBrowseTarget.Enabled = true; 
       _buttonConvert.Enabled = true; 
    } 
                      
         

16. Add an event handler to the _buttonBrowseSrc Click event to select the source video file for processing. Code this event handler as follows:

    VB

    Private Sub _buttonBrowseSrc_Click(ByVal sender As Object, ByVal e As EventArgs) Handles _buttonBrowseSrc.Click 
       Dim ofn As OpenFileDialog = New OpenFileDialog() 
       ofn.FileName = _sourceFile 
       ofn.FilterIndex = 0 
                     
       If _textSrcFile.Text <> String.Empty Then 
          ofn.InitialDirectory = Path.GetDirectoryName(_textSrcFile.Text) 
       Else 
          ofn.InitialDirectory = Directory.GetCurrentDirectory() 
       End If 
                     
       ofn.Multiselect = False 
       ofn.Title = "Select Source Video" 
       ofn.CheckFileExists = True 
                     
       ofn.Filter = "MPEG2 Video Files (*.mpg)|*.mpg |   All files (*.*)|*.*" 
                     
       If ofn.ShowDialog() = DialogResult.OK Then 
          _textSrcFile.Text = ofn.FileName 
          _sourceFile = _textSrcFile.Text 
       End If 
    End Sub 
                      
         

    C#

    private void _buttonBrowseSrc_Click(object sender, EventArgs e) 
    { 
       OpenFileDialog ofn = new OpenFileDialog(); 
       ofn.FileName = _sourceFile; 
       ofn.FilterIndex = 0; 
                     
       if (_textSrcFile.Text != string.Empty) 
          ofn.InitialDirectory = Path.GetDirectoryName(_textSrcFile.Text); 
       else 
          ofn.InitialDirectory = Directory.GetCurrentDirectory(); 
                     
       ofn.Multiselect = false; 
       ofn.Title = "Select Source Video"; 
       ofn.CheckFileExists = true; 
                     
       ofn.Filter = "MPEG2 Video Files (*.mpg)|*.mpg |   All files (*.*)|*.*"; 
                     
       if (ofn.ShowDialog() == DialogResult.OK) 
       { 
          _textSrcFile.Text = ofn.FileName; 
          _sourceFile = _textSrcFile.Text; 
       } 
    } 
                      
         

17. Next, add an event handler to the _buttonBrowseTarget Click event to handle browsing for the output path where the extracted video frames will be saved. Code this event handler as follows:

    VB

    Private Sub _buttonBrowseTarget_Click(ByVal sender As Object, ByVal e As EventArgs) Handles _buttonBrowseTarget.Click 
       Dim fbd As FolderBrowserDialog = New FolderBrowserDialog() 
       fbd.SelectedPath = _targetPath 
       fbd.ShowNewFolderButton = True 
       fbd.Description = "Frames Output Folder" 
       fbd.RootFolder = Environment.SpecialFolder.MyComputer 
                     
       If fbd.ShowDialog() = DialogResult.OK Then 
          _targetPath = fbd.SelectedPath 
          _textTargetPath.Text = _targetPath 
       End If 
    End Sub 
                      
         

    C#

    private void _buttonBrowseTarget_Click(object sender, EventArgs e) 
    { 
       FolderBrowserDialog fbd = new FolderBrowserDialog(); 
       fbd.SelectedPath = _targetPath; 
       fbd.ShowNewFolderButton = true; 
       fbd.Description = "Frames Output Folder"; 
       fbd.RootFolder = Environment.SpecialFolder.MyComputer; 
                     
       if (fbd.ShowDialog() == DialogResult.OK) 
       { 
          _targetPath = fbd.SelectedPath; 
          _textTargetPath.Text = _targetPath; 
       } 
    } 
                      
         

18. Finally, add an event handler to the _buttonConvert Click event to perform the conversion/extraction. Code this event handler as follows:

    VB

    Private Sub _buttonConvert_Click(ByVal sender As Object, ByVal e As EventArgs) Handles _buttonConvert.Click 
       Try 
          ExtractFrames() 
          _buttonBrowseSrc.Enabled = False 
          _buttonBrowseTarget.Enabled = False 
          _buttonConvert.Enabled = False 
       Catch ex As Exception 
          MessageBox.Show(Me, ex.Message) 
       End Try 
    End Sub 
                      
         

    C#

    private void _buttonConvert_Click(object sender, EventArgs e) 
    { 
       try 
       { 
          ExtractFrames(); 
          _buttonBrowseSrc.Enabled = false; 
          _buttonBrowseTarget.Enabled = false; 
          _buttonConvert.Enabled = false; 
       } 
       catch (Exception ex) 
       { 
          MessageBox.Show(this, ex.Message); 
       } 
    } 
                      
         

19. Now, with all the forms, elements, and events setup out of the way, we can write the code behind the ExtractFrames method. This is where the real work of starting the conversion, looping through all the sample frames and extracting them to bitmap files is done. Add the following methods and code as follows:

    VB

    Public Sub ExtractFrames() 
       ' input file and output files 
       Dim sampleFileName As String 
                     
       ' This example demonstrates how to use SampleTarget and MediaSample 
       ' objects to directly access sample frames from a conversion graph. 
                     
       Try 
          ' set the source and target files 
          _convertctrl.SourceFile = _sourceFile 
                     
          ' set the preview 
          _convertctrl.Preview = True 
                     
          ' create a new sample target 
          Dim st As SampleTarget = New SampleTarget() 
                     
          ' set the target media type for the video stream 
          Dim mtTarget As MediaType = New MediaType() 
          mtTarget.Type = Leadtools.Multimedia.Constants.MEDIATYPE_Video 
          mtTarget.SubType = Leadtools.Multimedia.Constants.MEDIASUBTYPE_RGB24 
                     
          ' set the sample target's accepted media type 
          st.SetAcceptedMediaType(mtTarget) 
                     
          ' assign the sample target to the capture control 
          _convertctrl.TargetObject = st 
          _convertctrl.TargetFormat = TargetFormatType.StillImage 
                     
          ' run the convert 
          _convertctrl.StartConvert() 
                     
          Dim msFrame As MediaSample = st.GetSample(1000) 
          Dim sampleStart, sampleStop As Long 
                     
          Do While Not msFrame Is Nothing 
             ' get the sample target's connected media type 
             Dim mtSample As MediaType = st.GetConnectedMediaType() 
                     
             ' get the sample times 
             msFrame.GetTime(sampleStart, sampleStop) 
                     
             ' create the file name for this frame bitmap 
             sampleFileName = String.Format("{0}\Frame_{1}-{2}.bmp", _targetPath, sampleStart, sampleStop) 
                     
             ' create the bitmap for this frame 
             WriteSampleBitmap(sampleFileName, msFrame, mtSample) ' mtCopy); 
                     
             Try 
                ' create a media sample using the captured sample from above  
                msFrame = st.GetSample(1000) 
             Catch cex As COMException 
                ' if we have reached the end of stream we are finished 
                If cex.ErrorCode = CInt(ErrorCode.VFW_E_SAMPLE_REJECTED_EOS) OrElse cex.ErrorCode = CInt(ErrorCode.VFW_E_WRONG_STATE) Then 
                   Exit Do 
                ElseIf cex.ErrorCode = CInt(ErrorCode.VFW_E_TIMEOUT) Then 
                   Continue Do 
                Else 
                   Throw cex 
                End If 
             End Try 
          Loop 
       Catch e1 As Exception 
          ' fail! 
       End Try 
    End Sub 
                     
    Private Function WriteSampleBitmap(ByVal outFile As String, ByVal ms As MediaSample, ByVal mt As MediaType) As Bitmap 
       ' get the video information  
       Dim vih As VideoInfoHeader = mt.GetVideoFormatData() 
                     
       ' create a bitmap to hold the sample and copy it 
       Dim bmp As Bitmap = New Bitmap(vih.bmiHeader.biWidth, vih.bmiHeader.biHeight, FormatFromBitCount(vih.bmiHeader.biBitCount)) 
       Dim bmpData As BitmapData = bmp.LockBits(New Rectangle(0, 0, bmp.Width, bmp.Height), ImageLockMode.ReadWrite, bmp.PixelFormat) 
       Marshal.Copy(ms.Buffer, 0, bmpData.Scan0, GetBitmapSize(bmp, vih.bmiHeader.biBitCount)) 
       bmp.UnlockBits(bmpData) 
                     
       ' flip the upside down buffer 
       bmp.RotateFlip(RotateFlipType.Rotate180FlipX) 
                     
       ' save the image 
       bmp.Save(outFile, ImageFormat.Bmp) 
       Return bmp 
    End Function 
                     
    Private Function FormatFromBitCount(ByVal bitCount As Integer) As PixelFormat 
       Select Case bitCount 
          Case 8 
             Return PixelFormat.Format8bppIndexed 
          Case 16 
             Return PixelFormat.Format16bppRgb555 
          Case 32 
             Return PixelFormat.Format32bppRgb 
          Case 48 
             Return PixelFormat.Format48bppRgb 
          Case 24 
             Return PixelFormat.Format24bppRgb 
       End Select 
       Throw New Exception("Unrecognized bit count") 
    End Function 
                     
    Private Function GetBitmapSize(ByVal bmp As Bitmap, ByVal bitCount As Integer) As Integer 
       Dim pixelSize As Integer = CInt(Math.Log(CDbl(bitCount))) 
       Return (bmp.Width * pixelSize + pixelSize And (Not 3)) * bmp.Height 
    End Function 
                     
    Private Function GetBitmapScanRowSize(ByVal bmpSize As Integer, ByVal stride As Integer, ByVal width As Integer) As Integer 
       Return CInt(bmpSize / (stride / width)) 
    End Function 
                      
         

    C#

    public void ExtractFrames() 
    { 
       // input file and output files 
       string sampleFileName; 
                     
       // This example demonstrates how to use SampleTarget and MediaSample 
       // objects to directly access sample frames from a conversion graph. 
                     
       try 
       { 
          // set the source and target files 
          _convertctrl.SourceFile = _sourceFile; 
                     
          // set the preview 
          _convertctrl.Preview = true; 
                     
          // create a new sample target 
          SampleTarget st = new SampleTarget(); 
                     
          // set the target media type for the video stream 
          MediaType mtTarget = new MediaType(); 
          mtTarget.Type = Constants.MEDIATYPE_Video; 
          mtTarget.SubType = Constants.MEDIASUBTYPE_RGB24; 
                     
          // set the sample target's accepted media type 
          st.SetAcceptedMediaType(mtTarget); 
                     
          // assign the sample target to the capture control 
          _convertctrl.TargetObject = st; 
          _convertctrl.TargetFormat = TargetFormatType.StillImage; 
                     
          // run the convert 
          _convertctrl.StartConvert(); 
                     
          MediaSample msFrame = st.GetSample(1000); 
          long sampleStart, sampleStop; 
                     
          while (msFrame != null) 
          { 
             // get the sample target's connected media type 
             MediaType mtSample = st.GetConnectedMediaType(); 
                     
             // get the sample times 
             msFrame.GetTime(out sampleStart, out sampleStop); 
                     
             // create the file name for this frame bitmap 
             sampleFileName = string.Format("{0}\\Frame_{1}-{2}.bmp", _targetPath, sampleStart, sampleStop); 
                     
             // create the bitmap for this frame 
             WriteSampleBitmap(sampleFileName, msFrame, mtSample); // mtCopy); 
                     
             try 
             { 
                // create a media sample using the captured sample from above  
                msFrame = st.GetSample(1000); 
             } 
             catch (COMException cex) 
             { 
                // if we have reached the end of stream we are finished 
                if (cex.ErrorCode == (int)ErrorCode.VFW_E_SAMPLE_REJECTED_EOS 
     
                   ||   cex.ErrorCode == (int)ErrorCode.VFW_E_WRONG_STATE) 
                   break; 
                else if (cex.ErrorCode == (int)ErrorCode.VFW_E_TIMEOUT) 
                   continue; 
                else 
                   throw cex; 
             } 
          } 
       } 
       catch (Exception) 
       { 
          // fail! 
       } 
    } 
                     
    private Bitmap WriteSampleBitmap(string outFile, MediaSample ms, MediaType mt) 
    { 
       // get the video information  
       VideoInfoHeader vih = mt.GetVideoFormatData(); 
                     
       // create a bitmap to hold the sample and copy it 
       Bitmap bmp = new Bitmap(vih.bmiHeader.biWidth, vih.bmiHeader.biHeight, FormatFromBitCount(vih.bmiHeader.biBitCount)); 
       BitmapData bmpData = bmp.LockBits(new Rectangle(0, 0, bmp.Width, bmp.Height), ImageLockMode.ReadWrite, bmp.PixelFormat); 
       Marshal.Copy(ms.Buffer, 0, bmpData.Scan0, GetBitmapSize(bmp, vih.bmiHeader.biBitCount)); 
       bmp.UnlockBits(bmpData); 
                     
       // flip the upside down buffer 
       bmp.RotateFlip(RotateFlipType.Rotate180FlipX); 
                     
       // save the image 
       bmp.Save(outFile, ImageFormat.Bmp); 
       return bmp; 
    } 
                     
    private PixelFormat FormatFromBitCount(int bitCount) 
    { 
       switch (bitCount) 
       { 
          case 8: 
             return PixelFormat.Format8bppIndexed; 
          case 16: 
             return PixelFormat.Format16bppRgb555; 
          case 32: 
             return PixelFormat.Format32bppRgb; 
          case 48: 
             return PixelFormat.Format48bppRgb; 
          case 24: 
             return PixelFormat.Format24bppRgb; 
       } 
       throw new Exception("Unrecognized bit count"); 
    } 
                     
    private int GetBitmapSize(Bitmap bmp, int bitCount) 
    { 
       int pixelSize = (int)Math.Log((double)bitCount); 
       return (bmp.Width * pixelSize + pixelSize & ~3) * bmp.Height; 
    } 
                     
    private int GetBitmapScanRowSize(int bmpSize, int stride, int width) 
    { 
       return bmpSize / (stride / width); 
    } 
                      
         

20. Now you are ready to Build, and Run the program to test it. Use the Source... button to browse for the source video file and then select the target output folder for frames by using the [...] browse button. Next click the Extract button to perform the conversion/extraction and then open the output folder to view the extracted bitmap files.