In This Topic ▼

Combine Video and Audio from Separate Sources - Console C#

This tutorial shows how to use the LEADTOOLS Multimedia SDK to create a C# Windows Console application that takes two media files, one video and one audio, and combines them into one media file.

Overview
Summary	This tutorial covers how to combine media files in a C# Windows Console application.
Completion Time	30 minutes
Visual Studio Project	Download tutorial project (4 KB)
Platform	Windows C# Console Application
IDE	Visual Studio 2017, 2019
Development License	Download LEADTOOLS

Required Knowledge

Get familiar with the basic steps of creating a project by reviewing the Add References and Set a License tutorial, before working on the Combine Video and Audio from Separate Sources - Console C# tutorial.

Create the Project and Add LEADTOOLS References

Start with a copy of the project created in the Add References and Set a License tutorial. If you do not have that project, follow the steps in that tutorial to create it.

The references needed depend upon the purpose of the project. References can be added using local DLL references, the following DLLs are needed.

The DLLs are located at <INSTALL_DIR>\LEADTOOLS21\Bin\Dotnet4\x64:

Leadtools.dll
Leadtools.Multimedia.dll

For a complete list of which DLL files are required for your LEADTOOLS Multimedia application, refer to Multimedia Files to be Included with your Application.

Set the License File

The License unlocks the features needed for the project. It must be set before any toolkit function is called. For details, including tutorials for different platforms, refer to Setting a Runtime License.

There are two types of runtime licenses:

Evaluation license, obtained at the time the evaluation toolkit is downloaded. It allows the toolkit to be evaluated.
Deployment license. If a Deployment license file and developer key are needed, refer to Obtaining a License.

Note

Adding LEADTOOLS local references and setting a license are covered in more detail in the Add References and Set a License tutorial.

Add the Code to Combine the Media Files

With the project created, the references added, and the license set, coding can begin.

In Solution Explorer, open Program.cs, then add using Leadtools.Multimedia; to the using block at the top.

using System; 
using System.IO; 
using Leadtools; 
using Leadtools.Multimedia;

Add a new method inside the Program class named CombineAudioandVideo(string _videoFile, string _audioFile, string _targetFile). Call this new method inside the Main() method after the call to the SetLicense() method. The string values passed into the new method are going to be the file paths to the video file, audio file, and target file path for the combined media file. Ensure that you add STAThreadAttribute to indicate to the COM components that the application is of single-thread threading model.

[STAThread] 
static void Main(string[] args) 
{ 
    string videoFile = @"FILE PATH TO VIDEO SOURCE FILE"; 
    string audioFile = @"FILE PATH TO AUDIO SOURCE FILE"; 
    string targetFile = @"FILE PATH TO COMBINED OUTPUT MEDIA FILE"; 
 
    SetLicense(); 
    CombineAudioandVideo(videoFile, audioFile, targetFile); 
}

For the purposes of this tutorial, you may use this audio file and this video file.

Add the code below in the CombineAudioandVideo() method to create the SampleTargets for the audio and video files, and start the ConvertCtrls.

static void CombineAudioandVideo(string _videoFile, string _audioFile, string _targetFile) 
{ 
    ConvertCtrl vidConvert = new ConvertCtrl(); 
    ConvertCtrl audConvert = new ConvertCtrl(); 
 
    // Init the SampleTargets. The Video and Audio data from our files will write to these 
    SampleTarget vidTarget = new SampleTarget(); 
    SampleTarget audTarget = new SampleTarget(); 
 
 
    // Set the Media Type of the VideoTarget to Video 
    MediaType mt = new MediaType(); 
    mt.Type = Constants.MEDIATYPE_Video; 
    vidTarget.SetAcceptedMediaType(mt); 
    vidConvert.TargetObject = vidTarget; 
 
    // Clear 
    mt = null; 
 
    // Set the Media type for our AudioTarget to Audio 
    mt = new MediaType(); 
    mt.Type = Constants.MEDIATYPE_Audio; 
    audTarget.SetAcceptedMediaType(mt); 
    audConvert.TargetObject = audTarget; 
 
    // Clear 
    mt = null; 
 
    // Set the ConvertCtrls to point to the Files as their sources 
    vidConvert.SourceFile = _videoFile; 
    audConvert.SourceFile = _audioFile; 
 
    // Start running the two conversion controls. These are writing to the Sample Targets 
    vidConvert.StartConvert(); 
    audConvert.StartConvert(); 
 
    // Enter the Combine Method 
    CombineFiles(vidTarget, audTarget, _targetFile); 
 
    // Stop running the ConvertCtrls 
    if (vidConvert.State == ConvertState.Running) 
        vidConvert.StopConvert(); 
    if (audConvert.State == ConvertState.Running) 
        audConvert.StopConvert(); 
 
    // Dispose of the targets we wrote the data to 
    vidTarget.Dispose(); 
    audTarget.Dispose(); 
 
    // Dispose of the ConvertCtrls that read the file data into the SampleTarget Buffers 
    vidConvert.Dispose(); 
    audConvert.Dispose(); 
}

After the above code has been added, create a new method named CombineFiles(SampleTarget _vidTarget, SampleTarget _audTarget, string _targetFile). This method will be called inside the CombineAudioandVideo() method code, as shown above. Add the code below to the new method to use the ConvertCtrl to combine the two media files.

static void CombineFiles(SampleTarget _vidTarget, SampleTarget _audTarget, string _targetFile) 
{ 
    MultiStreamSource pMSSource; 
    ConvertCtrl combine; 
 
    // Initialize the MultiStreamSource. This is the data that our Combine ConvertCtrl will be reading and then finnally writing to fill 
    // We have two streams. 0 = Video  1 = Audio 
    pMSSource = new MultiStreamSource(); 
    pMSSource.StreamCount = 2; 
 
    // Set the MediaType of the Sources Video Stream to that of the data connected to the VideoTarget 
    MediaType mt = _vidTarget.GetConnectedMediaType(); 
    pMSSource.SetMediaType(0, mt); 
 
    // Clear 
    mt = null; 
 
    // Set the Mediatype of the Sources Audio Stream to that of the data connected to the AudioTarget 
    mt = _audTarget.GetConnectedMediaType(); 
    pMSSource.SetMediaType(1, mt); 
 
    // Clear 
    mt = null; 
 
    // Init the Combine ConvertCtrl that will output our file. This ConvertCtrl will take in the MultiStream Source and output a file on disk 
    combine = new ConvertCtrl(); 
    combine.SourceObject = pMSSource; 
    combine.TargetFile = _targetFile; 
    combine.VideoCompressors.H264.Selected = true; 
    combine.AudioCompressors.AAC.Selected = true; 
    combine.TargetFormat = TargetFormatType.MPEG2Transport; 
 
    // Our MediaSamples. Both a source retreived from our SampleTargets and a destination that will be written to the MultiStreamSource 
    MediaSample pmsSrc = null; 
    MediaSample pmsDst = null; 
 
    long LastStart; 
    long LastStop; 
 
    int lActualDataLength; 
 
    // Begin the running the Combine ConvertCtrl 
    combine.StartConvert(); 
 
    // Video Write 
    while (true) 
    { 
        try 
        { 
            // Get the target sample.  
            // Note if we hit the end of the data stream an exception will trigger and break the loop. This is how we know to stop writing to the buffer 
            pmsSrc = _vidTarget.GetSample(6000); 
            // Get the source buffer 
            pmsDst = pMSSource.GetSampleBuffer(0, 2000); 
        } 
        catch (Exception) 
        { 
            break; 
        } 
        try 
        { 
        // get the source sample time 
        pmsSrc.GetTime(out LastStart, out LastStop); 
 
        // Set the destination sample time 
        pmsDst.SetTime(LastStart, LastStop); 
        } 
        catch (Exception) 
        { 
            pmsDst.ResetTime(); 
        } 
        // Copy the data 
        lActualDataLength = pmsSrc.ActualDataLength; 
 
        // Set the destination buffer  
        // We could Marshal the unmanaged buffer here, but no need since we are merely  
        // Setting the destination to the source buffer contents (unaltered data) 
        pmsDst.SetData(lActualDataLength, pmsSrc.GetData(lActualDataLength)); 
 
        // Copy the other flags 
        pmsDst.Discontinuity = pmsSrc.Discontinuity; 
        pmsDst.Preroll = pmsSrc.Preroll; 
        pmsDst.SyncPoint = pmsSrc.SyncPoint; 
 
        // Release the source sample 
        pmsSrc = null; 
 
        // Deliver the destination sample 
        pMSSource.DeliverSample(0, 1000, pmsDst); 
 
        // Release the destination sample 
        pmsDst = null; 
    } 
    // Audio Write 
    while (true) 
    { 
        try 
        { 
            // Get the target sample 
            // Note if we hit the end of the data stream an exception will trigger and break the loop. This is how we know to stop writing to the buffer 
            pmsSrc = _audTarget.GetSample(6000); 
            // Get the source buffer 
            pmsDst = pMSSource.GetSampleBuffer(1, 2000); 
        } 
        catch (Exception) 
        { 
            break; 
        } 
        try 
        { 
            // Get the source sample time 
            pmsSrc.GetTime(out LastStart, out LastStop); 
 
            // Set the destination sample time 
            pmsDst.SetTime(LastStart, LastStop); 
        } 
        catch (Exception) 
        { 
            pmsDst.ResetTime(); 
        } 
        // Copy the data 
        lActualDataLength = pmsSrc.ActualDataLength; 
 
        // Set the destination buffer  
        // We could Marshal the unmanaged buffer here, but no need since we are merely  
        // Setting the destination to the source buffer contents (unaltered data) 
        pmsDst.SetData(lActualDataLength, pmsSrc.GetData(lActualDataLength)); 
 
        // Copy the other flags 
        pmsDst.Discontinuity = pmsSrc.Discontinuity; 
        pmsDst.Preroll = pmsSrc.Preroll; 
        pmsDst.SyncPoint = pmsSrc.SyncPoint; 
 
        // Release the source sample 
        pmsSrc = null; 
 
        // Deliver the destination sample 
        pMSSource.DeliverSample(1, 1000, pmsDst); 
 
        // Release the destination sample 
        pmsDst = null; 
    } 
 
    // Deliver end of sample to stop the conversion 
    pMSSource.DeliverEndOfStream(0, 1000); 
    pMSSource.DeliverEndOfStream(1, 1000); 
 
    // Stop the Combine ConvertCtrl if it hasn't been already 
    if (combine.State == ConvertState.Running) 
        combine.StopConvert(); 
 
    // Reset the Source 
    combine.ResetSource(); 
 
    // Dispose 
    pMSSource.Dispose(); 
    combine.Dispose(); 
}

Run the Project

Run the project by pressing F5, or by selecting Debug -> Start Debugging.

If the steps were followed correctly, the application runs and converts the audio and video source files to one media file. If you used the sample files above, this is the expected output media file.

Wrap-up

This tutorial showed how to combine two media files, one video and one audio, to one file using the ConvertCtrl class.