In This Topic ▼

Combine Video and Audio from Separate Sources - C# .NET 6

This tutorial shows how to take two files, one video and one audio, and combine them to one media file in a C# .NET 6 Console application using the LEADTOOLS Multimedia SDK.

Overview
Summary	This tutorial covers how to combine media files in a C# .NET 6 Console application.
Completion Time	30 minutes
Visual Studio Project	Download tutorial project (2 KB)
Platform	C# .NET 6 Console Application
IDE	Visual Studio 2022
Runtime Target	.NET 6 or Higher
Development License	Download LEADTOOLS
Try it in another language	C#: .NET 6+ (Console) C API: C DLL (Windows)

Required Knowledge

Get familiar with the basic steps of creating a project by reviewing the Add References and Set a License tutorial, before working on this tutorial.

Create the Project and Add LEADTOOLS References

Start with a copy of the project created in the Add References and Set a License tutorial. If you do not have that project, follow the steps in that tutorial to create it.

The references needed depend upon the purpose of the project. References can be added using local DLL references, the following DLLs are needed.

The DLLs are located at <INSTALL_DIR>\LEADTOOLS23\Bin\net:

Leadtools.dll
Leadtools.Multimedia.dll

For a complete list of which DLL files are required for your LEADTOOLS Multimedia application, refer to Multimedia Files to be Included with your Application.

Set the License File

The License unlocks the features needed for the project. It must be set before any toolkit function is called. For details, including tutorials for different platforms, refer to Setting a Runtime License.

There are two types of runtime licenses:

Evaluation license, obtained at the time the evaluation toolkit is downloaded. It allows the toolkit to be evaluated.
Deployment license. If a Deployment license file and developer key are needed, refer to Obtaining a License.

Add the Code to Combine the Media Files

With the project created, the references added, and the license set, coding can begin.

In Solution Explorer, open Program.cs, then add using Leadtools.Multimedia; to the using block at the top.

using Leadtools; 
using Leadtools.Multimedia;

Add a new method inside the Program class named CombineAudioandVideo(string _videoFile, string _audioFile, string _targetFile). Call this new method inside the Main method after the call to the InitLEAD method. The string values passed into the new method are going to be the file paths to the video file, audio file, and target file path for the combined media file. Ensure that you add STAThreadAttribute to indicate to the COM components that the application is of single-thread threading model.

[STAThread] 
static void Main(string[] args) 
{ 
    string videoFile = @"FILE PATH TO VIDEO SOURCE FILE"; 
    string audioFile = @"FILE PATH TO AUDIO SOURCE FILE"; 
    string targetFile = @"FILE PATH TO COMBINED OUTPUT MEDIA FILE"; 
 
    InitLEAD(); 
    CombineAudioandVideo(videoFile, audioFile, targetFile); 
}

For the purposes of this tutorial, you may use this audio file and this video file.

Add the code below in the CombineAudioandVideo method to create the SampleTarget for the audio and video files, and start the ConvertCtrl.

static void CombineAudioandVideo(string _videoFile, string _audioFile, string _targetFile) 
{ 
   ConvertCtrl vidConvert = new(); 
   ConvertCtrl audConvert = new(); 
 
   //Init the SampleTargets. The Video and Audio data from our files will write to these. 
   SampleTarget vidTarget = new(); 
   SampleTarget audTarget = new(); 
 
 
   //Set the Media Type of the VideoTarget to Video 
   MediaType mt = new(); 
   mt.Type = Leadtools.Multimedia.Constants.MEDIATYPE_Video; 
   vidTarget.SetAcceptedMediaType(mt); 
   vidConvert.TargetObject = vidTarget; 
 
   //Clear 
   mt = null; 
 
   //Set the Media type for our AudioTarget to Audio. 
   mt = new MediaType(); 
   mt.Type = Leadtools.Multimedia.Constants.MEDIATYPE_Audio; 
   audTarget.SetAcceptedMediaType(mt); 
   audConvert.TargetObject = audTarget; 
 
   //Clear 
   mt = null; 
 
   //Set the ConvertCtrls to point to the Files as their sources. 
   vidConvert.SourceFile = _videoFile; 
   audConvert.SourceFile = _audioFile; 
 
   //Start running the two conversion controls. These are writing to the Sample Targets. 
   vidConvert.StartConvert(); 
   audConvert.StartConvert(); 
 
   //Enter the Combine Method 
   CombineFiles(vidTarget, audTarget, _targetFile); 
 
   //Stop running the ConvertCtrls 
   if (vidConvert.State == ConvertState.Running) 
      vidConvert.StopConvert(); 
   if (audConvert.State == ConvertState.Running) 
      audConvert.StopConvert(); 
 
   //Dispose of the targets we wrote the data to 
   vidTarget.Dispose(); 
   audTarget.Dispose(); 
 
   //Dispose of the ConvertCtrls that read the file data into the SampleTarget Buffers 
   vidConvert.Dispose(); 
   audConvert.Dispose(); 
}

After the above code has been added, create a new method named CombineFiles(SampleTarget _vidTarget, SampleTarget _audTarget, string _targetFile). This method will be called inside the CombineAudioandVideo method code, as shown above. Add the code below to the new method to use the ConvertCtrl to combine the two media files.

static void CombineFiles(SampleTarget _vidTarget, SampleTarget _audTarget, string _targetFile) 
{ 
 
    //Initialize the MultiStreamSource. This is the data that our Combine ConvertCtrl will be reading and then finnally writing to file. 
    //We have two streams. 0 = Video  1 = Audio 
    MultiStreamSource pMSSource = new(); 
    pMSSource.StreamCount = 2; 
 
    //Set the MediaType of the Sources Video Stream to that of the data connected to the VideoTarget 
    MediaType mt = _vidTarget.GetConnectedMediaType(); 
    pMSSource.SetMediaType(0, mt); 
 
    //Clear 
    mt = null; 
 
    //Set the Mediatype of the Sources Audio Stream to that of the data connected to the AudioTarget 
    mt = _audTarget.GetConnectedMediaType(); 
    pMSSource.SetMediaType(1, mt); 
 
    //Clear 
    mt = null; 
 
    //Init the Combine ConvertCtrl that will output our file. This ConvertCtrl will take in the MultiStream Source and output a file on disk. 
    ConvertCtrl combine = new(); 
    combine.SourceObject = pMSSource; 
    combine.TargetFile = _targetFile; 
    combine.VideoCompressors.H264.Selected = true; 
    combine.AudioCompressors.AAC.Selected = true; 
    combine.TargetFormat = TargetFormatType.MPEG2Transport; 
 
    //Our MediaSamples. Both a source retreived from our SampleTargets and a destination that will be written to the MultiStreamSource 
    MediaSample pmsSrc = null; 
    MediaSample pmsDst = null; 
 
    long LastStart; 
    long LastStop; 
 
    int lActualDataLength; 
 
    //Begin the running the Combine ConvertCtrl 
    combine.StartConvert(); 
 
 
    //Video Write 
    while (true) 
    { 
        try 
        { 
            // get the target sample. 
            //Note if we hit the end of the data stream an exception will trigger and break the loop. This is how we know to stop writing to the buffer. 
            pmsSrc = _vidTarget.GetSample(6000); 
            // get the source buffer 
            pmsDst = pMSSource.GetSampleBuffer(0, 2000); 
        } 
        catch (Exception) 
        { 
            break; 
        } 
        try 
        { 
            // get the source sample time 
            pmsSrc.GetTime(out LastStart, out LastStop); 
 
            // set the destination sample time 
            pmsDst.SetTime(LastStart, LastStop); 
        } 
        catch (Exception) 
        { 
            pmsDst.ResetTime(); 
        } 
        // copy the data 
        lActualDataLength = pmsSrc.ActualDataLength; 
 
        // set the destination buffer 
        // we could Marshal the unmanaged buffer here, but no need since we are merely 
        // setting the destination to the source buffer contents (unaltered data) 
        pmsDst.SetData(lActualDataLength, pmsSrc.GetData(lActualDataLength)); 
 
        // copy the other flags 
        pmsDst.Discontinuity = pmsSrc.Discontinuity; 
        pmsDst.Preroll = pmsSrc.Preroll; 
        pmsDst.SyncPoint = pmsSrc.SyncPoint; 
 
        // release the source sample 
        pmsSrc = null; 
 
        // deliver the destination sample 
        pMSSource.DeliverSample(0, 1000, pmsDst); 
 
        // release the destination sample 
        pmsDst = null; 
    } 
    //Audio Write 
    while (true) 
    { 
        try 
        { 
            // get the target sample 
            //Note if we hit the end of the data stream an exception will trigger and break the loop. This is how we know to stop writing to the buffer. 
            pmsSrc = _audTarget.GetSample(6000); 
            // get the source buffer 
            pmsDst = pMSSource.GetSampleBuffer(1, 2000); 
            } 
        catch (Exception) 
        { 
            break; 
        } 
 
        try 
        { 
            // get the source sample time 
            pmsSrc.GetTime(out LastStart, out LastStop); 
 
            // set the destination sample time 
            pmsDst.SetTime(LastStart, LastStop); 
        } 
        catch (Exception) 
        { 
            pmsDst.ResetTime(); 
        } 
        // copy the data 
        lActualDataLength = pmsSrc.ActualDataLength; 
 
        // set the destination buffer 
        // we could Marshal the unmanaged buffer here, but no need since we are merely 
        // setting the destination to the source buffer contents (unaltered data) 
        pmsDst.SetData(lActualDataLength, pmsSrc.GetData(lActualDataLength)); 
 
        // copy the other flags 
        pmsDst.Discontinuity = pmsSrc.Discontinuity; 
        pmsDst.Preroll = pmsSrc.Preroll; 
        pmsDst.SyncPoint = pmsSrc.SyncPoint; 
 
        // release the source sample 
        pmsSrc = null; 
 
        // deliver the destination sample 
        pMSSource.DeliverSample(1, 1000, pmsDst); 
 
        // release the destination sample 
        pmsDst = null; 
    } 
 
    // deliver end of sample to stop the conversion 
    pMSSource.DeliverEndOfStream(0, 1000); 
    pMSSource.DeliverEndOfStream(1, 1000); 
 
    //Stop the Combine ConvertCtrl if it hasn't been already 
    if (combine.State == ConvertState.Running) 
        combine.StopConvert(); 
 
    //Reset the Source 
    combine.ResetSource(); 
 
    //Dispose 
    pMSSource.Dispose(); 
    combine.Dispose(); 
}

Run the Project

Run the project by pressing F5, or by selecting Debug -> Start Debugging.

If the steps were followed correctly, the application converts the audio and video source files to one media file. If you used the sample files above, this is the expected output media file.

Wrap-up

This tutorial showed how to combine two media files, one video and one audio, to one file using the ConvertCtrl class.