Project Frame - How I Built This

Jun, 2014

The whole idea behind Project Frame was to provide the user the ability to download a part of the video, basically a online video clipper.
Note: I built this just for fun and its just a proof of concept.

Tools used:
- FFMPEG - http://www.ffmpeg.org
- ASP.NET 4.0
- GettyImages Connect API
- JQuery videoBG
- jQuery Wookmark plugin
- jQuery imagesLoaded plugin
- simplePagination.js
- Chardinjs
- jQuery pageSlide
- soundmanager 2
- SLY https://github.com/Darsain/sly

Homepage:

The page is pretty simple page except for the background video. I used videoBG plugin to play video. So how did i build the video? FFMpeg has a way to superimpose multiple video in a grid layout. An example for generating grid layout can be found at https://ffmpeg.org/trac/ffmpeg/wiki/FilteringGuide#multipleinputoverlayin4x4grid. Using this as my starting point, I downloaded 16 videos from GettyImages.com and generated four 4x4 videos and took the final 4 videos and superimposed on top of each other to generate 1 large video with 16videos embedded within in. But off course the final video was gigantic, so I resized the final video to web size.

FFMpeg script code to generate the superimposed video:


    ffmpeg -i 1.mov -i 2.mov -i 3.mov -i 4.mov -filter_complex "[0:0]pad=iw*2:ih*2[a],[a][1:0]overlay=w[b],[b][2:0]overlay=0:h[c],[c][3:0]overlay=w:h" -shortest output.mp4

FFMpeg script code to resize the video:


    ffmpeg -i output.mp4 -s 720x480 -acodec copy output.mp4[webm]

Search Result Page:

Move along nothing fancy here. Connect API was used to search videos and jQuery Wookmark in combination with ImageLoader was used to display the result in pinterest style fashion.

Video Edit Page:

Few things happen here when the page is loaded:
1) Download the video file to a temp. directory


     var webClient = new WebClient();
 if (!string.IsNullOrEmpty(videoUrl))
  await webClient.DownloadFileTaskAsync(videoUrl, fileName);
 else
  throw new Exception("No video file found to download");

2) Generate thumbnails using ffmpeg


     private Task GenerateThumbnails(string id, string videoFileLocation)
 {
  var tcs = new TaskCompletionSource();
  var ffmpegFile = ConfigurationManager.AppSettings["ffmpeg"];
  var thumbLocation = System.Web.HttpContext.Current.Server.MapPath("~/public/thumbs");
  var folderName = string.Format("{0}\\{1}", thumbLocation, id);

  if (!Directory.Exists(folderName))
   Directory.CreateDirectory(folderName);

  var arguments = string.Format("-i {0} -f image2 -vf fps=fps=1 -s 96x72 {1}\\out%d.jpg", videoFileLocation, folderName);

  var process = new Process
  {
   StartInfo = { FileName = ffmpegFile, Arguments = arguments },
   EnableRaisingEvents = true
  };
  process.Exited += (sender, args) => tcs.SetResult(true);
  process.Start();

  return tcs.Task;
 }

This generates thumbnails of 96x72 dimensions and with files as out{number}.jpg. Folder structure something like the following:

3) There is Video metadata information view for which I used jQuery Imageslide.

Following server side code snippet for generating the image information table which jquery pageslider uses for rendering.


    [HttpGet]
public HttpResponseMessage GetImageDetail(string id)
{
 _sb.AppendFormat(imageInfo.Urls.Thumb, imageInfo.Caption);
 _sb.AppendFormat(_pivotFormat, "Caption", imageInfo.Caption ?? string.Empty);
 _sb.AppendFormat(_pivotFormat, "Date Created", imageInfo.DateCreated.ToString("F"));
 _sb.AppendFormat(_pivotFormat, "Artist", imageInfo.FilmMaker ?? string.Empty);
 _sb.AppendFormat(_pivotFormat, "Asset Family", imageInfo.AssetFamily ?? string.Empty);
 _sb.AppendFormat(_pivotFormat, "Era", imageInfo.Era ?? string.Empty);
 _sb.AppendFormat(_pivotFormat, "Keywords", string.Join(", ", imageInfo.Keywords.Select(s => s.Text).ToList()));
 _sb.AppendFormat(_pivotFormat, "Clip Length", imageInfo.ClipLength ?? string.Empty);
 _sb.AppendFormat(_pivotFormat, "Copyright", imageInfo.Copyright ?? string.Empty);
 _sb.AppendFormat(_pivotFormat, "Collection Name", imageInfo.CollectionName ?? string.Empty);
 _sb.AppendFormat(_pivotFormat, "Color Type", imageInfo.Color ?? string.Empty);
 _sb.AppendFormat(_pivotFormat, "Mastered To", imageInfo.MasteredTo ?? string.Empty);
 _sb.AppendFormat(_pivotFormat, "Camera Type", imageInfo.OriginallyShotOn ?? string.Empty);

4) Once the user selects the preferred video range, following code is executed on the server side to clip the video using ffmpeg.


     private Task ClipVideo(string fileName, string outputName, int startSecond, int duration)
 {
  var tcs = new TaskCompletionSource();
  var ffmpegFile = ConfigurationManager.AppSettings["ffmpeg"];
  var clipLocation = System.Web.HttpContext.Current.Server.MapPath("~/public/clip");
  var videoLocation = System.Web.HttpContext.Current.Server.MapPath("~/public/downloads");

  var arguments = string.Format("-sameq -ss {0} -t {1} -i {2} {3}", startSecond, duration, string.Format("{0}\\{1}", videoLocation, fileName), 
   string.Format("{0}\\{1}", clipLocation, outputName));

  var process = new Process
  {
   StartInfo = { FileName = ffmpegFile, Arguments = arguments },
   EnableRaisingEvents = true
  };
  process.Exited += (sender, args) => tcs.SetResult(true);
  process.Start();

  return tcs.Task;
 }

Off-course this solution of downloading the video file and generating the thumbnail is not a viable solution for a real world scenario. But we could always pre-process the thumbnails and store the thumbnails are an image sprite (saving as an image sprite would save number of files the OS would have to handle; off-course this sprite technique was ment for real world scenario). Since the entire code based (server side) is all async/TPL based, multiple users using the tool works quite well. But there is always an alternate solution for video processing like Animoto or Amazon Transcoding service.