UT99 Time Squeeze

The Big Squeeze


For the last week I've been trying to make a time lapse video of the World Domination Map. The waiting was the hardest part. The result wasn't as good as I expected it to be, but it was an experiment anyway and I learned a lot of things along the way.


Here it is.





The Joy of ffmpeg


FACT: there is no joy.

ffmpeg is a wonderful tool for both Linux and Windows, but if you're not a video pro you're going to get lost in all the command line switches.

However, it was the only tool (I found) for this job, which required taking a snaphot of a browser once every 30(-ish) seconds.

If you read the fine manual page, you will notice there is a switch, -x11grab, that takes an X display (the desktop GUI in Linux) for input. When I saw that I knew immediately I had found the tool I needed.

But as it happens, there is no binary, ready-to-install, ffmpeg package on Earth for Debian or Ubuntu that includes this incredibly useful option (and just forget about finding it in the Windows package). You have to build it yourself.

No big deal. I've been compiling crap on Linux since 1994. But if you decide to roll your own ffmpeg to get x11grab, compile and run it in you home folder. Do not run "make install" because you do not have all the libraries required to make a version as "full featured" as the one that's already installed on your system.

As with all other desktop capture programs in the Known Universe, there is no "timelapse" option. However, ffmpeg will let you take a capture every so many seconds, but if you roll that into a video file, it won't play back as timelapse. You have to capture each frame, save it as a PNG (Portable Network Graphics) file, and stitch those files back together with ffmpeg to get a timelapse video.


Patience REQUIRED


I set up to take a snapshot once every 30 seconds. With the playback rate that means one second of vid time is going to take about a half hour to collect. Armed with that knowledge, I started capturing and took a nap.

After I woke up I stitched all the frames together to take a peek at the few seconds of video I managed to acquire. The results were not up to my expectations. The individual PNG files were perfect, as crystal-clear as they were originally on the screen.

Glommed together as a video, they were ugly as Hell.

I played around with the output options, but in the end I couldn't fix it. If there was fixing to be done, it could only be done in "post production", so I decided as long as I kept the PNG files I could mess around with the quality options at my leisure.


Final Run


There were other problems.

First, the Map is a nasty memory-leaker. This is a known issue with the Google Maps API. Nothing to be done there, but ffmpeg likes its CPU cycles and memory. Running both on the same box is pushing things to the limit.

Second, there is the issue of knowing where, exactly, the browser is on the X display. Not a problem if its full screen. This is where it gets weird.

I had to display on one Linux machine and capture from another. For this I used a VM (virtual machine) on my laptop. Then, on a Windows box, I ran an X session (with Cygwin-X) at the perfect screen resolution, which seemed to be 1050x800. This VM dedicated all its resources to running The Map in a browser. For days.

Then I dedicated another VM to grab the remote X display of the laptop VM on the Windows box over the network. Got that? Three physical boxes, two VMs.

Third, The Map has a tendency to just stop. It's not that big of a deal to get it started again, so I just decided to keep an eye on it during this test run.

It ran for four days. Sometime during the last day the VM running ffmpeg ran out of disk space. ffmpeg never complained or errored out. It just kept making zero-byte files, which, when you give it a thought or two, is not a very helpful way to operate.

During those four days I on worked the Map's stopping problem. Every time I thought I had it nailed, it would stop again. The Map is updated by a bash script that peals off the name, ping, score, and IP address data for latitude and longitude.

You may recall the fun I had with GoDaddy last year with this issue. As it turns out, the same problem(s) still exist. Every now and then, the ftp process that delivers the data to the server just HANGS. How and why I don't care anymore, since I don't like dealing with GoDaddy's Tech support.

But in the end I hacked the script around so that it can detect that it's HUNG and kill any outstanding ftp process.

As a result of that, The Map now works better than ever (knock on wood). But in the process of fixing the script, I broke it completely and The Map didn't update for about an hur and a half.

At that point I killed everything and started working on the video. Even with the thousands of zero byte files I still had 5 minutes of video. When I made the final version, I doubled the speed before I uploaded it to YouTube.


Version 2.0


Version 2.0 is now in the planning stage, using lessons learned during the production of version 1.0. I hope to get a full week's worth of capture this time around.