Short tutorial: Digital Television with GStreamer (ATSC setup)

GStreamer support for Digital Television (DTV) is still working. If you follow a few very simple steps you can get DTV signals tuned-to, captured, decoded and displayed with your computer and a few accessories. 

This short article presents basic instructions for a working DTV setup. I use ATSC in the US as an example, but the short process should work equally well for other terrestrial delivery systems and countries.

What hardware do you need?

Beyond a computer, you will need an antenna and a capture device. I recently tested a Hauppauge WinTV-dualHD (ATSC/QAM) USB dongle with a 5-years old Amazon basics flat indoor antenna (that looks pretty much like this one) and it worked quite well at roughly 60km from the repeater station.

Installation

The hardware setup is simple. Install the antenna pointing to your repeater station and as high as you can. Your local telecommunications and/or broadcasting association should have data about repeaters near you. You can get this information by ZIP code from the Federal Communications Commission (FCC) if you are in the US.

Don't stress about the "as high as you can" part of the antenna installation. It's great to have a uninterrupted and direct line of sight to the repeating station's antenna, but this is hard to accomplish in urban areas. My current test system is indoors, on a first floor, and it works fine for a few channels.

Software

You will need GStreamer. I'm building it from the master branch with gst-build but the DTV code hasn't changed much in the last few years so any recently-packaged version should work for you. You will also need dvbv5-scan from DVBv5 tools.

TL;DR

Build a channels.conf configuration file by scanning an initial set of frequencies for your area with dvbv5-scan. You can get initial-frequencies files from the the dtv-scan-tables repository. Here I'm using my Mountain View, California file as an example:

$ dvbv5-scan -o channels.conf ./dtv-scan-tables/atsc/us-CA-Mountain-View

The scanning can take several minutes.

If you grep the resulting file you can see what channels were detected:

$ grep "\\[" channels.conf

The output should look something like this:

[KNTV-HD]
[Hai Le]
[KPIX-TV]
[KQED-HD]
[KTVU-HD]
[ION]
[KFSF-HD]
[KICU-HD]

If you get no channels, and you have reasons to believe your hardware is working correctly, try repositioning your antenna and rescanning till you get some. Small direction and position changes can have big effects on VHF/UHF reception with semi-directional antennas like the flat one I'm using for this example.

To playback a channel with GStreamer you can use the generated channels.conf configuration file and any of the scanned-channel names to let playbin/dvbbasebin figure out all necessary parameters at runtime

$ GST_DVB_CHANNELS_CONF=channels.conf gst-play-1.0 dvb://KNTV-HD

And that's it.

There are lots of details, pitfalls and options I don't write about in this short tutorial but if there's some interest I will try to discuss the subject in more depth in future ones.

I leave you with two screenshot of the HD feed by KNTV in the bay area

 

 

 

 

 

Creating scenarios for gst-validate

[This article is a previous rendition of the copy-edited version published at the Samsung OSG blog]

The gst-validate utility, part of the developers' tools offered by GStreamer through the gst-devtols module, allows for the detection of known issues in GStreamer pipelines. In this brief guide, I will show you how to create playback scenarios to test your pipeline's reaction to a new set of controlling actions.

There are a couple of common and not-so-common scenarios already included with the GStreamer validate suite. This set, allows for the identification of known error conditions on a running pipeline subjected to the actions expressed by the provided scenarios. Now, what if you are after inspecting the reaction of your pipeline to a new set of actions? Then you need a new scenario that describes it.

Scenarios are built from serialized actions on a .scenario file. These actions and their parameters are expressed using a GstStructure-based text format. The current list of actions includes the following core ones: seek, pause, switch-track, wait, dot-pipeline, set-feature-rank, set-state, set-property, set-debug-threshold, emit-signal, disable-plugin; and a few others implemented by gst validate plugins, like: validatefaultinjection: corrupt-socket-recv: validategtk: gtk-put-event: validate-launcher: set-subtitle. For this introductory article we will be dealing only with a few of them so you get an idea of how everything fits together.

Let's say we want to test the resilience of a pipeline to a flush seek to 10s after the pipeline is paused at playback-time 1s for 5s. Lets also say that we want to stop playback at 20s. A preliminary version of our scenario file will look like:

description, seek=true, need-clock-sync=true
pause, name=Our-pause, playback-time=1.0, duration=5.0
seek, name=Our-forward-seek, playback-time=6.0, start=10.0, flags=accurate+flush
stop, playback-time=20.0

Whenever you run into a new action name (first member on each line), use the following command to get a full description of what the action does and what parameters you can use to control it:

gst-validate-1.0 -t

For example, the "description" line above talks about two important aspects of our scenario:

seek=true: This scenario performs seek operations
need-clock-sync=true: The scenario execution needs to be synchronized to the pipeline's clock

Pretty straightforward isn't it? Now, we do need to add some boilerplate for this scenario to be fully operational. For example, this one will likely not provide the results we want if our media has a duration that doesn't allow a proper forward seek at 5s to 10s. For the sake of example then, let's inform the scenario runner the test media should have a minimum duration of 30 seconds. We do this using the following "description" line:

description, seek=true, need-clock-sync=true, min-media-duration=30s

The scenario parsing system assumes each line on the scenario file represents an action but we can use backslashes to enter multi-line ones. Here's how our scenario file looks with a multi-line *description* action (sic) with a long "summary":

description, seek=true, need-clock-sync=true, min-media-duration=30s, \
summary="This example scenario seeks forward to 10s after pausing playback for 5. \
Then, it stops the execution at 20s"
pause, name=Our-pause, playback-time=1.0, duration=5.0
seek, name=Our-forward-seek, playback-time=6.0, start=10.0, flags=accurate+flush
stop, playback-time=20.0

By the way, we have been carrying this one over for a while, but "description" is not really an action. Sorry to break it to you but please consider this an implementation detail you will have to live with.

Now that we know what to write, we need to save this scenario script to a file named after the scenario. In this particular example, we will save the above content as test-seek.scenario and pass this file name (without extension) directly to gst-validate for testing a pipeline:

gst-validate-1.0 --set-scenario test-seek filesrc location=test20sVP9.mkv ! matroskademux ! vp9dec ! autovideosink

Beyond passing the full PATH to the scenario file to --set-scenario, you can also put this file somewhere else and make the GST_VALIDATE_SCENARIOS_PATH environment variable point to that location. These are not even the only options but both should get you going with the above example.

I assume you might be using your own pipeline to try this procedure out but if you want to follow the above example to the letter, you will need to download the test20sVP9.mkv sample file. Additionally if the syntax used to express the pipeline is not something you are familiar with you can find our best attempt at an explanation on the gst-launch documentation page.

And that's it. gst-validate will timely subject the pipeline to the actions we scripted via our scenario file, and will provide a report on whether any of the built-on gst-validate tests failed. Beyond a simple statement of problem, sometimes gst-validate will even provide hints towards a full diagnostic.

In this article we briefly discuss the scenario-writing aspect of gst-validate. While the suite of tools goes far beyond, a good understanding of the steps required to express action scenarios, provides a great foray into this GStreamer testing system. Moving forward, If you want additional examples, take a look at the real-life ones being included with gst-validate and do consider contributing your own by submitting a patch to the gst-devtools component on the project's buzilla.

Good luck!

Wayland uninstalled. The easy way

This article is an adaptation of another one first posted on the Samsung Open Source Group blog.

I recently started looking at some GStreamer & Wayland integration issues and, as everyone would, commenced by trying to setup a Wayland development environment.

Before getting my feet wet though, I decided to have a chat about this with Derek Forman, OSG's resident Wayland expert. This isn't surprising because on our team, pretty much every task starts by having a conversation with one of our field specialists. The idea is to save time, as you might have guessed.

This time around I was looking for a fairly trivial piece of info:

Me - "Hey Derek, I have Wayland installed on my distro for some reason - I don't really want to take a look at now - and I would like to setup an upstream Wayland environment without messing it up. Do you have some script like GStreamer's gst-uninstalled so I can perform this feat without messing with my local install?

Derek - "Hey Reynaldo, No."

He said something else I forgot already. Guess he was ranting as usual. Anyhow, the conversation continued with me taking care of the usual I-forgot-to-ask-how-you-were-doings, and later, I'm not sure who's idea it was but we decided to try upstreaming a

wl_uninstalled script --The name had to match existing infrastructure, but that's a boring story you don't want to know about-- that, like gst-uninstalled, would enable developers to quickly setup and use a build/run-time environment consisting of an uninstalled set of interdependent Wayland repos. The idea is not new, but it's still extremely useful. Sure, you can perform similar feats by littering your local disk with custom-prefix installs of pretty much anything, but hey, it is bad taste OK? and you will waste a lot of time, at the very least once.

As if the foreword wasn't boring enough and to continue trying your resilience, here are the instructions to use the thing we come up with:

First the What

Essentially, wl_uninstalled is a helper script that automates the setup needed to build and work with an uninstalled Wayland/Weston environment comprised at least of the wayland, wayland-protocols, libinput, and weston repositories. The provides a shell environment where all build and run-time dependencies are resolved so the uninstalled versions of these modules take precedence.

Then the How

I'll use Weston as an example although other Wayland-based projects should work as well.

Edit a local copy of the script to make $WLD point to the base directory where the repositories are located, make sure to use the absolute path. Then, after executing the script, issue the following commands to get everything built and Weston running from the uninstalled environment:

cd $WLD
for i in wayland wayland-protocols libinput weston; do cd $i && ./autogen.sh && make && cd ..; done
weston &

This work is now upstream, you can see the discussions by searching for 'uninstalled' in the Wayland mailing lists archive. As an added bonus, it turned out that fiddling with this uninstalled environment mambo-jambo lead to the uncovering of a few additional broken pieces of buildsystem/pkg-config cruft that we fixed too. 'Cause we are good as that see ;). Not everything has been merged though. A word on that at the end.

Back to my own use case --GStreamer / Wayland integration work-- now I can simply call the wl_uninstalled script before gst-uninstalled and I'm all set. My uninstalled GStreamer environment will use my (in-turn) uninstalled Wayland/Weston environment making debugging, fixing and integrating against the upstream version of both projects a breeze --Not really but still--, and my filesystem more FHS-compliant. But then again, who cares about that; you probably just want all this to be quick and painless. Hopeless.

Note:

While all patches for this are on Wayland's public mailing list, some have not been merged yet. If you want to try the procedure described here you will need to manually apply the missing ones till they find their way into the official repository:

• wayland
• weston

The actual script is in the wayland-build-tools repository now.

Giving a lecture at this year's expolibre

For those of you around Talca and wanting to talk FOSS, Youness Aloui, Thibault Saunier and your's truly are going to be speaking at this year's Expolibre thanks to the gentle support of our company and the organizer's invitation.

I was talking yesterday with a fellow coworker on how community events like the good old ones get shadowed under the massive, corporate backed ones you see in the tech news by the day. Sadly as it might sound to a few (at least --And I hope), we are slowly beginning to forget there are still some guys out there that resisting the thrust of being only motivated by what to sell next year and how to make it so it goes good and cheap at the same time (like if it was possible, someway), still gather to share views, knowledge and maybe-naive-but-still-valid desires of steer collaborative innovation a bit away from the dreaded coin-only focus. We need to take an step back sometimes and think about some important things we forget on the rush of our everyday paid developer's life; there's a community that saw us shooting at the stars once and has continued backing up most of the technology we rely on. Supplying not only that but: tools, a beyond-technical environment and a reason for trying not only to do things right but do good while we are at it. Attending these kind of events is my way of taking that needed step back and I couldn't be more happier about working for a company that not only allows me to attend but go as far as directly supporting the event with the time of its developers.

So, if you find yourself wandering around Talca (VII Region, Chile. Some 3 hours away from Santiago) by the 24/12/2011 and wanna go do some knowledge sharing and hear and talk about FOSS without having to pay a penny, then go there. I have been giving talks at this conference for the last 3 years and one thing I can guarantee you is that it's worth the time you will spend.

Thibault Saunier will be talking about video editing with PiTiVi, Youness Aloui will be talking about FOSS and the fights some users have to go through to make sure their rights don't get crippled by merely buying a product, and I'm going to be sharing the goods of GStreamer through a gently introduction for beginners. Don't worry about not having a strong software-development background, our talks are geared towards anyone that doesn't run away from a mouse.

C you there!

GStreamer on Android, The NDK way

Long story short; thanks to my employer, Collabora, I have been working on getting GStreamer built and installed as a native support library under Android using the NDK. We had this working and announced for last GSTConf at Prague but there were a few details to iron out to get our work in shape for external testing.

The idea behind this adventure is showing the world you can benefit from this marvelous, swissknife-like media framework under the green droid's platform utilizing a least intrusive path.

We worked a few months ago on having GStreamer built as part of Android itself and while I do believe that approach should benefit system integrators rolling out their own customized Android version, it has the drawback of requiring both patching and building the entire Android OS and having administrative (root) access to your device. This is arguably not a problem for the most adventurous among you but we figured out providing a way to benefit from all the goods in GStreamer without forcing you to perform any major hacks was worth trying. So we did.

To be really honest, most of this work wouldn't have been possible (or at least really, like _REALLY_ harder) without the help of Collabora's own Derek Foreman's Androgenizer. You will need this tool if you want to try building our NDK bundle. And if you are working on porting some other complex project to Android; You need it too! so go check it out.

Right now you can get instructions on how to build GStreamer using the NDK and install it on your device at our freedesktop wikipage. Here is a quick run down of what is currently workig:

• Building most of gstreamer, -base, -good, -bad, -ugly & -openmax. The first 5 entirely from upstream!
• Building of our set of support libraries for this bundle (glib, x264, ogg, libmad, faad, libid3tag)
• Building gstaudioflingersink from gst-android. We are still working on gstsurfaceflingersink to adapt it to some late API changes in Gingerbread
• APK generation and Installation. Mind you this is not a real Android GUI application, just the set of GStreamer libraries and executables for you to build upon
• Execution of gst- binaries on the device using run-as

Suported Android versions:

• Gingerbread and Honeycomb

Test devices:

• Samsung Galaxy Tab 10.1
• Google Nexus S

This is still a work in progress but as I trust the community to be a great vehicle for driving innovation forward and have the luxury of working for a Company that supports this very same principle, I decided to have this aired so I can benefit from both your testing and feedback.

If you give this process a try and feel like supporting our work, please consider subscribing to our mailing list and sharing your experience!

Adding XSUB support to GStreamer

So I have been working on adding XSUB support to GStreamer as a landing task at my new job. To be honest, I didn't expect it to go as easy. Don't get me wrong, I do know most subpicture encoding schemes are kind of trivial, what I expected to be hard was to work with the GStreamer's plugin API, which was a completely new monster for me. Long history short; with the help of several coworkers and some good documentation, I have patch ready to be tested. There are still a few things here and there that might benefit from some tweaking but overall the code does its job and its in dire need of some testing love :) Dunno how long it would take for it to land in -bad but for the time being you can follow current development here.

If you want to test the code you might consider using a pipeline along these lines:

gst-launch-0.10 -v --gst-debug=xsub:5 xsub name=overlay show-background=FALSE ! ffmpegcolorspace ! xvimagesink filesrc location=/somepath/small.divx ! avidemux name=d d.video_00 ! queue ! decodebin2 ! ffmpegcolorspace ! overlay.video_sink d.video_01 ! queue ! overlay.xsub_sink

I'd put some screenshots here but I have been unable to find samples I can share.

Making great ER diagrams without drawing

This one will be short :-)

I needed to draw a decent looking Entity Relationship diagram for an academic assignment --BORING. This is the kind of thing that makes me wonder why I surrendered my soul to this degree hunting battle. Anyway, there are two utilities (Maybe more, who knows) that might help the candidate ER Michelangelo: Dia and kivio. Now, if you are into buttons and mousing that's all you should need; pick one, search for some online examples and you will be done in no time. Chances are your drawing will look like shit but then again, you'd be an artist you proly wouldn't be reading this anyway. Now, I'm not really into WYSIWYG so I searched a little more and discovered a life savior: The Tikz-er2 LaTeX package.

Not wanting to bore you more than what's strictly needed to communicate my joy, here is a complete example almost right from the documentation.

Neat isn't it?

Well, If you are still interested you should take a look at the

package documentation. It's quite short, just a few pages

long. For the time being and to aid your experiments here

is the LaTeX source for the above displayed diagram:


\documentclass[a4paper,10pt]{report}
\usepackage{ucs}
\usepackage[utf8]{inputenc}
\usepackage[spanish]{babel}
\usepackage{tikz-er2}


\usetikzlibrary{positioning}
\usetikzlibrary{shadows}



\tikzstyle{every entity} = [top color=white, bottom color=blue!30,
draw=blue!50!black!100, drop shadow]
\tikzstyle{every weak entity} = [drop shadow={shadow xshift=.7ex,
shadow yshift=-.7ex}]
\tikzstyle{every attribute} = [top color=white, bottom color=yellow!20,
draw=yellow, node distance=7em, drop shadow]
\tikzstyle{every relationship} = [top color=white, bottom color=red!20,
draw=red!50!black!100, drop shadow]
\tikzstyle{every isa} = [top color=white, bottom color=green!20,
draw=green!50!black!100, drop shadow]



\begin{document}
\begin{tikzpicture}[node distance=8em, every edge/.style={link}]
\node[entity] (persona) {Persona};
\node[attribute] (pid) [left of=persona] {\key{id\_persona}} edge (persona);
\node[attribute] (name) [below left of=persona] {nombre} edge (persona);
\node[multi attribute] (fono) [above of=persona] {fono} edge (persona);
\node[attribute] (anex) [right of=fono] {anexo} edge (fono);
\node[attribute] (number) [below left of=fono, node distance=5em] {number} edge (fono);
\node[attribute] (email) [above right of=persona] {email} edge (persona);
\node[relationship] (pertenece) [right of=persona] {Pertenece} edge node[auto,swap] {1:1} (persona);

\node[entity] (depto) [right of=pertenece] {Depto} edge node[auto,swap] {0:N} (pertenece);
\node[attribute] (iddepto) [above of=depto] {\key{id\_depto}} edge (depto);
\node[attribute] (dname) [above right of=depto] {d\_name} edge (depto);
\node[rectangle, draw=black, fit=(pertenece), inner sep=0em] (ipertenece) {};
\node[relationship] (tieneun) [below of=pertenece] {Ocupa} edge node[auto,swap] {1:1} (pertenece);

\node[entity] (cargo) [right of=tieneun] {Cargo} edge node[auto,swap] {0:N} (tieneun);
\node[attribute] (idcargo) [right of=cargo] {\key{id\_cargo}} edge (cargo);
\node[attribute] (cname) [above right of=cargo] {c\_name} edge (cargo);
\node[attribute] (plevel) [below right of=cargo] {nivel} edge (cargo);

\node[relationship] (responde) [below of=persona] {Responde} edge node[auto,swap] {1:N} (ipertenece);

\node[entity] (pregunta) [below of=tieneun] {Pregunta};
\node[attribute] (idpreg) [below of=pregunta] {\key{id\_preg}} edge (pregunta);
\node[attribute] (ptexto) [below right of=pregunta] {p\_texto} edge (pregunta);
\node[attribute] (pnumber) [right of=pregunta] {p\_numero} edge (pregunta);

\node[relationship] (tiene) [below of=responde] {Tiene} edge node[auto,swap] {1:N} (pregunta) edge node[auto,swap] {0:N} (responde);
\node[rectangle, draw=black, fit=(tiene), inner sep=0em] (itiene) {};

\node[entity] (alternativa) [left of=tiene] {Alternativa} edge node[auto,swap] {1:1} (tiene);
\node[attribute] (idalt) [above of=alternativa] {\key{id\_altern}} edge (alternativa);
\node[attribute] (atexto) [below of=alternativa] {a\_texto} edge (alternativa);
\node[attribute] (anumber) [below right of=alternativa] {a\_numero} edge (alternativa);
\end{tikzpicture}
\end{document}


Hope it helps.