Category Archives: OpenCV

Geek Home Theater Update – Minnowboard Max and lights in action

Quick update to the home theater system.  I installed the max with silverjaw lure (adds mpci-e and msata).  To attach the lights, I also required a level shifter to take the 3.3V up to 5V required by the Apa102’s.  Adafruit had just the component for the job: The 74AHCT125.

Here’s the panel with the max in there:

The results are pretty awesome.  The APA102’s peform much better than the WS2801’s.  No flicker.  Fast, and most of all, more lights!

Next up is to add some buttons to turn the thing off when needed and add USB capabilities and a remote.

Minnowboard Max Video Renderer with Ambilights – Part1: Software

2015-03-13

The minnowboard max is a pretty cool platform.  It’s small -just a little larger than a credit card, power efficient and best of all: powerful.  It uses the Intel Atom single or dual core with hyperthreading.  The best part about it, however, may be the integrated graphics with open source accelerated drivers.

Because the drivers are open source, you can expect them to generally “just work” on a typical linux distro.  No extra EULA or compiling necessary like on other embedded system.

The max’s Intel HD graphics also supports OpenCL 1.2 via the open source beignet project.  OpenCL allows you to offload otherwise CPU intensive computations onto the GPU which is specialized for specific tasks.  Having OpenCL available in an embedded system opens up a lot of possibilities including image processing via the open source OpenCV (Computer Vision) project.  I will be using all of these components in this project.Goal:To create a DLNA renderer that uses and LED strip to display an ambient light which correlates to the image on the screen.  There are several projects out there that do this: boblight, hyperion are a few.  In effort to teach myself some new skills, I opted not to use any of these projects and instead start from scratch with an architecture where I could utilize the CPU power that the max avails.  I believe this exercise has created something simple, yet unique.Components of the system:

  • OpenCV for image analysis
  • Gstreamer to play the video
  • Beignet for OpenCL/GPU offloading
  • Rygel for DLNA renderer support
  • Vaapi for hardware accelerated decoding/encoding
  • MRAA for accessing IO on the Max
  • MaxVideoRenderer
  • Python – the language
  • Ubuntu 15.04

Hardware:

  • Minnowboard Max
  • LED Strip with the WS2801 IC (google for LED strip and WS2801 and you’ll find dozens of options that aren’t very expensive)
  • Aluminium right-angle bracket I got from Home Depot for $2
  • Double-sided heavy duty 3M tape.
    (More about hardware in Part 2!)

OpenCV

OpenCV is a library for computer vision.  It’s used for object recognition, detection and has a lot of image manipulation routines that can take advantage of hardware acceleration where available.  OpenCV 3.0, now in beta, features transparent OpenCL usage when available.  In the version 2.4-days, you had to use special opencv function calls to take advantage of OpenCL.  In 3.0, all these functions have been unified into the same call.  The underlying OpenCV system will then decide if it can use OpenCL on the GPU or not.

Ubuntu 15.04 doesn’t have OpenCV 3.0, so we will have to get it from source.  First, lets get the dependencies going.

sudo apt-get install build-essential cmake cmake-gui python-dev python-numpy git

sudo apt-get build-dep opencv

Next, checkout out opencv from github:

git clone https://github.com/Itseez/opencv.git

cd opencv/

mkdir build

cd build/

cmake-gui ..

These commands will have brought you to the cmake gui.   click configure to make Unix-style makefiles and then make sure you click to enable python and the python examples.  After configuring, look at the output to make sure the python module was enabled.  If it wasn’t, look for clues in the output as to what was missing.

Tip: To make compiling faster and to eliminate errors, I usually turn off the opencv_java module in cmake.

type “make -j5”, get yourself a drink and maybe something to eat.  It takes a little bit to compile opencv.  After make is done, run “sudo make install” to install opencv.

Beignet

Beignet is an open source project that provides OpenCL support for Intel graphics platforms.  It supports the minnowboard max as well as Core “i” platforms.  Ubuntu 15.04 has version 1.0.1 already in the repository.  That will work wonderfully for our needs:

sudo apt-get install beignet ocl-icd-libopencl1 ocl-icd-dev

Gstreamer and Vaapi

Gstreamer is a powerful media framework that supports decoding and encoding of numerous media types.  It has a plugin framework system where you can combine several “elements” into a “pipeline”.  We will use this framework with our own customized and optimized pipeline.  Ubuntu comes with Gstreamer 1.0 by default, but we need a few extra packages for rygel and for vaapi support

sudo apt-get install libgstreamer1.0-dev gstreamer1.0-vaapi libgstreamer-plugins-base1.0-dev gstreamer1.0-tools

Test out gstreamer with vaapi support by using gst-launch-1.0:

gst-launch-1.0 videotestsrc ! vaapisink

You should see a test video image.

Rygel

Rygel is a DLNA framework for serving and rendering DLNA content.  Ubuntu has a slightly older version of rygel that doesn’t have python bindings enabled.  Further, upstream rygel does not yet have python bindings for the gstreamer renderer library.  I created a patch to be merged upstream that enables the bindings.  So for now, we’ll use my github fork until the patch is merged upstream.

git clone https://github.com/tripzero/rygel.git

Next, let’s get the build dependencies:

sudo apt-get build-dep rygel

sudo apt-get install python-gi libgirepository1.0-dev

We also need to grab mediaart 2 from github.

git clone https://github.com/GNOME/libmediaart.git

cd libmediaart

./autogen.sh –enable-introspection=yes

make -j5

sudo make install

Build Rygel:

cd rygel

./autogen.sh –enable-introspection=yes

make -j5

sudo make install

If everything compiled and installed, we can now test rygel out.  I use BubbleUPNP on my android to control DLNA renderers.  It also allows me to play content from my phone.  There are probably DLNA apps for other platforms.  Look around and find the one that’s best for you.

To run the example rygel renderer, navigate to rygel/examples/gi and run “python example-gst-renderer.py”.  Note that you may have to edit the interface which is hardcoded to “eth1” at the time of this writing to the interface on your system that has an active connection.  When I run this, I see some output on the screen about some deprecated “SOUP” calls.  This usually indicates to me that it’s working.  I can now launch up BubbleUPNP on my phone and select the “rygel gst renderer” renderer from the renderers list.

MRAA

MRAA is a library for accessing IO on various devices including the Max, RPI, Intel Edison and some others.  It has c++ and python bindings and is pretty easy to use.  It supports SPI, I2C, GPIO, PWM, and AnalogIO.

We will need to grab the source:

git clone https://github.com/intel-iot-devkit/mraa.git
sudo apt-get install swig
cd mraa
mkdir build && cd build
cmake -DBUILDSWIGNODE=OFF ..
make && sudo make install

To test, run python and enter in the following:

import mraa
mraa.getPlatformName()
quit()

This should output “MinnowBoard MAX”.  If it did.  It works!

MaxVideoRenderer – putting it all together

My source code for this project is found on github:

git clone https://github.com/tripzero/MaxVideoRenderer

cd MaxVideoRenderer

mkdir build

cd build

cmake ..

In part 3 of this project series will go into greater detail of how this system works and the discoveries I made alone the way.  For now, let’s just run it.

python videocolor.py eth0 MaxRenderer 0

This will run a DLNA renderer named “MaxRenderer” on “eth0” with “0” lights.  We don’t have any lights hooked up yet, so this should be fine.

Now we should be able to see MaxRenderer in our DLNA control app and play content to it.

Part 2 will go into setting up the LEDs.  Stay tuned!

NOTE: much of this comes from memory.  If you run into issues, drop me a comment and I may remember more of what I did to get this all going.