Step by step guide - Beagle Bone Black voice recognition on an LED matrix

So after I made a BeagleBone black output voice recognition to an LED Matrix, I thought I will post a "How-to" guide to make one yourself.

NOTE: If at any point you find yourself confused with a technical term, try to google it, if you are still having trouble, or you would like to make things more clear about this post, add a comment and I will follow up on it :)

First, Materials, Make sure you have all of these when you start:
1. BeagleBone Black
2. USB Audio card  - I narrowed it down for you (1$-3$)
3. Microphone - the smaller the better
4. MicroSD card 8GB
5. AdaFruit I2C LED Backpack - I used green.
6. Male - Female Jumper wire - You will need at least 4 wires
7. Internet connection - I used a LAN cable to keep it simple.

Step 1: Setting up the Angstrom distro on the Beagle
Use this tutorial to create an Angstrom image on the MicroSD card http://learn.adafruit.com/beaglebone-black-installing-operating-systems/windows
Next, plug the card into the Beaglebone and Boot to the new OS, make sure you have the USB cable connected.
Once all this is done, open up your com port and login as root. once you get a shell proceed to the next step.

Step 2: Setting up pocket sphinx 
Plug in the LAN cable to your BeagleBone, if you have WiFi good for you, just use a hub to have the USB audio enabled later.
Follow these steps:

1. run opkg update
2. opkg install python-distutils
3. From here (https://groups.google.com/forum/#!topic/beagleboard/aBznzq_bNuU) follow these steps:
1. Install libasound2, alsa-dev, alsa-lib-dev (http://www.angstrom-distribution.org/repo/) before compiling sphinxbase and pocketsphinx.
2. Get sphinxbase-0.8, and pocketsphinx-0.8 from http://cmusphinx.sourceforge.net/ and extract the tar files.           http://downloads.sourceforge.net/project/cmusphinx/sphinxbase/0.8/sphinxbase-0.8.tar.gz       http://downloads.sourceforge.net/project/cmusphinx/pocketsphinx/0.8/pocketsphinx-0.8.tar.gz
3. Change directory to the sphinxbase directory.
4. Run "./autogen.sh" to generate the configure file.
5. If needed run "./configure" 
6. Run "make"
7. Run "make install"
8. Change directory to pocketsphinx directory and redo steps 4, 5, 6, 7.
4. To check if the pocket sphinx installation worked power off the BeagleBone, make sure the USB audio card is plugged in and power on.
5. run this command to do a quick test:
   "pocketsphinx_continuous -adcdev hw:1,0 -nfft 2048 -samprate 48000 2>/dev/null"
           note: hw:1,0 is the hw node of your usb audio.

Step 3: Setting up the environment for the AdaFruit LED Backpack

To setup the adaFruit libs and files

(ref:http://learn.adafruit.com/setting-up-io-python-library-on-beaglebone-black/installation)

1. run "opkg install python-pip python-setuptools python-smbus"

2. run "pip install Adafruit_BBIO"

Step 4: Connect the LED Backpack to the BeagleBone

Switch P8 and P9, the person who created this image mixed them up. 

We will only use the left side header (P8 in this photo)

Connect the LED Backpack like this:

LED Backpack VCC --> PIN 7 SYS_5V

LED Backpack GND --> PIN 1 DGND

LED Backpack SDA --> PIN 20 I2C2_SDA

LED Backpack SCL  --> PIN 19 I2C2_SCL

Step 5: Setting up the code

Copy these files over to your BeagleBone, I put them on my ~/Desktop/ but you can choose wherever.

1. Adafruit_LEDBackpack.py - From AdaFruit RaspberryPi example code
   https://docs.google.com/file/d/0B4cQqo_E3D-iS19rcnR1ZnpYVDg/edit?usp=sharing
2. Adafruit_I2C.py - From AdaFruit RaspberryPi example code, Modified to use I2C 1 on the BeagleBone.
   https://docs.google.com/file/d/0B4cQqo_E3D-iUTh0VFZidDBXSVk/edit?usp=sharing
3. Adafruit_8x8.py - From AdaFruit RaspberryPi example code.
   https://docs.google.com/file/d/0B4cQqo_E3D-idnRDY2ZybU5mdUU/edit?usp=sharing
4. runVisibleVoice.py - My code to run PocketSphinx and to display the output on the LED Backpack
   https://docs.google.com/file/d/0B4cQqo_E3D-iVFdwUlVNa0lLTzQ/edit?usp=sharing

Once copied, simply run "python runVisibleVoice.py" and watch the magic :)

Feel free to modify my code in any way you want, but do not use it to make money. 

Share it, Don't sell it.

Here is a picture of my final version:

BeagleBone Black based voice recognition on an LED Matrix.

A little over a month ago I was at the BrainSilo hacker space in Portland with some friends,

we were playing around with our HackRF JawBreaker boards, after a while we got board and started chatting and throwing crazy ideas in the air, I got a BeagleBone Black at Defcon and I really wanted to do something with it.

And so one of the ideas was:

"Let's have the beagle bone do speech recognition and output it on an LED matrix and see how it messes up and laugh at it"

so far so good, seem like a fun thing to do for us geeks, so I decided to try it, it is kind of a challenge that won't take too much time off my hands.

At this point I only had the BeagleBone and nothing else, so I started with the largest hurdle, running voice recognition on the BBB (from this point on I will refer to the BeagleBone Black as BBB).

So I search the web looking for solutions, one of them was Texas Instruments Embedded Speech Recognition solution that recently went on the open source path, which oddly requires you to register and wait to be approved as a member before you even get to see a byte of code.

That turned out to be a bust, WAY too complicated to build run and expand for a lazy hacker like me, I want something that is script language friendly and will run without fancy compile tricks.

So... 

I turned to the internet again and looked for python based/friendly voice recognition options, one of them was pocket sphinx that also happen to run on the BBB almost smoothly. why almost smoothly?

Well, because the python side of the Sphinx never worked for me, so I had to do an ugly hack which I will explain later.

Anyway, Now that I got a Pocket sphinx running on the BBB with this command line:

pocketsphinx_continuous -adcdev hw:1,0 -nfft 2048 -samprate 48000 2>/dev/null

(Using a USB sound card off ebay I connected a microphone to the BBB)

So at this point I have in my hands, a BBB with a fairly decent voice recognition software that actually runs!

yay!

Now the next step is to have the BBB display stuff on an LED Matrix, 

I looked into a few solutions, an SPI based controller combined with an L:ED Matrix, which didn't really work on the BBB, and because I didn't want to spend too much time on SPI based re-coding on the BBB I moved on to an I2C based controller, 

I found the right one in AdaFruit, the I2C LED Backpack was perfect, it had code examples in python and someone was using it on the BBB.

But I could not find the code for displaying scrolling text or any text for that matter...

I had no choice but to do some coding, My code is at the bottom of this post, I could not upload a .py file so I just decided to paste to code at the bottom of this post.

The last step left was to combine the two, this is where the ugly hack comes in to view, I used python to run pocket sphinx in a command line ,read it's stdout stream, parse it and display it accordingly on the LED Matrix.

Technicals:

BBB running Angstrom distro unmodified at all, just download and use.

pocket sphinx 0.8 

Refrences:

BBB - http://www.ti.com/tool/beaglebk

TIESR - https://gforge.ti.com/gf/project/tiesr/

Demo Video - http://youtu.be/Ssd_tyXPa5E

Note, 

All I did was hack a few things together, there has been a lot of work done on each ingredient that led to this result. Thanks to AdaFruit and their code and Pocket-Sphinx and their implementation working on the BBB This could not have worked.

I went through many links on the internets with many forum posts and suggested solution to many problems I encountered, I do apologize for not listing them all.

The code I wrote is posted here under the "do not be a douche" license, meaning you can use it, but don't try to make any profit off it.

Hope you liked this thing!

runVisibleVoice.py

<CODE>

#!/usr/bin/python

import time

import datetime

import math

from Adafruit_8x8 import EightByEight

import sys, select, subprocess

grid = EightByEight(address=0x70)

#print "Press CTRL+Z to exit"

AZ = [0x7E, 0x11, 0x11, 0x11, 0x7E, #  A

0x7F, 0x49, 0x49, 0x49, 0x36,   #  B

0x3E, 0x41, 0x41, 0x41, 0x22,   #  C

0x7F, 0x41, 0x41, 0x22, 0x1C,   #  D

0x7F, 0x49, 0x49, 0x49, 0x41,   #  E

0x7F, 0x09, 0x09, 0x01, 0x01,   #  F

0x3E, 0x41, 0x41, 0x51, 0x32,   #  G

0x7F, 0x08, 0x08, 0x08, 0x7F,   #  H

0x00, 0x41, 0x7F, 0x41, 0x00,   #  I

0x20, 0x40, 0x41, 0x3F, 0x01,   #  J

0x7F, 0x08, 0x14, 0x22, 0x41,   #  K

0x7F, 0x40, 0x40, 0x40, 0x40,   #  L

0x7F, 0x02, 0x04, 0x02, 0x7F,   #  M

0x7F, 0x04, 0x08, 0x10, 0x7F,   #  N

0x3E, 0x41, 0x41, 0x41, 0x3E,   #  O

0x7F, 0x09, 0x09, 0x09, 0x06,   #  P

0x3E, 0x41, 0x51, 0x21, 0x5E,   #  Q

0x7F, 0x09, 0x19, 0x29, 0x46,   #  R

0x46, 0x49, 0x49, 0x49, 0x31,   #  S

0x01, 0x01, 0x7F, 0x01, 0x01,   #  T

0x3F, 0x40, 0x40, 0x40, 0x3F,   #  U

0x1F, 0x20, 0x40, 0x20, 0x1F,   #  V

0x7F, 0x20, 0x18, 0x20, 0x7F,   #  W

0x63, 0x14, 0x08, 0x14, 0x63,   #  X

0x03, 0x04, 0x78, 0x04, 0x03,   #  Y

0x61, 0x51, 0x49, 0x45, 0x43]   #  Z

az = [0x20, 0x54, 0x54, 0x54, 0x78,

0x7F, 0x48, 0x44, 0x44, 0x38,

0x38, 0x44, 0x44, 0x44, 0x20,

0x38, 0x44, 0x44, 0x48, 0x7F,

0x38, 0x54, 0x54, 0x54, 0x18,

0x08, 0x7E, 0x09, 0x01, 0x02,

0x08, 0x14, 0x54, 0x54, 0x3C,

0x7F, 0x08, 0x04, 0x04, 0x78,

0x00, 0x44, 0x7D, 0x40, 0x00,

0x20, 0x40, 0x44, 0x3D, 0x00,

0x00, 0x7F, 0x10, 0x28, 0x44,

0x00, 0x41, 0x7F, 0x40, 0x00,

0x7C, 0x04, 0x18, 0x04, 0x78,

0x7C, 0x08, 0x04, 0x04, 0x78,

0x38, 0x44, 0x44, 0x44, 0x38,

0x7C, 0x14, 0x14, 0x14, 0x08,

0x08, 0x14, 0x14, 0x18, 0x7C,

0x7C, 0x08, 0x04, 0x04, 0x08,

0x48, 0x54, 0x54, 0x54, 0x20,

0x04, 0x3F, 0x44, 0x40, 0x20,

0x3C, 0x40, 0x40, 0x20, 0x7C,

0x1C, 0x20, 0x40, 0x20, 0x1C,

0x3C, 0x40, 0x30, 0x40, 0x3C,

0x44, 0x28, 0x10, 0x28, 0x44,

0x0C, 0x50, 0x50, 0x50, 0x3C,

0x44, 0x64, 0x54, 0x4C, 0x44]

    

space = [0x00,0x00,0x00,0x00,0x00] # ord = 32

dot = [0x00, 0x60, 0x60, 0x00, 0x00] # .  ord = 46

   

def main():

    while(True):

      i=0

      while (i<130):

          grid.writeRowRaw(5,AZ[i])

          grid.writeRowRaw(4,AZ[i+1])

          grid.writeRowRaw(3,AZ[i+2])

          grid.writeRowRaw(2,AZ[i+3])

          grid.writeRowRaw(1,AZ[i+4])

          time.sleep(1)

          grid.writeRowRaw(5,az[i])

          grid.writeRowRaw(4,az[i+1])

          grid.writeRowRaw(3,az[i+2])

          grid.writeRowRaw(2,az[i+3])

          grid.writeRowRaw(1,az[i+4])

          i=i+5

          time.sleep(1)

          print i

      

      grid.clear()

      time.sleep(0.05)

def runstring(text):

    #ord a = 97 ==> first element in the array, 97 == 0 98 == 5

    #print text

    grid.clear()

    scroll = [];

    #first append empty 8 columns

    scroll.append(0x00);scroll.append(0x00);scroll.append(0x00);scroll.append(0x00);

    scroll.append(0x00);scroll.append(0x00);scroll.append(0x00);scroll.append(0x00);

    for c in text:

        #print ord(c)

        num = ord(c)

        if ((num > 64) and (num < 123)): # is a letter

            #Build a scrolling string

            if (num in range(65,90)):

                i=int(math.fabs(65-num))*5

                scroll.append(AZ[i]);

                scroll.append(AZ[i+1]);

                scroll.append(AZ[i+2]);

                scroll.append(AZ[i+3]);

                scroll.append(AZ[i+4]);

                scroll.append(0x00);

                #print 'CAPITAL'

            else:

                i=int(math.fabs(97-num))*5

                scroll.append(az[i]);

                scroll.append(az[i+1]);

                scroll.append(az[i+2]);

                scroll.append(az[i+3]);

                scroll.append(az[i+4]);

                scroll.append(0x00);

                #print 'regular'

        else:

            if (num == 46):

                scroll.append(dot[0]);

                scroll.append(dot[1]);

                scroll.append(dot[2]);

                scroll.append(dot[3]);

                scroll.append(dot[4]);

            else:

                scroll.append(space[0]);

                scroll.append(space[1]);

                scroll.append(space[2]);

                scroll.append(space[3]);

                scroll.append(space[4]);

    #end with empty 8 columns

    scroll.append(0x00);scroll.append(0x00);scroll.append(0x00);scroll.append(0x00);

    scroll.append(0x00);scroll.append(0x00);scroll.append(0x00);scroll.append(0x00);

               

    i=0

    while i <= len(scroll):

        try :

            grid.writeRowRaw(7,scroll[i])

            grid.writeRowRaw(6,scroll[i+1])

            grid.writeRowRaw(5,scroll[i+2])

            grid.writeRowRaw(4,scroll[i+3])

            grid.writeRowRaw(3,scroll[i+4])

            grid.writeRowRaw(2,scroll[i+5])

            grid.writeRowRaw(1,scroll[i+6])

            grid.writeRowRaw(0,scroll[i+7])

            time.sleep(0.04)

            i=i+1

        except :

            #print 'exception'

            break

            pass

    grid.clear()  

    #print 'end of func'

   

#runstring("Hello My name is Inigo Montoya. You killed my father. Prepare to die");

proc = subprocess.Popen(['sh', '-c', 'pocketsphinx_continuous -adcdev hw:1,0 -nfft 2048 -samprate 48000 2>/dev/null'],stdout=subprocess.PIPE)

while True:

    line = proc.stdout.readline()

    if line != '':

        #the real code does filtering here

        output = line.rstrip()

        print output

        if (len(output.split("READY"))>1):

            runstring("Speak")

        if (len(output.split("please wait"))>1):

            runstring("Please wait")

        if (len(output.split(":"))>1):

            runstring(str(output.split(":")[1])+'.')

    else:

        break

</CODE> 

 

Example - Reporting a security vulnerability when there is no clear security contact

Today I wanted to report an authentication bypass vulnerability in a router to it's vendor,
I went online to the company web site and looked for any contact info, the only thing I found was a "live chat" option, at this point I stopped and thought to myself "Why the hell not"

Just to clarify, the support person I was chatting with did his very best to do his job, and I tried to be polite, I have the outmost respect for the individuals working in support centers and would never intend to insult or demean them in any way.

Here is the result of this interesting experiment:

Support Tue, 7/9/2013 08:38:44 pm
Thank you for choosing ***********. How may I help you today?

Me Tue, 7/9/2013 08:39:32 pm
Please provide me with an email address of a security team representative to which I can rely the details of a security vulnerability I have discovered in the ****

Support Tue, 7/9/2013 08:41:20 pm
As I understand, you are asking for the security team representative so that you can address your concern reagrding the **** router?

Me Tue, 7/9/2013 08:41:48 pm
almost, I want to report a possibly new security vulnerability in the **** Router

Support Tue, 7/9/2013 08:42:12 pm
May I ask, what do you mean about security vulnerability?

Me Tue, 7/9/2013 08:42:48 pm
you know when you put in 192.168.1.1 and the router asks you for a username and password before you can view all the options in it?
I found a way to get in the router menu's without authenticating via username and password
and I would like to report it to the relevant contact in your comapny at ***********

Support Tue, 7/9/2013 08:44:46 pm
I see. When you accessed the page with no authentication required, is that your first time to access the setup page then?

Me Tue, 7/9/2013 08:46:44 pm
no,after performing a few specific actions i have access to the setup pages. without those certain actions i would not have access at all

Support Tue, 7/9/2013 08:47:10 pm
What browser did you use to access the ********** setup page?

Me Tue, 7/9/2013 08:47:17 pm
chrome

Support Tue, 7/9/2013 08:48:10 pm
I see. I believe the security password has been saved on the Chrome's settings that is why it dis not ask for a password the next time you access the setup page.

Me Tue, 7/9/2013 08:49:01 pm
i have never set it to save, nor have I ever authenticated to the router, every time I have tried to access the router IP i was prompted for authentication but i never did.

Support Tue, 7/9/2013 08:49:35 pm
Can you try to access the setup page using another computer?

Me Tue, 7/9/2013 08:50:40 pm
yes, I did that. do you have an **** test device online at the moment that I can reach?

Support Tue, 7/9/2013 08:51:11 pm
I do apologize but we do not have that support.

Me Tue, 7/9/2013 08:52:22 pm
what do you suggest?
I do not want support, I want to report a security vulnerability

Support Tue, 7/9/2013 08:53:10 pm
May I ask, when you try to use another computer, did it ask for authentication when you access the ********** setup page?

Me Tue, 7/9/2013 08:53:34 pm
yes it did
do you have 2nd level support?

Support Tue, 7/9/2013 08:54:05 pm
Did it ask authentication again the next time you accessed it?

Me Tue, 7/9/2013 08:54:33 pm
it allways asked me for authentication, until i bypassed ti with this vulnerability

Support Tue, 7/9/2013 08:55:49 pm
I see. This is actually an isolated case since this is our first time to know this concern.

Me Tue, 7/9/2013 08:56:23 pm
I know, I dont expect you to solve this for me, just start escalating me

Support Tue, 7/9/2013 08:56:42 pm
Let me forward your concern to our management so that they can check and verify your concern.

Me Tue, 7/9/2013 08:56:54 pm
thank you!

Support Tue, 7/9/2013 08:57:12 pm
But as of the moment, we cannot provide the precise solution for your concern.

Me Tue, 7/9/2013 08:57:40 pm
I know, I dont want a solution for it right now, I want to help you guys understand it and fix it

Support Tue, 7/9/2013 08:58:33 pm
We highly acknowledged your side.
May I ask your full name so that I can log this session?

Me Tue, 7/9/2013 08:58:57 pm
thank you I guess
use my email: *********@*****.***

Support Tue, 7/9/2013 08:59:32 pm
Do you have phone number for us to reach you?

Me Tue, 7/9/2013 09:00:38 pm
Let's start with email first, after that we can exchange phone numbers, I tend not to give out personal information on the "first date"

Support Tue, 7/9/2013 09:01:07 pm
I understand.

Me Tue, 7/9/2013 09:01:53 pm
awesome :) so I will expect someone to contact me at my email. thanks for your help ****

Support Tue, 7/9/2013 09:02:23 pm
You are most welcome! Is there anything else that I can assist you with today?

Me Tue, 7/9/2013 09:02:35 pm
nope, that was it :)

Support Tue, 7/9/2013 09:02:45 pm
Alright then. Remember that you are always a valued customer with ***********. Once again, thank you for choosing ***********. Have a wonderful day!

CVE 2013-3518 - Belkin WeMo Information Exposure

# Title: Belkin WeMo Information Exposure

# Date: 5/9/13

# Author: Mickey Shkatov

# Vendor Homepage: http://www.belkin.com/us/wemo

# Version: Any version prior to

US : WeMo_US_2.00.2176.PVT

World Wide :WeMo_WW_2.00.2176.PVT

 # CVE: CVE 2013-3518

Overview:

Belkin WeMo devices with firmware prior to WeMo_US_2.00.2176.PVT allow physically proximate attackers to access the file system and extract the private key, public key, trust chain and passphrase used to encrypt Belkin firmware.

Impact:

Affected products:

 - Belkin WeMo

 - Other: Since the same encryption keys are used for other Belkin products, all those products are susceptible to malicious modification.

Timeline:

Jan 10 2013 - Contacted Belkin support.

Jan 11 2013 - Belkin support replies with request for details.

Jan 11 2013 - Description of vulnerability sent.

Mar 28 2013 - A fix to the Firmware has been published by Belkin.

Apr  7 2013 - Fix confirmed.