Categories
general

HLS Video on Demand streaming

In this blog post we will show you how you can stream Video on Demand through HLS protocol, using only free open-source tools, as nginx web server and ffmpeg.

First of all, let’s explain shortly what HLS is. HTTP Live Streaming (also known as HLS) is an HTTP-based media streaming communications protocol implemented by Apple. Since it requires only standard HTTP transactions, HTTP Live Streaming is capable of traversing any firewall or proxy server that lets through standard HTTP traffic, unlike UDP-based protocols such as RTP. This also allows content to be delivered over widely available CDNs. In a few words, HLS works by breaking the overall stream into a sequence of small HTTP-based file downloads. At the start of the streaming session, the client downloads an extended M3U (m3u8) playlist containing the metadata for the various sub-streams (called TS files) which are available. You can read more about HLS Architecture on Apple Developer website.

Currently, there are two main solutions for streaming Adobe RTMP streaming, that we covered in the previous blog post and Apple HLS streaming. However, both technologies allow you to play your video as you record it, automatically adjust video quality to available bandwidth, and seek to different parts of a video.

The major differences between the two technologies are, that while Adobe RTMP works only in Flash and requires you to have a dedicated RTMP server installed,  Apple HLS works with both Flash and HTML5 and can be used with an ordinary web server.

Furthermore, when Apple decided to drop Flash support for iOS (the affected devices are iPhones, iPads, laptops, etc) the developers had to think to a solution for the users of these devices.

The first step in our tutorial it is to compile nginx web server with rtmp modules. We described the necessary steps also in our previous post , but we will mention them again here.

[code language=”bash”]
cd ~
mkdir nginx
cd nginx

# for compiler and git
apt-get install git gcc make

#for the HTTP rewrite module which requires the PCRE library
apt-get install libpcre3-dev

# for SSL modules
apt-get install libssl-dev

git clone https://github.com/arut/nginx-rtmp-module

wget http://nginx.org/download/nginx-1.4.3.tar.gz
tar zxpvf nginx-1.4.3.tar.gz
cd nginx-1.4.3

./configure –add-module=/root/nginx/nginx-rtmp-module/ –with-http_ssl_module –prefix=/usr/local/nginx-streaming/
make
make install
[/code]

We have to change the default nginx configuration file:

[code language=”bash”]
cd /usr/local/nginx-streaming/conf
mv nginx.conf nginx.conf.bkp
nano nginx.conf

worker_processes 1;

events {
worker_connections 1024;
}

http {
access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log;
include mime.types;
default_type application/octet-stream;
sendfile on;
keepalive_timeout 65;

server {
# in case we have another web server on port 80
listen 8080;

location /hls {
types {
application/vnd.apple.mpegurl m3u8;
video/mp2t ts;
}
#where the m3u8 and ts files are
alias /var/www/hls;
}

location / {
# here we can put our website
root /var/www/html;
index index.html index.htm;
}

}

}
[/code]

We compiled ffmpeg according to this guide.
After we finished to compile ffmpeg, we created an executable also for qt-faststart (we will explain later the use of this tool)

[code language=”bash”]
cd ~/ffmpeg_sources/ffmpeg/tools
make qt-faststart
cp qt-faststart $HOME/bin
[/code]

Now, we have all the necessary tools to start streaming… but, we don’t have the files. We are going to use the well known movie of Big Buck Bunny (right click and save the file in /var/www/hls folder on your webserver).

In order to prepare the ts files (media segments) and m3u8 playlist we are going to use ffmpeg. We compiled according with the guide recommended above, and we have ffmpeg executable in $HOME/bin/ffmpeg. You can define in FFMPEG variable the path to your own ffmpeg executable.

[code language=”bash”]
cd /var/www/hls
FFMPEG=$HOME/bin/ffmpeg
#pay attention
FILENAME=bbb
$FFMPEG -i $FILENAME.mp4 -codec copy -map 0 -f segment -vbsf h264_mp4toannexb -flags -global_header -segment_format mpegts -segment_list $FILENAME.m3u8 -segment_time 10 $FILENAME-%03d.ts
[/code]

Unfortunately, the streaming doesn’t work all the time, and we need to re-order the MP4 “atoms”. These atoms are the meta-information about the movie (like the timescale, duration, other characteristics). For fast delivering over the network these meta information should be at the beginning of the file, not at the end. You can read more about atoms in the following article Understanding the MPEG-4 movie atom.
To move the atoms at the beginning of the file, we will use qt-faststart tool in this way.

[code language=”bash”]
FASTSTART=$HOME/bin/qt-faststart
mv bbb.mp4 bbb_original.mp4
input=bbb_original.mp4
$FFMPEG -i $input -vcodec libx264 -vprofile high -preset slow -b:v 500k -maxrate 500k -bufsize 1000k -vf scale=854:480 -threads 0 -acodec aac -ac 2 -strict experimental -b:a 128k bbb_tmp.mp4
$FASTSTART bbb_tmp.mp4 bbb.mp4
[/code]

and now let’s execute again the previous command:

[code language=”bash”]$FFMPEG -i bbb.mp4 -codec copy -map 0 -f segment -vbsf h264_mp4toannexb -flags -global_header -segment_format mpegts -segment_list bbb.m3u8 -segment_time 10 bbb-%03d.ts[/code]

Anyway, sometimes (if the video file is broken or not standard), qt-faststart can fail with some side effects (high CPU load). Fortunately, there is a python script that can handle better the move of the atoms.

We can see the result of our work, using a m3u8 player (eg: VLC) and opening the stream from Media->Open Network Stream and in URL field pasting: http://mydomain.com:8080/hls/bbb.m3u8 (replace mydomain.com with your domain name or IP).

Hit the Play button and… the video will start to be played.

If you want to play the movie in the browser, you can read the post about How to play HLS with JW Player.

You can see an example here – if the server is up or put in your VLC the URL http://cdn.razvantudorica.com/hls/2/bbb.m3u8

Categories
Programming

How to play HLS with JWPlayer

In this post we will show you a simple way how to play HLS into web browser with JW PLayer – the free version.

Even on JW Player website is saying you need the Premium version for HLS, we found it is working also with the free version together with HLSprovider plugin.

We assume the root website is in /var/www/html.

First, you must download jwplayer-6.7 from http://www.jwplayer.com/ and save the archive in /var/www/html, on the web server.
Unpack the archive:

[code language=”bash”]
unzip jwplayer-6.7.zip
cd jwplayer
[/code]

Clone the HLSProvider plugin:

[code language=”bash”]git clone https://github.com/mangui/HLSprovider[/code]

We will create the HTML file where we will display the player:

[code language=”bash”]nano index.html[/code]

and paste the code below (of course you need to adjust it, replacing mydomain.com with your real domain name or IP):

[code language=”bash”]
<html>
<head>
<script type="text/javascript" src="http://mydomain.com/jwplayer/HLSprovider/test/jwplayer6/jwplayer.js"></script>
</head>
<body>
<div id="player">Loading the player…</div>

<script type="text/javascript">
jwplayer("player").setup({
playlist: [{
// file is the url to m3u8 file
file:’http://mydomain.com/hls/bbb.m3u8′,
provider: ‘http://mydomain.com/jwplayer/HLSprovider/test/jwplayer6/HLSProvider6.swf’,
type:’hls’
}],
width: 640,
height: 480,
primary: "flash"
});
</script>

</body>
</html>
[/code]

That’s all. Now point your browser to http://mydomain.com/jwplayer/ and you can play the video.
This example ‘should’ work both on iOS devices and PCs with Flash support.

You can see the example here – if the server is up

You can read also the blog post about HLS Video on Demand streaming

Categories
linux Programming

How to do VoD with nginx

You can read my last post on LeaseWeb Labs – Streaming Video on Demand with nginx and RTMP Module

Categories
linux

Find the wireless password on Ubuntu

How many times happen to have a guest that asks you about the wireless password and you cannot remember it? On Ubuntu Linux, you can easily retrieve the wireless password like this:

[code language=”bash”]
sudo cat /etc/NetworkManager/system-connections/YOUR_NETWORK_NAME_HERE | grep "psk="
[/code]

The output will be something like:
[code language=”bash”]
psk=myultrasecretpassword
[/code]

Categories
general

Cassandra cannot parse ‘my_key’ as hex bytes

If you have “cannot parse ‘whatever_my_key’ as hex bytes” when you try to query Cassandra for some data, as in example below:

[code language=”bash”]
[default@CassandraCluster] get ServerLog[‘my_key’] limit 10;
org.apache.cassandra.db.marshal.MarshalException: cannot parse ‘my_key’ as hex bytes
[/code]

then you must set the client-side encoding with assume command:

[code language=”bash”]
[default@CassandraCluster] assume ServerLog keys as utf8;
[/code]

Categories
linux

Ubuntu 12/13 install and configure Java JRE

1. Download JRE 7 from Oracle website.
2. Unarchive the file:
[code language=”bash”]
tar zxpvf jre-7u25-linux-x64.tar.gz
[/code]
3. Create the installation directory
[code language=”bash”]
sudo mkdir -p /usr/lib/jvm/
[/code]
4. Move the unarchived directory in the newly created one
[code language=”bash”]
sudo mv ./jre1.7.0* /usr/lib/jvm/jre1.7.0
[/code]
5. execute this wonderful command:
[code language=”bash”]
sudo update-alternatives –install /usr/bin/java java /usr/lib/jvm/jre1.7.0/bin/java 3
[/code]
Done.
You can see the java version doing:
[code language=”bash”]
java -version
[/code]

KISS

Categories
Programming

Web server in one line of bash

If you want to quickly save a file through http but you don’t want to install a web server, you can just use netcat.
You can run:
[code language=”bash”]
while true; do { echo -e ‘HTTP/1.1 200 OK\r\n’; cat index.html; } | nc -l 8080; done
[/code]

index.html can be any file you want to serve it.

You can access it after that as: http://host_ip:8080/

Categories
python

Example for singleton decorator pattern in python

I know there is not very common in Python to use the singleton pattern, but I found a nice implementation of this pattern in Python 3 Patterns, Recipes and Idioms book. Starting with that example I implemented an equivalent of the well known PHP getConnection example.

You have the code below:

This is the class that implements the Singleton pattern.

[code language=”python”]
class Singleton:
def __init__(self, klass):
self.klass = klass
self.instance = None
def __call__(self, *args, **kwds):
if self.instance == None:
self.instance = self.klass(*args, **kwds)
return self.instance
[/code]

Now, we create a class and we decorate it with the Singleton class. Let’s import also MySQLdb module*.

[code language=”python”]
import MySQLdb

@Singleton
class Database:
connection = None
def get_connection(self):
if self.connection is None:
self.connection = MySQLdb.connect(host="localhost", user="root", passwd="razvan", db="mydatabase")
return self.connection
[/code]

Let’s test this:

[code language=”python”]
db1 = Database().get_connection()
db2 = Database().get_connection()

print (db2)
print (db1)
[/code]

You will see something like:

[code language=”bash”]
<_mysql.connection open to ‘localhost’ at 16b4800>
<_mysql.connection open to ‘localhost’ at 16b4800>
[/code]

As you can see there is only one object.

For fun, let’s remove the line “@Singleton” and re-run the example. This time you will see different objects:

[code language=”bash”]
<_mysql.connection open to ‘localhost’ at c91e20>
<_mysql.connection open to ‘localhost’ at bccba0>
[/code]

You can find the fully example here.

* If you don’t know how to install MySQLdb, you can check our previous post.

Categories
Programming python

Install MySQLdb for Python 2.x

If you want to install MySQLdb for Python 2.x, without using the package manager of your Linux distro (Ubuntu in our case), you can do it through pip like this:

First you must to have installed libmysqlclient-dev (and, of course pip)

[source language=”bash”]
sudo apt-get install libmysqlclient-dev
[/source]

After that execute this two commands:

[source language=”bash”]
sudo easy_install -U distribute
sudo pip install mysql-python
[/source]

Test if is working:
Type in the command line:

[source language=”bash”]
python
[/source]

and in the python console:

[source language=”python”]
>>> import MySQLdb
>>> help(MySQLdb)
[/source]

That’s all. Enjoy!

Categories
general

Delete Google Analytics Profile/Website

It took me a while to figure out how to delete a Property (web site) from Google Analytics, mainly because is not so obvious and also Google Help… actually doesn’t help too much.
In your Google Analytics account:

1. Click on the Property you want to delete (eg: razvantudorica.com).

2013-07-27 21:26:20

2. Click on first profile (eg: All Web Site Data)

2013-07-27 21:29:05

3. In the next page, Click Profile Settings

2013-07-27 21:30:35

4. in the right-bottom corner, click “Delete this profile” and confirm clicking “Delete profile” button.

2013-07-27 21:30:56

You will be redirected on the first Properties page and you have to repeat the the actions starting with step 1, for the all profiles.
The property (website) will disappear automatically when ALL the profiles associated with this Property will be deleted.