Categories
general

HLS Video on Demand streaming

In this blog post we will show you how you can stream Video on Demand through HLS protocol, using only free open-source tools, as nginx web server and ffmpeg.

First of all, let’s explain shortly what HLS is. HTTP Live Streaming (also known as HLS) is an HTTP-based media streaming communications protocol implemented by Apple. Since it requires only standard HTTP transactions, HTTP Live Streaming is capable of traversing any firewall or proxy server that lets through standard HTTP traffic, unlike UDP-based protocols such as RTP. This also allows content to be delivered over widely available CDNs. In a few words, HLS works by breaking the overall stream into a sequence of small HTTP-based file downloads. At the start of the streaming session, the client downloads an extended M3U (m3u8) playlist containing the metadata for the various sub-streams (called TS files) which are available. You can read more about HLS Architecture on Apple Developer website.

Currently, there are two main solutions for streaming Adobe RTMP streaming, that we covered in the previous blog post and Apple HLS streaming. However, both technologies allow you to play your video as you record it, automatically adjust video quality to available bandwidth, and seek to different parts of a video.

The major differences between the two technologies are, that while Adobe RTMP works only in Flash and requires you to have a dedicated RTMP server installed,  Apple HLS works with both Flash and HTML5 and can be used with an ordinary web server.

Furthermore, when Apple decided to drop Flash support for iOS (the affected devices are iPhones, iPads, laptops, etc) the developers had to think to a solution for the users of these devices.

The first step in our tutorial it is to compile nginx web server with rtmp modules. We described the necessary steps also in our previous post , but we will mention them again here.

[code language=”bash”]
cd ~
mkdir nginx
cd nginx

# for compiler and git
apt-get install git gcc make

#for the HTTP rewrite module which requires the PCRE library
apt-get install libpcre3-dev

# for SSL modules
apt-get install libssl-dev

git clone https://github.com/arut/nginx-rtmp-module

wget http://nginx.org/download/nginx-1.4.3.tar.gz
tar zxpvf nginx-1.4.3.tar.gz
cd nginx-1.4.3

./configure –add-module=/root/nginx/nginx-rtmp-module/ –with-http_ssl_module –prefix=/usr/local/nginx-streaming/
make
make install
[/code]

We have to change the default nginx configuration file:

[code language=”bash”]
cd /usr/local/nginx-streaming/conf
mv nginx.conf nginx.conf.bkp
nano nginx.conf

worker_processes 1;

events {
worker_connections 1024;
}

http {
access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log;
include mime.types;
default_type application/octet-stream;
sendfile on;
keepalive_timeout 65;

server {
# in case we have another web server on port 80
listen 8080;

location /hls {
types {
application/vnd.apple.mpegurl m3u8;
video/mp2t ts;
}
#where the m3u8 and ts files are
alias /var/www/hls;
}

location / {
# here we can put our website
root /var/www/html;
index index.html index.htm;
}

}

}
[/code]

We compiled ffmpeg according to this guide.
After we finished to compile ffmpeg, we created an executable also for qt-faststart (we will explain later the use of this tool)

[code language=”bash”]
cd ~/ffmpeg_sources/ffmpeg/tools
make qt-faststart
cp qt-faststart $HOME/bin
[/code]

Now, we have all the necessary tools to start streaming… but, we don’t have the files. We are going to use the well known movie of Big Buck Bunny (right click and save the file in /var/www/hls folder on your webserver).

In order to prepare the ts files (media segments) and m3u8 playlist we are going to use ffmpeg. We compiled according with the guide recommended above, and we have ffmpeg executable in $HOME/bin/ffmpeg. You can define in FFMPEG variable the path to your own ffmpeg executable.

[code language=”bash”]
cd /var/www/hls
FFMPEG=$HOME/bin/ffmpeg
#pay attention
FILENAME=bbb
$FFMPEG -i $FILENAME.mp4 -codec copy -map 0 -f segment -vbsf h264_mp4toannexb -flags -global_header -segment_format mpegts -segment_list $FILENAME.m3u8 -segment_time 10 $FILENAME-%03d.ts
[/code]

Unfortunately, the streaming doesn’t work all the time, and we need to re-order the MP4 “atoms”. These atoms are the meta-information about the movie (like the timescale, duration, other characteristics). For fast delivering over the network these meta information should be at the beginning of the file, not at the end. You can read more about atoms in the following article Understanding the MPEG-4 movie atom.
To move the atoms at the beginning of the file, we will use qt-faststart tool in this way.

[code language=”bash”]
FASTSTART=$HOME/bin/qt-faststart
mv bbb.mp4 bbb_original.mp4
input=bbb_original.mp4
$FFMPEG -i $input -vcodec libx264 -vprofile high -preset slow -b:v 500k -maxrate 500k -bufsize 1000k -vf scale=854:480 -threads 0 -acodec aac -ac 2 -strict experimental -b:a 128k bbb_tmp.mp4
$FASTSTART bbb_tmp.mp4 bbb.mp4
[/code]

and now let’s execute again the previous command:

[code language=”bash”]$FFMPEG -i bbb.mp4 -codec copy -map 0 -f segment -vbsf h264_mp4toannexb -flags -global_header -segment_format mpegts -segment_list bbb.m3u8 -segment_time 10 bbb-%03d.ts[/code]

Anyway, sometimes (if the video file is broken or not standard), qt-faststart can fail with some side effects (high CPU load). Fortunately, there is a python script that can handle better the move of the atoms.

We can see the result of our work, using a m3u8 player (eg: VLC) and opening the stream from Media->Open Network Stream and in URL field pasting: http://mydomain.com:8080/hls/bbb.m3u8 (replace mydomain.com with your domain name or IP).

Hit the Play button and… the video will start to be played.

If you want to play the movie in the browser, you can read the post about How to play HLS with JW Player.

You can see an example here – if the server is up or put in your VLC the URL http://cdn.razvantudorica.com/hls/2/bbb.m3u8

Categories
general

Cassandra cannot parse ‘my_key’ as hex bytes

If you have “cannot parse ‘whatever_my_key’ as hex bytes” when you try to query Cassandra for some data, as in example below:

[code language=”bash”]
[default@CassandraCluster] get ServerLog[‘my_key’] limit 10;
org.apache.cassandra.db.marshal.MarshalException: cannot parse ‘my_key’ as hex bytes
[/code]

then you must set the client-side encoding with assume command:

[code language=”bash”]
[default@CassandraCluster] assume ServerLog keys as utf8;
[/code]

Categories
general

Delete Google Analytics Profile/Website

It took me a while to figure out how to delete a Property (web site) from Google Analytics, mainly because is not so obvious and also Google Help… actually doesn’t help too much.
In your Google Analytics account:

1. Click on the Property you want to delete (eg: razvantudorica.com).

2013-07-27 21:26:20

2. Click on first profile (eg: All Web Site Data)

2013-07-27 21:29:05

3. In the next page, Click Profile Settings

2013-07-27 21:30:35

4. in the right-bottom corner, click “Delete this profile” and confirm clicking “Delete profile” button.

2013-07-27 21:30:56

You will be redirected on the first Properties page and you have to repeat the the actions starting with step 1, for the all profiles.
The property (website) will disappear automatically when ALL the profiles associated with this Property will be deleted.

Categories
general

What is the use of Instagram?

None.

Categories
general

FreeBSD mount iso

Mount FreeBSD iso image (or any other iso, on freebsd machine).

[bash]
mdconfig -a -t vnode -f /patg/to/FreeBSD-9.0-RELEASE-i386-disc1.iso  -u 1

#mkdir -p /mnt/cdrom

mount -t cd9660 /dev/md1 /mnt/cdrom/
[/bash]
To unmount
[bash]
umount /mnt/cdrom/

mdconfig -d -u 1
[/bash]

Categories
general

SVN how to merge branch into trunk

After I messed up a svn repo because of a wrong merge I started to read svn book and how to properly merge a branch into trunk.

Let’s assume you have a branch called my-branch.

1. Go into branch folder (/var/www/my-branch)

To be sure you are in the branch folder just type svn info. You should see: URL: svn://svn.example.com/project/branches/my-branch

2. Commit or revert everything to have a clean copy and update: svn up

3. Bring the last changes from trunk into your branch
svn merge http://svn.example.com/project/trunk

see changes and test
svn status

4. If everything is fine, commit those changes into your branch
svn ci -m ‘last changes from trunk into branch’

If something went wrong you can revert with svn revert . -R

5. Go into trunk folder (you should also have a trunk clone)

6. Update your trunk copy: svn up

7. Most important. Bring changes from BRANCH to TRUNK.

svn merge —reintegrate svn://svn.example.com/project/branches/my-branch

8. svn st to see changes and test

9. Finally, commit your branch changes to trunk

svn ci -m “Merge my-branch into trunk”

 

Now, you can delete the old branch:

svn delete svn://svn.example.com/project/branches/my-branch -m “Remove my-branch”

Categories
general

Batch remove from Mercurial repo

If you did something wrong and created a lot of files in a mercurial repo, you can delete all very quickly with this
[bash]
hg st | grep ‘^?’ | cut -d ‘ ‘ -f 2 | xargs rm
[/bash]
Be carefully to execute “hg add” for files you intend to keep otherwise you will lose those files forever.

Categories
general

World friendship with Facebook

What can I say?… wow…

High-res of this photo here.

Categories
general

First post

I’m a web developer from Romania (currently working in Netherlands) and this is my first WebDev Blog.

I don’t know yet what I will write here so meanwhile I invite you to read my blogroll.