198 thoughts on “iPhone HTTP Streaming with FFMpeg and an Open Source Segmenter”

1. Robert Swain June 29, 2009 at 7:07 am

   I noticed you're using the old method of explicitly specifying all options rather than using the preset files. It's highly recommended that you use the preset files for specifying options as per the guide. It makes your life easier and the command lines more readable. :)
2. carson Post authorJune 29, 2009 at 8:00 am

   Robert, thanks for the tip. I think if I follow the way that works someone could create a file named libx264-mobile.ffpreset and stick this in it:

   flags=+loop
   cmp=+chroma
   partitions=+parti4×4+partp8×8+partb8×8
   subq=5
   trellis=1
   refs=1
   coder=0
   me_range=16
   keyint_min=25
   sc_threshold=40
   i_qfactor=0.71
   qcomp=0.6
   qmin=10
   qmax=51
   qdiff=4
   g=30

   Then the command line would change to:

   ffmpeg -i <in file> -f mpegts -acodec libmp3lame -ar 48000 -ab 64k -vcodec libx264 -s 320×240 -b 96k -bt 200k -maxrate 96k -bufsize 96k -rc_eq 'blurCplx^(1-qComp)' -level 30 -aspect 320:240 -async 2 -vpre mobile <output file>
3. vz June 30, 2009 at 11:20 pm

   I am trying to compile the segmenter, but I allways have the following error, does anyone have the same problem?

   segmenter.c: In function ‘add_output_stream’:
   segmenter.c:44: error: ‘AVCodecContext’ has no member named ‘ticks_per_frame’
   segmenter.c:46: error: ‘AVCodecContext’ has no member named ‘ticks_per_frame’
   segmenter.c:54: error: ‘AVCodecContext’ has no member named ‘channel_layout’
   segmenter.c:54: error: ‘AVCodecContext’ has no member named ‘channel_layout’
   segmenter.c: In function ‘main’:
   segmenter.c:219: warning: implicit declaration of function ‘avformat_alloc_context’
   segmenter.c:219: warning: assignment makes pointer from integer without a cast
   make: *** [all] Error 1
4. chris July 2, 2009 at 3:43 am

   I got a segmentation fault running the ffmpeg command. Which libx264 version you are using?
5. carson Post authorJuly 2, 2009 at 7:34 am

   @vz It may be picking up an older version of libavformat so you might want to specify where the version you want it to use lives with -L and -I flags.
6. carson Post authorJuly 2, 2009 at 7:36 am

   @chris I'm using the version from Fedora 11: x264-devel-0.0.0-0.24.20090319gitc109c8.fc11.i586 if that helps. I am also using the git repo version as of June 24th of ffmpeg.
7. ian July 2, 2009 at 9:51 pm

   Very interested in how you get the live stream going – what are you using to encode the live transport stream- VLC?
8. carson Post authorJuly 2, 2009 at 10:40 pm

   @ian I'm going to try to get another post ready in a few days detailing. I will say that you can do everything with just FFMpeg depending on the location of your input. I'm going to involve VLC just to get the video stream from my camera on a laptop to a server where the encoding happens.
9. Mike July 6, 2009 at 7:39 am

   Great work,
   \here is what I am trying to accomplish and I am not quite sure yet how.
   Perhaps you could help.

   I want to have a continous live stream for an event so say a camera setup to stream a live event or even a non stop webcam to the iphone. I think the flow should be something like

   Camera–>VLC—>SEGMENTER—>WEB_SERVER

   If you could provide some further explanation on this I would be very greatful, \

   Thanks,
10. carson Post authorJuly 6, 2009 at 7:42 am

    Mike, check out my newer post for details on a way to do it: http://www.ioncannon.net/programming/475/iphone-windowed-http-live-streaming-using-amazon-s3-and-cloudfront-proof-of-concept/
11. djsashaz July 7, 2009 at 2:21 pm

    Is the segmenter absolutely necessary? Couldn't I just make a m3u file with a url in it?
12. carson Post authorJuly 7, 2009 at 2:31 pm

    It is necessary unless you want to have just one big stream.

    FFMpeg by itself has two limitations as far as I can see with working this on its own:

    1) It can't encode the segments correctly on its own. I did a good bit of experimenting because I felt like it should be able to do the same thing the segmenter does even though I've read the reasons why it can't. One of those reasons is that it is said to not produce timestamps but there is actually a way to do that so you can offset the timestamps in the resulting stream so they match the offset. The other reason has to do with key frames at certain locations and I think you could solve that by just adding key frames with every frame. I've got this feeling that the true issue has to do with timestamps not actually being correctly added to the streams even though they are seemingly reported to be correct by mplayer.

    2) Now assuming you could chunk the source with FFMpeg using the current command line you still would be out of luck for windowed streaming.

    So there is actually a real reason to use the segmenter.
13. Sylvain July 9, 2009 at 4:42 am

    Nice howto, but I'm stuck at segmenting part

    When using the preset I always get:
    Unable to parse option value "parti4X4+partp8X8+partb8X8": undefined constant or missing (
    Invalid value '+parti4X4+partp8X8+partb8X8' for option 'partitions'

    I remove that line and this is OK but then, when I use segmenter, I always get
    [mpegts @ 0xed1110]dts < pcr, TS is invalid
    Last message repeated 17090 times
    [NULL @ 0xecf190]missing picture in access unit

    No matter which MPEG file I use as input.
14. carson Post authorJuly 9, 2009 at 6:31 am

    @Sylvain, that sounds like it could be an old version of ffmpeg or a bad input stream.
15. Sylvain July 9, 2009 at 7:51 am

    Yep, could be wrong version of ffmpeg. I'll try with some other files as an input but doesn't seem the cause.

    On top of your howto, I tried to use libaacplus and builds recommended for merge on http://tipok.org.ua/
16. Graeme July 9, 2009 at 8:32 am

    What a fantastic tutorial. I'm having trouble encoding and getting an audio error relating to mpegts [mpegts @ 0x1009400]dts < pcr, TS is invalid which ends up corrupting the clip.
    Wondering if you encountered this issue? Am using latest snapshots of everything.
17. anonymouscoward July 9, 2009 at 8:59 am

    i was testing this out of curiosity and i got it working from the first try on 550kbps bitrate even though segmenter spit some warnings

    # ./segmenter sample_550k_pre.ts 10 sample_550k stream-550k.m3u8 http://www.mytesting.tld
    Output #0, mpegts, to 'sample_550k':
    Stream #0.0: Video: libx264, yuv420p, 320×240, q=2-31, 90k tbn, 59.96 tbc
    Stream #0.1: Audio: libmp3lame, 48000 Hz, stereo, s16, 64 kb/s
    [mpegts @ 0x81108b0]dts < pcr, TS is invalid
    Last message repeated 36163 times
    [NULL @ 0x805f560]missing picture in access unit
    [mpegts @ 0x81108b0]dts = 63000
    Could not write frame of stream

    Any clues?
18. anonymouscoward July 9, 2009 at 11:50 am

    What the… my previous comment is not as complete as i wrote it :|

    Let's try again (please delete the previous one):

    So the following encode/segment is working even though i get some warnings but playback is a bit jerky and sometimes stops for 2-4 seconds

    # ./segmenter sample_550k_pre.ts 10 sample_550k stream-550k.m3u8 http://www.mytesting.tld
    Output #0, mpegts, to 'sample_550k':
    Stream #0.0: Video: libx264, yuv420p, 320×240, q=2-31, 90k tbn, 59.96 tbc
    Stream #0.1: Audio: libmp3lame, 48000 Hz, stereo, s16, 64 kb/s
    [mpegts @ 0x81108b0]dts < pcr, TS is invalid
    Last message repeated 36163 times
    [NULL @ 0x805f560]missing picture in access unit
    [mpegts @ 0x81108b0]dts = 63000
    Could not write frame of stream

    And got no output.
    Googled a bit for a solution but not much luck.
19. anonymouscoward July 9, 2009 at 11:53 am

    ok it seems your blog strips my comment a lot so here is a pastebin
    http://pastebin.com/m2d95ceb0
    thanks!
20. Sylvain July 9, 2009 at 4:47 pm

    Ok, despite errors, streaming is OK… as long as we don't use NPVR controls and adaptive rates (video playback is not at normal speed when switching higher bitrate)…

    NPVR issue: 30s back goes to start, and forward doesn't work

    Any similar experience ?
21. Vikas Jain July 10, 2009 at 8:58 am

    As per HTTP Live Streaming requirement, each segmented .ts file should contain PAT and PMT tables.
    Is it Segmenter's responsibility to ensure that while segmenting it puts those?
    If not then how does FFMPEG ensures it?
22. Leon Chen July 13, 2009 at 3:20 am

    Hi I am having problem in configuring ffmpeg with the configuration option "–enable-libfaad" as it says faad.h file is missing. Can you advise how I could find the consistent faad.h file with that?

    thanks.
23. Chris J. July 15, 2009 at 12:04 am

    I cant get the Media Stream Segmenter from the apple ADC site and im a IPhone Developer with paid provisioning. from what I hear a lot of people are having the same issue with sht site not giving rights to validated Developers. Question is does anybody else know where to get the Apple Media Stream Segmenter! Thanks! Chris J.
24. jellisii July 16, 2009 at 7:10 am

    Has anyone compiled this on Win32 yet?
25. Frank Lowney July 17, 2009 at 11:00 am

    At WWDC '07, Apple detailed byte-range serving in QuickTime, a fairly straightforward method and one that many are familiar with. So why do all of this awkward segmenting of multiple versions of a video when byte-range serving might have been the more elegant way to go about this. Latency?

    Apple also uses an MPEG-2 stream containing H.264/AAC. What's the rationale for doing that? Encryption?
26. randall July 17, 2009 at 4:18 pm

    does anyone have a 32-bit x86 ubuntu binary of 'segmenter'? i get compilation errors – trying to build it from the ffmpeg directory as that's where libavformat/avformat.h is but it still has troubles with file paths.
27. cameron July 21, 2009 at 10:07 pm

    if anybody i'm also interested in a binary for x86 ubuntu 9
28. David July 22, 2009 at 7:04 pm

    This looks great, but the issue I am having appears to be with the ffmpeg command itself. I have a camera that is sending out H.264 with AAC audio wrapped up in mpeg4. What I reall y need, I think, is to just rewrap it in mpeg2. Any ideas on how to accomplish that? Your ffmpeg doesn't work for me at all as it stands.

    Thanks.
29. Ludovic August 5, 2009 at 10:41 am

    repost : interested in binary for windows 32bits too
30. Stephen August 6, 2009 at 9:17 pm

    Hi, I encoded my ts stream using gstreamer. It plays fine on VLC and other player but when i run it thru the segmenter. I get the follow error (lots of them):

    [mpegts @ 0x9047160]invalid dts/pts combination
    [h264 @ 0x9052800]error, non monotone timestamps 25566400 >= -9223372036854775808
    Warning: Could not write frame of stream
    [mpegts @ 0x908d460]dts < pcr, TS is invalid

    Anyone can shed some lights on whats going on?
    Many thanks.
31. damhack August 9, 2009 at 7:02 am

    Great work!

    It would be really really useful if someone could supply a download link to a static compile of ffmpeg and the segmenter for Mac OSX 10.5.7 or 10.5.8 Intel.

    It's a lot of hit and miss work getting everything to compile properly on OSX if you have your compiler environment set up for other types of project.

    Thanks,

    DH
32. finney August 11, 2009 at 4:02 am

    I am trying to compile the segmenter, but I allways have the following error, does anyone have the same problem?

    /usr/local/lib/libavcodec.a(pthread.o): In function `avcodec_thread_free':
    /home/finney/ffmpeg/libavcodec/pthread.c:95: undefined reference to `pthread_join'
    /usr/local/lib/libavcodec.a(pthread.o): In function `avcodec_thread_init':
    /home/finney/ffmpeg/libavcodec/pthread.c:159: undefined reference to `pthread_create'
    /usr/local/lib/libx264.a(encoder.o): In function `x264_encoder_close':
    encoder.c:(.text+0x23ee): undefined reference to `pthread_join'
    /usr/local/lib/libx264.a(encoder.o): In function `x264_encoder_frame_end':
    encoder.c:(.text+0x34ec): undefined reference to `pthread_join'
    /usr/local/lib/libx264.a(encoder.o): In function `x264_encoder_encode':
    encoder.c:(.text+0x7053): undefined reference to `pthread_create'
    collect2: ld Return 1
33. Stanley August 13, 2009 at 4:00 am

    I have succeeded to make on-demand stream for iPhone with this guide.
    But how can I avtivate seek-bar like the YouTube player?
34. daviddaviddavidwong August 18, 2009 at 1:36 am

    Hi.. finney
    below flag can help
    -lpthread
35. David August 18, 2009 at 1:11 pm

    I would love to see that, too. From what I've seen, the player interface supports only 30 second skip backwards or skip to live. It doesn't appear to have a random seek capability. While annoying, it does fit in with the "live streaming" Apple is going for. I really wish you could skip forward 30 seconds the same as skip backwards.
36. Phil August 19, 2009 at 4:23 pm

    I am getting this error, not sure why. Any help is appreciated. Thanks.

    phil@ubuntu:~/ffmpeg$ ffmpeg -i /home/phil/Desktop/test1.mpeg -f mpegts -acodec libmp3lame -ar 48000 -ab 64k -s 480×320 -vcodec libx264 -b 96k -flags +loop -cmp +chroma -partitions +parti4×4+partp8×8+partb8×8 -subq 5 -trellis 1 -refs 1 -coder 0 -me_range 16 -keyint_min 25 -sc_threshold 40 -i_qfactor 0.71 -bt 200k -maxrate 96k -bufsize 96k -rc_eq 'blurCplx^(1-qComp)' -qcomp 0.6 -qmin 10 -qmax 51 -qdiff 4 -level 30 -aspect 480:320 -g 30 -async 2 /home/phil/Desktop/output.mp4

    FFmpeg version SVN-r19633, Copyright (c) 2000-2009 Fabrice Bellard, et al.
    configuration: –enable-gpl –enable-nonfree –enable-pthreads –enable-libfaac –enable-libfaad –enable-libmp3lame –enable-libtheora –enable-libx264 –enable-libxvid –enable-x11grab
    libavutil 50. 3. 0 / 50. 3. 0
    libavcodec 52.32. 0 / 52.32. 0
    libavformat 52.37. 0 / 52.37. 0
    libavdevice 52. 2. 0 / 52. 2. 0
    libswscale 0. 7. 1 / 0. 7. 1
    built on Aug 12 2009 15:34:06, gcc: 4.1.3 20080623 (prerelease) (Ubuntu 4.1.2-24ubuntu1)
    [wmv3 @ 0x9349970]Extra data: 8 bits left, value: 0

    Seems stream 1 codec frame rate differs from container frame rate: 1000.00 (1000/1) -> 30.00 (30/1)
    Input #0, asf, from '/home/phil/Desktop/test1.mpeg':
    Duration: 00:00:28.71, start: 3.000000, bitrate: 3324 kb/s
    Stream #0.0(eng): Audio: wmav2, 44100 Hz, 2 channels, s16, 96 kb/s
    Stream #0.1(eng): Video: wmv3, yuv420p, 640×480, 3049 kb/s, 30 tbr, 1k tbn, 1k tbc
    Metadata
    SfOriginalFPS : 300000
    WMFSDKVersion : 11.0.6002.18005
    WMFSDKNeeded : 0.0.0.0000
    IsVBR : 0
    [NULL @ 0x9332020]Unable to parse option value "parti4×4+partp8×8+partb8×8": undefined constant or missing (
    Invalid value '+parti4×4+partp8×8+partb8×8' for option 'partitions'

37. Pingback: Encodage de video pour l’iPhone | Digital adventures

38. Nathan Ziarek August 26, 2009 at 3:53 pm

    No answers to the segmenter issue, huh? I installed FFMpeg using MacPorts. The header files for libavformat are in /opt/local/include/. I've included that in my $PATH, but still "make"ing the segmenter errors out.

    segmenter.c:21:34: error: libavformat/avformat.h: No such file or directory
    segmenter.c:23: error: syntax error before ‘*’ token
    segmenter.c:23: error: syntax error before ‘*’ token
    ….

    I assume this means it still can't find the header files, but I'm at a complete loss on how to instruct it. Any advice is appreciated.
39. hEMANT August 27, 2009 at 7:45 am

    Hi Carson,
    I referred all the istruction from your above blog, except I use the Apple's Media Segmenter.
    First, I used ffmpeg to get mpeg2 ts file.
    Then, I provided this to Apples media segmenter and it creates .m3u8 and other .ts files in o/p folder.
    Then, I publish these files on web server, and created a page with Video tag in it as descrribed by you.
    And, now I am trying to access the page on iPhone 3.0, but I can't see any video.
    When I supplied this page in the Apple's Video Demo exaample, then it says "Media Format not supported".
    aNY IDEA what may be going wrong here?

    Thanks,
    Hemant
40. Nathan Ziarek August 27, 2009 at 1:31 pm

    Not sure if this will help, but …

    1) Download and install MacPorts
    2) Install FFMpeg using "sudo port install ffmpeg +gpl +lame +x264 +xvid"
    3) Download the three files for the segmenter app (Makefile, segmenter.c and COPYING) from the link above.
    4) Edit the Makefile file to read:

    all:
    gcc -I/opt/local/include -L/opt/local/lib -Wall -g segmenter.c -o segmenter -lavformat -lavcodec -lavutil -lbz2 -lm -lz -lfaac -lmp3lame -lx264 -lfaad
    clean:
    rm segmenter

    5) CD to the directory with Makefile and segmenter.c and run make.

    It worked on 10.5.8 after much trial and error.
41. Shine August 29, 2009 at 6:43 pm

    Could you please tell me, what versions of libraries, FFMpeg, YASM, NASM and GCC did you use, if you succeeded to make the live stream for iphone with this guide under linux?
    I follow the steps of this guide on Suse with
    GCC 4.3.1, YASM-0.8.0, libfaad2.2.7, libmp3lame-398-2, libfaac-1.28,libx264 from git repository:git://git.videolan.org/x264.git, FFMpeg-0.5.
42. Shine August 29, 2009 at 6:44 pm

    There were no problem, I think, during building them with shared libraries and installing the FFMpeg with the following configuration:
    
./configure --prefix=/home/shine/iphone/shared/ --extra-cflags=-I/home/shine/iphone/shared/include/ --extra-ldflags=-L/home/shine/iphone/shared/lib/ --enable-shared --disable-static --enable-gpl --enable-nonfree --enable-pthreads --enable-libfaac --enable-libfaad --enable-libmp3lame --enable-libx264

    And it works fine for converting the media. But I got the error message, when I used the segmenter with:
    
./segmenter ../test.ts 10 ../ts/test ../ts/stream.m3u8 /home/shine/iphone/ts/
Output #0, mpegts, to '../ts/test':
Stream #0.0: Video: libx264, yuv420p, 640x480, q=2-31, 90k tbn, 25 tbc
Stream #0.1: Audio: libfaac, 24000 Hz, stereo, s16, 77 kb/s
[h264 @ 0x815af90]error, non monotone timestamps 63000 >= 63000
Warning: Could not write frame of stream
[mpegts @ 0x815a000]dts < pcr, TS is invalid
Last message repeated 2291 times
[NULL @ 0x805f5a0]missing picture in access unit
[mpegts @ 0x815a000]dts < pcr, TS is invalid

    MPlayer can play the separat transport stream, though. But VLC not.

    Any idea? What players or tools, which works like iphone, can play m3u8 and the streaming?
    Thanks a lot!

    @Phil: rewrite 'x' of '+parti4×4+partp8×8+partb8×8' .

43. Pingback: Inside Mac OS X Snow Leopard: QuickTime X — RoughlyDrafted Magazine

44. Jason September 4, 2009 at 2:19 am

    I went through all steps, outputting the .ts files from the segmenter but no video displays with the HTML5 tag. I've troubleshot from every imaginable angle. ffmpeg, segmenter, mime-types, new browser (Safari 4.x).

    My non-working example sits here:
    http://visualplant.net/test/streamtest/index.html

    Appreciate if anyone sees the source of the problem here.
45. Jason Schmidt September 5, 2009 at 12:33 pm

    I answered my own question.

    My test files do stream successfully on iPhone, but stream files do not play on the Web, at least not for me, using the HTML 5 video tag viewing on Mac 10.5.7 Safari 4. HTML 5 video tag works only when calling standard .mp4 files, not .m3u8 -> .ts files.

    So, HTTP stream tests need to be conducted via Safari on the iPhone. Unless, someone knows another way…

46. Pingback: Long-term Memory » Blog Archive » Video encoding for iPhone

47. Rick Stevens September 21, 2009 at 4:52 pm

    If you're having problems building the segmenter, I've built it successfully on Fedora 11. You must have the ffmpeg-devel RPM (V0.5-2 or later) and the ffmpeg-libs RPM (V0.5-2 or later) installed.

    NOTE: Earlier versions of these RPMs do NOT include the "ticks_per_frame" or "channel_layout" members in the various structures and you'll get compile errors.

    You must also modify segmenter's "Makefile" by inserting

    -I/usr/include/ffmpeg

    on the "gcc" line so the compiler knows where to find the header files:

    [root@bigdog work]# cat Makefile
    all:
    gcc -Wall -I/usr/include/ffmpeg -g segmenter.c -o segmenter -lavformat -lavcodec -lavutil

    clean:
    rm segmenter

    install: segmenter
    cp segmenter /usr/local/bin/

    uninstall:
    rm /usr/local/bin/segmenter

    Hope that helps.
48. Rex September 23, 2009 at 2:00 am

    Hi Carson, thanks for your very detailed posts.

    We've incorporated the segmenter into our cloud based video transcoder.
    http://www.ankoder.com/2009/09/23/cloud-based-video-ankoder-iphone-http-stream-segment
49. ColletJb September 23, 2009 at 10:09 am

    Hi,

    Thank you for this very interesting tuto.

    I'm trying to use FFMpeg to encode my video (step 2), but an error occurs and I don't understand how to fix it ? Does anyone has an idea ?

    Thank ;)

    FFmpeg version SVN-r19948, Copyright (c) 2000-2009 Fabrice Bellard, et al.
      configuration: 
      libavutil     50. 3. 0 / 50. 3. 0
      libavcodec    52.35. 0 / 52.35. 0
      libavformat   52.38. 0 / 52.38. 0
      libavdevice   52. 2. 0 / 52. 2. 0
      libswscale     0. 7. 1 /  0. 7. 1
      built on Sep 21 2009 14:08:13, gcc: 4.3.3
    
    Seems stream 0 codec frame rate differs from container frame rate: 47.95 (5000000/104271) -> 23.98 (24000/1001)
    Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'demo.mp4':
      Duration: 00:02:09.23, start: 0.000000, bitrate: 5843 kb/s
        Stream #0.0(und): Video: h264, yuv420p, 1280x544, 23.98 tbr, 24k tbn, 47.95 tbc
        Stream #0.1(und): Audio: aac, 48000 Hz, 2 channels, s16
      Metadata
        author          : 20th Century Fox
        title           : Fantastic Four: Rise of the Silver Surfer - Theatrical Trailer
        year            : 2007
    Incorrect frame size
    
50. shnee October 2, 2009 at 3:27 pm

    Hi, I'm also getting this when I try to build the segmenter

    In function `add_output_stream':
segmenter.c:28: undefined reference to `av_new_stream'
In function `main':
segmenter.c:177: undefined reference to `av_register_all'
segmenter.c:225: undefined reference to `av_find_input_format'
segmenter.c:231: undefined reference to `av_open_input_file'
segmenter.c:237: undefined reference to `av_find_stream_info'
segmenter.c:242: undefined reference to `guess_format'
segmenter.c:248: undefined reference to `avformat_alloc_context'
...

    my makefile has

    gcc -I//ffmpeg/ -Wall -g segmenter.c -o segmenter -lbz2 -lm -lz -lfaac -lmp3lame -lx264 -lfaad

    and my libavformat thats with ffmpeg is 52.39.0

    any help would be greatly appecriated