HLS streaming not working reliably

[dorinclisu]

First of all, I'm not sure if I'm missing something and can't pinpoint the exact problem (not being familiar with HLS), so I'd like to ask:

1. The community - has anyone actually managed to get HLS stream working well?
2. @aler9 - is HLS support supposed to be just an experimental feature or can we expect to rely on it?

Basically, I have the default config with a sample rtsp source (but also tried a few live IP cameras), and access http://localhost:8888/test from Safari (which has 1st class HLS support):

paths:
  test:
    source: rtsp://wowzaec2demo.streamlock.net/vod/mp4:BigBuckBunny_115k.mov
    sourceOnDemand: false
    sourceProtocol: tcp

I could never view a continuous stream for at least 1 minute:

1. The first time loading the page, player waits a few seconds and then it gives up, suggesting the stream is unplayable (even though the rtsp soure is ready already)
2. After the 2nd or Nth time reloading the page, the video plays right away, but it freezes after a few seconds. Depending on the source, it can freeze after 5 seconds or 33 seconds. Also depending on the source, after it freezes it automatically seeks back and restarts playing from the initial entry point. For other sources it stays freezed, for other sources it skips a few seconds into the future (presumably the live point). Either way, the cycle keeps repeating with the same kind of interruptions.
3. I find it strange that the stream appears seekable instead of a live broadcast. I don't think all RTSP sources are seekable, and I don't know whether the HLS converter can automatically figure that out and how it implements seeking internally. I mean it's certainly convenient to be able to seek back a few seconds and then jump back to the live point, but if it can't be done reliably then I suppose it's best to give up this convenience.
4. Similar problems on Chrome, if not more. Didn't test it thoroughly because I thought it's pointless if it doesn't work on Safari.

There doesn't appear to be something obviously wrong in the logs, except the <nil> at the end but it's probably unrelated and it appears 1 minute after closing the browser tab.

2021/06/01 15:18:15 I [0/0] rtsp-simple-server v0.16.2
2021/06/01 15:18:15 I [0/0] [RTSP] TCP listener opened on :8554
2021/06/01 15:18:15 I [0/0] [HLS] listener opened on :8888
2021/06/01 15:18:16 I [0/0] [path test] [rtsp source] started
2021/06/01 15:18:16 D [0/0] [path test] [rtsp source] connecting
2021/06/01 15:18:21 D [0/0] [path test] [rtsp source] c->s OPTIONS rtsp://...
... gortsplib debug stuff
2021/06/01 15:18:22 I [0/0] [path test] [rtsp source] ready
2021/06/01 15:18:24 I [0/0] [HLS] [conn 172.18.0.1:63950] GET /test/stream.m3u8
2021/06/01 15:18:24 I [0/0] [HLS] [converter test] opened
2021/06/01 15:18:24 I [0/1] [HLS] [converter test] is converting into HLS
2021/06/01 15:18:24 I [0/1] [HLS] [conn 172.18.0.1:63950] GET /test/1622560704.ts
2021/06/01 15:18:27 I [0/1] [HLS] [conn 172.18.0.1:63954] GET /test/
2021/06/01 15:18:27 I [0/1] [HLS] [conn 172.18.0.1:63954] GET /test/stream.m3u8
2021/06/01 15:18:27 I [0/1] [HLS] [conn 172.18.0.1:63954] GET /test/stream.m3u8
2021/06/01 15:18:27 I [0/1] [HLS] [conn 172.18.0.1:63958] GET /test/1622560704.ts
2021/06/01 15:18:27 I [0/1] [HLS] [conn 172.18.0.1:63954] GET /test/stream.m3u8
2021/06/01 15:18:28 I [0/1] [HLS] [conn 172.18.0.1:63954] GET /test/stream.m3u8
2021/06/01 15:18:30 I [0/1] [HLS] [conn 172.18.0.1:63954] GET /test/stream.m3u8
2021/06/01 15:18:31 I [0/1] [HLS] [conn 172.18.0.1:63954] GET /test/stream.m3u8
2021/06/01 15:18:33 I [0/1] [HLS] [conn 172.18.0.1:63954] GET /test/stream.m3u8
2021/06/01 15:18:33 I [0/1] [HLS] [conn 172.18.0.1:63954] GET /test/1622560711.ts
2021/06/01 15:18:34 I [0/1] [HLS] [conn 172.18.0.1:63958] GET /test/
2021/06/01 15:18:34 I [0/1] [HLS] [conn 172.18.0.1:63958] GET /test/stream.m3u8
2021/06/01 15:18:34 I [0/1] [HLS] [conn 172.18.0.1:63958] GET /test/1622560704.ts
2021/06/01 15:18:35 I [0/1] [HLS] [conn 172.18.0.1:63958] GET /test/1622560711.ts
2021/06/01 15:18:35 I [0/1] [HLS] [conn 172.18.0.1:63962] GET /test/stream.m3u8
2021/06/01 15:18:36 I [0/1] [HLS] [conn 172.18.0.1:63962] GET /test/stream.m3u8
2021/06/01 15:18:36 I [0/1] [HLS] [conn 172.18.0.1:63958] GET /test/1622560704.ts
2021/06/01 15:18:37 I [0/1] [HLS] [conn 172.18.0.1:63958] GET /test/
2021/06/01 15:18:37 I [0/1] [HLS] [conn 172.18.0.1:63958] GET /test/stream.m3u8
2021/06/01 15:18:37 I [0/1] [HLS] [conn 172.18.0.1:63958] GET /test/1622560704.ts
2021/06/01 15:18:37 I [0/1] [HLS] [conn 172.18.0.1:63958] GET /test/1622560711.ts
2021/06/01 15:18:37 I [0/1] [HLS] [conn 172.18.0.1:63958] GET /test/1622560704.ts
2021/06/01 15:18:37 I [0/1] [HLS] [conn 172.18.0.1:63958] GET /test/stream.m3u8
2021/06/01 15:18:38 I [0/1] [HLS] [conn 172.18.0.1:63958] GET /test/stream.m3u8
2021/06/01 15:18:39 I [0/1] [HLS] [conn 172.18.0.1:63958] GET /test/
2021/06/01 15:18:39 I [0/1] [HLS] [conn 172.18.0.1:63958] GET /test/stream.m3u8
2021/06/01 15:18:39 I [0/1] [HLS] [conn 172.18.0.1:63958] GET /test/1622560704.ts
2021/06/01 15:18:39 I [0/1] [HLS] [conn 172.18.0.1:63958] GET /test/1622560711.ts
2021/06/01 15:18:39 I [0/1] [HLS] [conn 172.18.0.1:63958] GET /test/stream.m3u8
2021/06/01 15:18:39 I [0/1] [HLS] [conn 172.18.0.1:63962] GET /test/1622560704.ts
2021/06/01 15:18:40 I [0/1] [HLS] [conn 172.18.0.1:63958] GET /test/stream.m3u8
2021/06/01 15:18:41 I [0/1] [HLS] [conn 172.18.0.1:63958] GET /test/
2021/06/01 15:18:41 I [0/1] [HLS] [conn 172.18.0.1:63958] GET /test/stream.m3u8
2021/06/01 15:18:41 I [0/1] [HLS] [conn 172.18.0.1:63958] GET /test/1622560711.ts
2021/06/01 15:18:41 I [0/1] [HLS] [conn 172.18.0.1:63958] GET /test/1622560716.ts
2021/06/01 15:18:41 I [0/1] [HLS] [conn 172.18.0.1:63958] GET /test/1622560720.ts
2021/06/01 15:18:41 I [0/1] [HLS] [conn 172.18.0.1:63962] GET /test/stream.m3u8
2021/06/01 15:18:42 I [0/1] [HLS] [conn 172.18.0.1:63962] GET /test/stream.m3u8
2021/06/01 15:18:43 I [0/1] [HLS] [conn 172.18.0.1:63962] GET /test/stream.m3u8
2021/06/01 15:18:45 I [0/1] [HLS] [conn 172.18.0.1:63958] GET /test/stream.m3u8
2021/06/01 15:18:45 I [0/1] [HLS] [conn 172.18.0.1:63958] GET /test/1622560724.ts
2021/06/01 15:19:02 I [0/1] [HLS] [conn 172.18.0.1:63958] GET /test/stream.m3u8
2021/06/01 15:19:02 I [0/1] [HLS] [conn 172.18.0.1:63958] GET /test/1622560737.ts
2021/06/01 15:19:02 I [0/1] [HLS] [conn 172.18.0.1:63958] GET /test/1622560732.ts
2021/06/01 15:19:14 I [0/1] [HLS] [conn 172.18.0.1:63958] GET /test/stream.m3u8
2021/06/01 15:19:14 I [0/1] [HLS] [conn 172.18.0.1:63958] GET /test/1622560749.ts
2021/06/01 15:19:15 I [0/1] [HLS] [conn 172.18.0.1:63958] GET /test/1622560753.ts
2021/06/01 15:19:24 I [0/1] [HLS] [conn 172.18.0.1:63958] GET /test/stream.m3u8
2021/06/01 15:19:24 I [0/1] [HLS] [conn 172.18.0.1:63958] GET /test/1622560757.ts
2021/06/01 15:19:25 I [0/1] [HLS] [conn 172.18.0.1:63958] GET /test/1622560762.ts
2021/06/01 15:19:34 I [0/1] [HLS] [conn 172.18.0.1:63958] GET /test/stream.m3u8
2021/06/01 15:19:34 I [0/1] [HLS] [conn 172.18.0.1:63958] GET /test/1622560770.ts
2021/06/01 15:19:34 I [0/1] [HLS] [conn 172.18.0.1:63958] GET /test/1622560774.ts
2021/06/01 15:19:44 I [0/1] [HLS] [conn 172.18.0.1:63958] GET /test/stream.m3u8
2021/06/01 15:19:44 I [0/1] [HLS] [conn 172.18.0.1:63958] GET /test/1622560778.ts
2021/06/01 15:19:45 I [0/1] [HLS] [conn 172.18.0.1:63958] GET /test/1622560783.ts
2021/06/01 15:20:45 I [0/1] [HLS] [converter test] ERR: %!s(<nil>)
2021/06/01 15:20:45 I [0/0] [HLS] [converter test] closed

Since HLS is just http, I don't think that running inside a Docker container is relevant at all, but thought I'd mention anyway.

[aler9]

Hello, first of all i'd like to clarify a couple of things about HLS. HLS works by generating files, that can be served with a standard HTTP server. These files are:

• TS segments. In case of video tracks, these segments MUST begin with an I-Frame.
• A m3u8 playlist which contains a list of these segments, and is updated continuously.

Second thing to keep in mind: desktop browsers don't support HLS natively. They support it by using libraries like hls.js that basically unwrap tracks from the TS container, wrap them again in fMP4 and append this fMP4 chunk to a <video> element by using the Media Source Extensions API.

The first time loading the page, player waits a few seconds and then it gives up, suggesting the stream is unplayable (even though the rtsp soure is ready already)

The server must wait for an I-Frame to appear, then it can start producing a segment, then it must wait for another I-Frame to appear to finalize the current segment and send it to a client. rtsp-simple-server don't re-encode the stream, therefore the I-Frame interval is fixed and dictated by the source (camera, file or other server). It can vary from 1 second to 60 seconds. Therefore, this issue depends largely on the original stream. If the stream causes too much problems, you can choose to re-encode it with a fixed I-frame interval before sending it to the server.

After the 2nd or Nth time reloading the page, the video plays right away, but it freezes after a few seconds

An improvement consists in editing the server to block HLS client requests until at least 2-3 segments are ready to be served. The cons is that this increases delay and again depends on the I-Frame interval.

I find it strange that the stream appears seekable instead of a live broadcast

Browsers don't support live streams natively (except iOS Safari), they support only standard videos, that are seekable. As i explained above, HLS libraries allow to support live streams by appending video chunks to a video element, one after the other. This is the reason why the video appears seekable. To solve the issue, you can use a HLS library of your choice (most of them hide the seek bar) and load the HLS URL provided by the server.

[dorinclisu]

Actually Safari desktop does support HLS natively as well (at least on macOS 11), and it works flawlessly with both seekable and live broadcasts such as this one: https://moctobpltc-i.akamaihd.net/hls/live/571329/eight/playlist.m3u8

Anyway, I understand your explanation about I-Frames but it still doesn't explain my pain point number 2. Do you have an example sample stream that will work smoothly? Because the wowza demo stream doesn't (and we can't blame it for being an obscure IP camera) so perhaps either there are some issues with the converter algorithm or the server defaults should be changed. But I don't know what values would be more reasonable.

I cannot comment as to how the server works with a HLS library because it's not yet possible to use one due to CORS issue (#415)

[dorinclisu]

OK I think I've identified the issue preventing smooth playing: #418

[dorinclisu]

I see that content-type for stream.m3u8 is text/plain.
Wouldn't it better be application/x-mpegURL?

[aler9]

This has been implemented.

[aler9]

With respect to all the issues and enhancements listed here, some of them have been dealt with, some of them are still WIP. I think it's better to discuss each problem in a dedicated issue.