Pixel format conversion from yuvj420p to yuv420p

[pickledgator]

I'm tapping an RTSP stream that is broadcasting with pixel format yuvj420p, I can verify this with both ffmpeg directly and pyav.

Stream #0:0: Video: h264 (Main), yuvj420p(pc, bt709, progressive), 800x450 [SAR 1:1 DAR 16:9], 30 fps, 30 tbr, 90k tbn, 180k tbc

In the h264 and vp8 codec sources, we currently explicitly check for the pixel format to be yuv420p and run a conversion, if not. In ffmpeg 4.x, this results in a lovely deprecated warning, which has come up here before in other topics.

[swscaler @ 0x7f7f7cde3000] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 0x7f7f7cde3000] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 0x7f7f7cde3000] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 0x7f7f7cde3000] deprecated pixel format used, make sure you did set range correctly

The problem that I'm having is that the warning happens every frame, and is completely crushing the performance of the application.

I can resolve the issue by either commenting out the pixel format conversion or changing the conversion check to always format to yuvj420p instead here. This removes the warning and the video stream still transports successfully (at least in my testing with Chrome).

My question is then, would this change to yuvj420p be warranted permanently, or is there a work around that you can suggest such that I can specify the pixel format of the codec for my use case?

Here's a simple modified server.py with a public AXIS RTSP stream that should allow you to reproduce things:

import argparse
import asyncio
import json
import logging
import os
import ssl
import uuid

# import cv2
from aiohttp import web
from av import VideoFrame

from aiortc import MediaStreamTrack, RTCPeerConnection, RTCSessionDescription
from aiortc.contrib.media import MediaBlackhole, MediaPlayer, MediaRecorder, MediaRelay

ROOT = os.path.dirname(__file__)

logger = logging.getLogger("pc")
pcs = set()
relay = MediaRelay()


async def index(request):
    content = open(os.path.join(ROOT, "index.html"), "r").read()
    return web.Response(content_type="text/html", text=content)


async def javascript(request):
    content = open(os.path.join(ROOT, "client.js"), "r").read()
    return web.Response(content_type="application/javascript", text=content)


async def offer(request):
    params = await request.json()
    offer = RTCSessionDescription(sdp=params["sdp"], type=params["type"])

    pc = RTCPeerConnection()
    pc_id = "PeerConnection(%s)" % uuid.uuid4()
    pcs.add(pc)

    def log_info(msg, *args):
        logger.info(pc_id + " " + msg, *args)

    log_info("Created for %s", request.remote)

    # prepare local media
    player = MediaPlayer("rtsp://174.6.126.86/axis-media/media.amp", format="rtsp")
    if args.record_to:
        recorder = MediaRecorder(args.record_to)
    else:
        recorder = MediaBlackhole()

    @pc.on("datachannel")
    def on_datachannel(channel):
        @channel.on("message")
        def on_message(message):
            if isinstance(message, str) and message.startswith("ping"):
                channel.send("pong" + message[4:])

    @pc.on("connectionstatechange")
    async def on_connectionstatechange():
        log_info("Connection state is %s", pc.connectionState)
        if pc.connectionState == "failed":
            await pc.close()
            pcs.discard(pc)

    @pc.on("track")
    def on_track(track):
        log_info("Track %s received", track.kind)

        if track.kind == "audio":
            pc.addTrack(player.audio)
            recorder.addTrack(track)
        elif track.kind == "video":
            pc.addTrack(player.video)

        @track.on("ended")
        async def on_ended():
            log_info("Track %s ended", track.kind)
            await recorder.stop()

    # handle offer
    await pc.setRemoteDescription(offer)
    await recorder.start()

    # send answer
    answer = await pc.createAnswer()
    await pc.setLocalDescription(answer)

    return web.Response(
        content_type="application/json",
        text=json.dumps(
            {"sdp": pc.localDescription.sdp, "type": pc.localDescription.type}
        ),
    )

async def on_shutdown(app):
    # close peer connections
    coros = [pc.close() for pc in pcs]
    await asyncio.gather(*coros)
    pcs.clear()


if __name__ == "__main__":
    parser = argparse.ArgumentParser(
        description="WebRTC audio / video / data-channels demo"
    )
    parser.add_argument("--cert-file", help="SSL certificate file (for HTTPS)")
    parser.add_argument("--key-file", help="SSL key file (for HTTPS)")
    parser.add_argument(
        "--host", default="0.0.0.0", help="Host for HTTP server (default: 0.0.0.0)"
    )
    parser.add_argument(
        "--port", type=int, default=8080, help="Port for HTTP server (default: 8080)"
    )
    parser.add_argument("--record-to", help="Write received media to a file."),
    parser.add_argument("--verbose", "-v", action="count")
    args = parser.parse_args()

    if args.verbose:
        logging.basicConfig(level=logging.DEBUG)
    else:
        logging.basicConfig(level=logging.INFO)

    if args.cert_file:
        ssl_context = ssl.SSLContext()
        ssl_context.load_cert_chain(args.cert_file, args.key_file)
    else:
        ssl_context = None

    app = web.Application()
    app.on_shutdown.append(on_shutdown)
    app.router.add_get("/", index)
    app.router.add_get("/client.js", javascript)
    app.router.add_post("/offer", offer)
    web.run_app(
        app, access_log=None, host=args.host, port=args.port, ssl_context=ssl_context
    )

Thanks!

[jlaine]

I don't understand what you mean, could you clarify?

From what I understand:

• you are reading a file from RTSP, and this file gets decoded to yuvj420p by ffmpeg
• you then feed this video into aiortc to be sent over the network
• as libvpx expects yuv420p input, a conversion occurs

Which step is producing the warning? How do you plan to feed the frames to libvpx if the conversion is "removed"?

[benhacka]

Which step is producing the warning? How do you plan to feed the frames to libvpx if the conversion is "removed"?

I faced a similar issue, the warning is thrown on the first encoding thread, so if the second encoding thread starts (a second client appears), the script crashes.

[ParthibanBA]

Hi, I am also facing this below mentioned warning. Is there any way to hide those warning printing in
console?

[swscaler @ 0x7f7f7cde3000] deprecated pixel format used, make sure you did set range correctly

[johnboiles]

Possibly a separate issue, but I'm getting messed up colors when streaming using the webcam.py example unless I switch the pixel format to codec.pix_fmt = "yuvj420p" in h264.py.

This is using codec.pix_fmt = "yuv420p". (Notice the color ghosting)

This is using codec.pix_fmt = "yuvj420p". Nice clean picture as expected.

I'm on a Pi using the h264_omx encoder. Again possibly a separate issue, but changing codec.pix_fmt solves both the warning and my color ghosting issue.

[nagi1999a]

I recently try to use this great library and encounter these warnings, while server decoding client's yuvj420p format video. Since the warning is happened at decoding progress, I cannot bypass the warning by the above-mentioned solution and also hard to change the client's stream pixel format, which results in huge number of warning logs at every frame.

And since jlaine discovered the deadlock issue in PyAV, and had disabled the log handler in PyAV in #480, the log message cannot be ignored through setting log levels in PyAV.

Is there anyway, other than redirect the shell output to /dev/null, to drop the warning message in this case?

[skvark]

Obviously, this is not recommended, but you can revert that patch during runtime by monkey patching av.logging before importing aiortc:

import av.logging
# monkey patch av.logging.restore_default_callback 
restore_default_callback = lambda *args: args
av.logging.restore_default_callback = restore_default_callback
av.logging.set_level(av.logging.ERROR)

# import aiortc etc.

[atharvnuthi]

Anybody found a recommended solution to this problem?
I can't have that error popping on my console for each frame.

[osom8979]

I also found the same problem and solved it as follows.

1. Find libavutil.so in pyav.
2. Load shared libraries using ctypes.CDLL.
3. Call av_log_set_level(ERROR).

Please refer to the following link:

• av_modules.py
• av_logging.py