Commit 051f8d2d authored by Tomasz Figa's avatar Tomasz Figa Committed by Mauro Carvalho Chehab

media: docs-rst: Document memory-to-memory video decoder interface

Due to complexity of the video decoding process, the V4L2 drivers of
stateful decoder hardware require specific sequences of V4L2 API calls
to be followed. These include capability enumeration, initialization,
decoding, seek, pause, dynamic resolution change, drain and end of
stream.

Specifics of the above have been discussed during Media Workshops at
LinuxCon Europe 2012 in Barcelona and then later Embedded Linux
Conference Europe 2014 in Düsseldorf. The de facto Codec API that
originated at those events was later implemented by the drivers we already
have merged in mainline, such as s5p-mfc or coda.

The only thing missing was the real specification included as a part of
Linux Media documentation. Fix it now and document the decoder part of
the Codec API.
Signed-off-by: default avatarTomasz Figa <tfiga@chromium.org>
Signed-off-by: default avatarHans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: default avatarMauro Carvalho Chehab <mchehab+samsung@kernel.org>
parent b867d9ce
This diff is collapsed.
...@@ -39,4 +39,10 @@ file handle is visible through another file handle). ...@@ -39,4 +39,10 @@ file handle is visible through another file handle).
One of the most common memory-to-memory device is the codec. Codecs One of the most common memory-to-memory device is the codec. Codecs
are more complicated than most and require additional setup for are more complicated than most and require additional setup for
their codec parameters. This is done through codec controls. their codec parameters. This is done through codec controls.
See :ref:`mpeg-controls`. See :ref:`mpeg-controls`. More details on how to use codec memory-to-memory
devices are given in the following sections.
.. toctree::
:maxdepth: 1
dev-decoder
...@@ -39,6 +39,11 @@ Single-planar format structure ...@@ -39,6 +39,11 @@ Single-planar format structure
to a multiple of the scale factor of any smaller planes. For to a multiple of the scale factor of any smaller planes. For
example when the image format is YUV 4:2:0, ``width`` and example when the image format is YUV 4:2:0, ``width`` and
``height`` must be multiples of two. ``height`` must be multiples of two.
For compressed formats that contain the resolution information encoded
inside the stream, when fed to a stateful mem2mem decoder, the fields
may be zero to rely on the decoder to detect the right values. For more
details see :ref:`decoder` and format descriptions.
* - __u32 * - __u32
- ``pixelformat`` - ``pixelformat``
- The pixel format or type of compression, set by the application. - The pixel format or type of compression, set by the application.
......
...@@ -60,6 +60,10 @@ Authors, in alphabetical order: ...@@ -60,6 +60,10 @@ Authors, in alphabetical order:
- Original author of the V4L2 API and documentation. - Original author of the V4L2 API and documentation.
- Figa, Tomasz <tfiga@chromium.org>
- Documented the memory-to-memory decoder interface.
- H Schimek, Michael <mschimek@gmx.at> - H Schimek, Michael <mschimek@gmx.at>
- Original author of the V4L2 API and documentation. - Original author of the V4L2 API and documentation.
...@@ -68,6 +72,10 @@ Authors, in alphabetical order: ...@@ -68,6 +72,10 @@ Authors, in alphabetical order:
- Documented the Digital Video timings API. - Documented the Digital Video timings API.
- Osciak, Pawel <posciak@chromium.org>
- Documented the memory-to-memory decoder interface.
- Osciak, Pawel <pawel@osciak.com> - Osciak, Pawel <pawel@osciak.com>
- Designed and documented the multi-planar API. - Designed and documented the multi-planar API.
...@@ -92,7 +100,7 @@ Authors, in alphabetical order: ...@@ -92,7 +100,7 @@ Authors, in alphabetical order:
- Designed and documented the VIDIOC_LOG_STATUS ioctl, the extended control ioctls, major parts of the sliced VBI API, the MPEG encoder and decoder APIs and the DV Timings API. - Designed and documented the VIDIOC_LOG_STATUS ioctl, the extended control ioctls, major parts of the sliced VBI API, the MPEG encoder and decoder APIs and the DV Timings API.
**Copyright** |copy| 1999-2016: Bill Dirks, Michael H. Schimek, Hans Verkuil, Martin Rubli, Andy Walls, Muralidharan Karicheri, Mauro Carvalho Chehab, Pawel Osciak, Sakari Ailus & Antti Palosaari. **Copyright** |copy| 1999-2018: Bill Dirks, Michael H. Schimek, Hans Verkuil, Martin Rubli, Andy Walls, Muralidharan Karicheri, Mauro Carvalho Chehab, Pawel Osciak, Sakari Ailus & Antti Palosaari, Tomasz Figa
Except when explicitly stated as GPL, programming examples within this Except when explicitly stated as GPL, programming examples within this
part can be used and distributed without restrictions. part can be used and distributed without restrictions.
......
...@@ -56,14 +56,16 @@ The ``cmd`` field must contain the command code. Some commands use the ...@@ -56,14 +56,16 @@ The ``cmd`` field must contain the command code. Some commands use the
A :ref:`write() <func-write>` or :ref:`VIDIOC_STREAMON` A :ref:`write() <func-write>` or :ref:`VIDIOC_STREAMON`
call sends an implicit START command to the decoder if it has not been call sends an implicit START command to the decoder if it has not been
started yet. started yet. Applies to both queues of mem2mem decoders.
A :ref:`close() <func-close>` or :ref:`VIDIOC_STREAMOFF <VIDIOC_STREAMON>` A :ref:`close() <func-close>` or :ref:`VIDIOC_STREAMOFF <VIDIOC_STREAMON>`
call of a streaming file descriptor sends an implicit immediate STOP call of a streaming file descriptor sends an implicit immediate STOP
command to the decoder, and all buffered data is discarded. command to the decoder, and all buffered data is discarded. Applies to both
queues of mem2mem decoders.
These ioctls are optional, not all drivers may support them. They were In principle, these ioctls are optional, not all drivers may support them. They were
introduced in Linux 3.3. introduced in Linux 3.3. They are, however, mandatory for stateful mem2mem decoders
(as further documented in :ref:`decoder`).
.. tabularcolumns:: |p{1.1cm}|p{2.4cm}|p{1.2cm}|p{1.6cm}|p{10.6cm}| .. tabularcolumns:: |p{1.1cm}|p{2.4cm}|p{1.2cm}|p{1.6cm}|p{10.6cm}|
...@@ -167,26 +169,32 @@ introduced in Linux 3.3. ...@@ -167,26 +169,32 @@ introduced in Linux 3.3.
``V4L2_DEC_CMD_RESUME`` for that. This command has one flag: ``V4L2_DEC_CMD_RESUME`` for that. This command has one flag:
``V4L2_DEC_CMD_START_MUTE_AUDIO``. If set, then audio will be ``V4L2_DEC_CMD_START_MUTE_AUDIO``. If set, then audio will be
muted when playing back at a non-standard speed. muted when playing back at a non-standard speed.
For a device implementing the :ref:`decoder`, once the drain sequence
is initiated with the ``V4L2_DEC_CMD_STOP`` command, it must be driven
to completion before this command can be invoked. Any attempt to
invoke the command while the drain sequence is in progress will trigger
an ``EBUSY`` error code. The command may be also used to restart the
decoder in case of an implicit stop initiated by the decoder itself,
without the ``V4L2_DEC_CMD_STOP`` being called explicitly. See
:ref:`decoder` for more details.
* - ``V4L2_DEC_CMD_STOP`` * - ``V4L2_DEC_CMD_STOP``
- 1 - 1
- Stop the decoder. When the decoder is already stopped, this - Stop the decoder. When the decoder is already stopped, this
command does nothing. This command has two flags: if command does nothing. This command has two flags: if
``V4L2_DEC_CMD_STOP_TO_BLACK`` is set, then the decoder will set ``V4L2_DEC_CMD_STOP_TO_BLACK`` is set, then the decoder will set
the picture to black after it stopped decoding. Otherwise the last the picture to black after it stopped decoding. Otherwise the last
image will repeat. mem2mem decoders will stop producing new frames image will repeat. If
altogether. They will send a ``V4L2_EVENT_EOS`` event when the
last frame has been decoded and all frames are ready to be
dequeued and will set the ``V4L2_BUF_FLAG_LAST`` buffer flag on
the last buffer of the capture queue to indicate there will be no
new buffers produced to dequeue. This buffer may be empty,
indicated by the driver setting the ``bytesused`` field to 0. Once
the ``V4L2_BUF_FLAG_LAST`` flag was set, the
:ref:`VIDIOC_DQBUF <VIDIOC_QBUF>` ioctl will not block anymore,
but return an ``EPIPE`` error code. If
``V4L2_DEC_CMD_STOP_IMMEDIATELY`` is set, then the decoder stops ``V4L2_DEC_CMD_STOP_IMMEDIATELY`` is set, then the decoder stops
immediately (ignoring the ``pts`` value), otherwise it will keep immediately (ignoring the ``pts`` value), otherwise it will keep
decoding until timestamp >= pts or until the last of the pending decoding until timestamp >= pts or until the last of the pending
data from its internal buffers was decoded. data from its internal buffers was decoded.
For a device implementing the :ref:`decoder`, the command will initiate
the drain sequence as documented in :ref:`decoder`. No flags or other
arguments are accepted in this case. Any attempt to invoke the command
again before the sequence completes will trigger an ``EBUSY`` error
code.
* - ``V4L2_DEC_CMD_PAUSE`` * - ``V4L2_DEC_CMD_PAUSE``
- 2 - 2
- Pause the decoder. When the decoder has not been started yet, the - Pause the decoder. When the decoder has not been started yet, the
...@@ -209,6 +217,11 @@ On success 0 is returned, on error -1 and the ``errno`` variable is set ...@@ -209,6 +217,11 @@ On success 0 is returned, on error -1 and the ``errno`` variable is set
appropriately. The generic error codes are described at the appropriately. The generic error codes are described at the
:ref:`Generic Error Codes <gen-errors>` chapter. :ref:`Generic Error Codes <gen-errors>` chapter.
EBUSY
A drain sequence of a device implementing the :ref:`decoder` is still in
progress. It is not allowed to issue another decoder command until it
completes.
EINVAL EINVAL
The ``cmd`` field is invalid. The ``cmd`` field is invalid.
......
...@@ -389,14 +389,19 @@ call. ...@@ -389,14 +389,19 @@ call.
decoder. Applications will have to query the new resolution (if decoder. Applications will have to query the new resolution (if
any, the signal may also have been lost). any, the signal may also have been lost).
For stateful decoders follow the guidelines in :ref:`decoder`.
Video Capture devices have to query the new timings using
:ref:`VIDIOC_QUERY_DV_TIMINGS` or
:ref:`VIDIOC_QUERYSTD <VIDIOC_QUERYSTD>`.
*Important*: even if the new video timings appear identical to the old *Important*: even if the new video timings appear identical to the old
ones, receiving this event indicates that there was an issue with the ones, receiving this event indicates that there was an issue with the
video signal and you must stop and restart streaming video signal and you must stop and restart streaming
(:ref:`VIDIOC_STREAMOFF <VIDIOC_STREAMON>` (:ref:`VIDIOC_STREAMOFF <VIDIOC_STREAMON>`
followed by :ref:`VIDIOC_STREAMON <VIDIOC_STREAMON>`). The reason is followed by :ref:`VIDIOC_STREAMON <VIDIOC_STREAMON>`). The reason is
that many devices are not able to recover from a temporary loss of that many Video Capture devices are not able to recover from a temporary
signal and so restarting streaming I/O is required in order for the loss of signal and so restarting streaming I/O is required in order for
hardware to synchronize to the video signal. the hardware to synchronize to the video signal.
Return Value Return Value
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment