media: visl: add virtual stateless decoder driver

A virtual stateless device for stateless uAPI development purposes.

This tool's objective is to help the development and testing of
userspace applications that use the V4L2 stateless API to decode media.

A userspace implementation can use visl to run a decoding loop even when
no hardware is available or when the kernel uAPI for the codec has not
been upstreamed yet. This can reveal bugs at an early stage.

This driver can also trace the contents of the V4L2 controls submitted
to it.  It can also dump the contents of the vb2 buffers through a
debugfs interface. This is in many ways similar to the tracing
infrastructure available for other popular encode/decode APIs out there
and can help develop a userspace application by using another (working)
one as a reference.

Note that no actual decoding of video frames is performed by visl. The
V4L2 test pattern generator is used to write various debug information
to the capture buffers instead.
Signed-off-by: Daniel Almeida <daniel.almeida@collabora.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>

Documentation/admin-guide/media/v4l-drivers.rst

@@ -31,4 +31,5 @@ Video4Linux (V4L) driver-specific documentation
 	si4713
 	si476x
 	vimc
+	visl
 	vivid

Documentation/admin-guide/media/visl.rst

+.. SPDX-License-Identifier: GPL-2.0
+
+The Virtual Stateless Decoder Driver (visl)
+===========================================
+
+A virtual stateless decoder device for stateless uAPI development
+purposes.
+
+This tool's objective is to help the development and testing of
+userspace applications that use the V4L2 stateless API to decode media.
+
+A userspace implementation can use visl to run a decoding loop even when
+no hardware is available or when the kernel uAPI for the codec has not
+been upstreamed yet. This can reveal bugs at an early stage.
+
+This driver can also trace the contents of the V4L2 controls submitted
+to it.  It can also dump the contents of the vb2 buffers through a
+debugfs interface. This is in many ways similar to the tracing
+infrastructure available for other popular encode/decode APIs out there
+and can help develop a userspace application by using another (working)
+one as a reference.
+
+.. note::
+
+        No actual decoding of video frames is performed by visl. The
+        V4L2 test pattern generator is used to write various debug information
+        to the capture buffers instead.
+
+Module parameters
+-----------------
+
+- visl_debug: Activates debug info, printing various debug messages through
+  dprintk. Also controls whether per-frame debug info is shown. Defaults to off.
+  Note that enabling this feature can result in slow performance through serial.
+
+- visl_transtime_ms: Simulated process time in milliseconds. Slowing down the
+  decoding speed can be useful for debugging.
+
+- visl_dprintk_frame_start, visl_dprintk_frame_nframes: Dictates a range of
+  frames where dprintk is activated. This only controls the dprintk tracing on a
+  per-frame basis. Note that printing a lot of data can be slow through serial.
+
+- keep_bitstream_buffers: Controls whether bitstream (i.e. OUTPUT) buffers are
+  kept after a decoding session. Defaults to false so as to reduce the amount of
+  clutter. keep_bitstream_buffers == false works well when live debugging the
+  client program with GDB.
+
+- bitstream_trace_frame_start, bitstream_trace_nframes: Similar to
+  visl_dprintk_frame_start, visl_dprintk_nframes, but controls the dumping of
+  buffer data through debugfs instead.
+
+What is the default use case for this driver?
+---------------------------------------------
+
+This driver can be used as a way to compare different userspace implementations.
+This assumes that a working client is run against visl and that the ftrace and
+OUTPUT buffer data is subsequently used to debug a work-in-progress
+implementation.
+
+Information on reference frames, their timestamps, the status of the OUTPUT and
+CAPTURE queues and more can be read directly from the CAPTURE buffers.
+
+Supported codecs
+----------------
+
+The following codecs are supported:
+
+- FWHT
+- MPEG2
+- VP8
+- VP9
+- H.264
+- HEVC
+
+visl trace events
+-----------------
+The trace events are defined on a per-codec basis, e.g.:
+
+.. code-block:: bash
+
+        $ ls /sys/kernel/debug/tracing/events/ | grep visl
+        visl_fwht_controls
+        visl_h264_controls
+        visl_hevc_controls
+        visl_mpeg2_controls
+        visl_vp8_controls
+        visl_vp9_controls
+
+For example, in order to dump HEVC SPS data:
+
+.. code-block:: bash
+
+        $ echo 1 >  /sys/kernel/debug/tracing/events/visl_hevc_controls/v4l2_ctrl_hevc_sps/enable
+
+The SPS data will be dumped to the trace buffer, i.e.:
+
+.. code-block:: bash
+
+        $ cat /sys/kernel/debug/tracing/trace
+        video_parameter_set_id 0
+        seq_parameter_set_id 0
+        pic_width_in_luma_samples 1920
+        pic_height_in_luma_samples 1080
+        bit_depth_luma_minus8 0
+        bit_depth_chroma_minus8 0
+        log2_max_pic_order_cnt_lsb_minus4 4
+        sps_max_dec_pic_buffering_minus1 6
+        sps_max_num_reorder_pics 2
+        sps_max_latency_increase_plus1 0
+        log2_min_luma_coding_block_size_minus3 0
+        log2_diff_max_min_luma_coding_block_size 3
+        log2_min_luma_transform_block_size_minus2 0
+        log2_diff_max_min_luma_transform_block_size 3
+        max_transform_hierarchy_depth_inter 2
+        max_transform_hierarchy_depth_intra 2
+        pcm_sample_bit_depth_luma_minus1 0
+        pcm_sample_bit_depth_chroma_minus1 0
+        log2_min_pcm_luma_coding_block_size_minus3 0
+        log2_diff_max_min_pcm_luma_coding_block_size 0
+        num_short_term_ref_pic_sets 0
+        num_long_term_ref_pics_sps 0
+        chroma_format_idc 1
+        sps_max_sub_layers_minus1 0
+        flags AMP_ENABLED|SAMPLE_ADAPTIVE_OFFSET|TEMPORAL_MVP_ENABLED|STRONG_INTRA_SMOOTHING_ENABLED
+
+
+Dumping OUTPUT buffer data through debugfs
+------------------------------------------
+
+If the **VISL_DEBUGFS** Kconfig is enabled, visl will populate
+**/sys/kernel/debug/visl/bitstream** with OUTPUT buffer data according to the
+values of bitstream_trace_frame_start and bitstream_trace_nframes. This can
+highlight errors as broken clients may fail to fill the buffers properly.
+
+A single file is created for each processed OUTPUT buffer. Its name contains an
+integer that denotes the buffer sequence, i.e.:
+
+.. code-block:: c
+
+	snprintf(name, 32, "bitstream%d", run->src->sequence);
+
+Dumping the values is simply a matter of reading from the file, i.e.:
+
+For the buffer with sequence == 0:
+
+.. code-block:: bash
+
+        $ xxd /sys/kernel/debug/visl/bitstream/bitstream0
+        00000000: 2601 af04 d088 bc25 a173 0e41 a4f2 3274  &......%.s.A..2t
+        00000010: c668 cb28 e775 b4ac f53a ba60 f8fd 3aa1  .h.(.u...:.`..:.
+        00000020: 46b4 bcfc 506c e227 2372 e5f5 d7ea 579f  F...Pl.'#r....W.
+        00000030: 6371 5eb5 0eb8 23b5 ca6a 5de5 983a 19e4  cq^...#..j]..:..
+        00000040: e8c3 4320 b4ba a226 cbc1 4138 3a12 32d6  ..C ...&..A8:.2.
+        00000050: fef3 247b 3523 4e90 9682 ac8e eb0c a389  ..${5#N.........
+        00000060: ddd0 6cfc 0187 0e20 7aae b15b 1812 3d33  ..l.... z..[..=3
+        00000070: e1c5 f425 a83a 00b7 4f18 8127 3c4c aefb  ...%.:..O..'<L..
+
+For the buffer with sequence == 1:
+
+.. code-block:: bash
+
+        $ xxd /sys/kernel/debug/visl/bitstream/bitstream1
+        00000000: 0201 d021 49e1 0c40 aa11 1449 14a6 01dc  ...!I..@...I....
+        00000010: 7023 889a c8cd 2cd0 13b4 dab0 e8ca 21fe  p#....,.......!.
+        00000020: c4c8 ab4c 486e 4e2f b0df 96cc c74e 8dde  ...LHnN/.....N..
+        00000030: 8ce7 ee36 d880 4095 4d64 30a0 ff4f 0c5e  ...6..@.Md0..O.^
+        00000040: f16b a6a1 d806 ca2a 0ece a673 7bea 1f37  .k.....*...s{..7
+        00000050: 370f 5bb9 1dc4 ba21 6434 bc53 0173 cba0  7.[....!d4.S.s..
+        00000060: dfe6 bc99 01ea b6e0 346b 92b5 c8de 9f5d  ........4k.....]
+        00000070: e7cc 3484 1769 fef2 a693 a945 2c8b 31da  ..4..i.....E,.1.
+
+And so on.
+
+By default, the files are removed during STREAMOFF. This is to reduce the amount
+of clutter.

MAINTAINERS

@@ -21757,6 +21757,12 @@ F:	include/linux/virtio*.h
 F:	include/uapi/linux/virtio_*.h
 F:	tools/virtio/
 
+VISL VIRTUAL STATELESS DECODER DRIVER
+M:	Daniel Almeida <daniel.almeida@collabora.com>
+L:	linux-media@vger.kernel.org
+S:	Supported
+F:	drivers/media/test-drivers/visl
+
 IFCVF VIRTIO DATA PATH ACCELERATOR
 R:	Zhu Lingshan <lingshan.zhu@intel.com>
 F:	drivers/vdpa/ifcvf/

drivers/media/test-drivers/Kconfig

@@ -20,6 +20,7 @@ config VIDEO_VIM2M
 source "drivers/media/test-drivers/vicodec/Kconfig"
 source "drivers/media/test-drivers/vimc/Kconfig"
 source "drivers/media/test-drivers/vivid/Kconfig"
+source "drivers/media/test-drivers/visl/Kconfig"
 
 endif #V4L_TEST_DRIVERS
 

drivers/media/test-drivers/Makefile

@@ -12,3 +12,4 @@ obj-$(CONFIG_VIDEO_VICODEC) += vicodec/
 obj-$(CONFIG_VIDEO_VIM2M) += vim2m.o
 obj-$(CONFIG_VIDEO_VIMC) += vimc/
 obj-$(CONFIG_VIDEO_VIVID) += vivid/
+obj-$(CONFIG_VIDEO_VISL) += visl/

drivers/media/test-drivers/visl/Kconfig

+# SPDX-License-Identifier: GPL-2.0+
+config VIDEO_VISL
+	tristate "Virtual Stateless Decoder Driver (visl)"
+	depends on VIDEO_DEV
+	select FONT_SUPPORT
+	select FONT_8x16
+	select VIDEOBUF2_VMALLOC
+	select V4L2_MEM2MEM_DEV
+	select MEDIA_CONTROLLER
+	select MEDIA_CONTROLLER_REQUEST_API
+	select VIDEO_V4L2_TPG
+	help
+
+	  A virtual stateless decoder device for uAPI development purposes.
+
+	  A userspace implementation can use visl to run a decoding loop even
+	  when no hardware is available or when the kernel uAPI for the codec
+	  has not been upstreamed yet. This can reveal bugs at an early stage.
+
+	  When in doubt, say N.
+
+config VISL_DEBUGFS
+	bool "Enable debugfs for visl"
+	depends on VIDEO_VISL
+	depends on DEBUG_FS
+
+	help
+	  Choose Y to dump the bitstream buffers through debugfs.
+	  When in doubt, say N.

drivers/media/test-drivers/visl/Makefile

+# SPDX-License-Identifier: GPL-2.0+
+visl-y := visl-core.o visl-video.o visl-dec.o visl-trace-points.o
+
+ifeq ($(CONFIG_VISL_DEBUGFS),y)
+  visl-y += visl-debugfs.o
+endif
+
+obj-$(CONFIG_VIDEO_VISL) += visl.o

drivers/media/test-drivers/visl/visl-debugfs.c

+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * Debugfs tracing for bitstream buffers. This is similar to VA-API's
+ * LIBVA_TRACE_BUFDATA in that the raw bitstream can be dumped as a debugging
+ * aid.
+ *
+ * Produces one file per OUTPUT buffer. Files are automatically cleared on
+ * STREAMOFF unless the module parameter "keep_bitstream_buffers" is set.
+ */
+
+#include <linux/debugfs.h>
+#include <linux/list.h>
+#include <linux/mutex.h>
+#include <media/v4l2-mem2mem.h>
+
+#include "visl-debugfs.h"
+
+int visl_debugfs_init(struct visl_dev *dev)
+{
+	dev->debugfs_root = debugfs_create_dir("visl", NULL);
+	INIT_LIST_HEAD(&dev->bitstream_blobs);
+	mutex_init(&dev->bitstream_lock);
+
+	if (IS_ERR(dev->debugfs_root))
+		return PTR_ERR(dev->debugfs_root);
+
+	return visl_debugfs_bitstream_init(dev);
+}
+
+int visl_debugfs_bitstream_init(struct visl_dev *dev)
+{
+	dev->bitstream_debugfs = debugfs_create_dir("bitstream",
+						    dev->debugfs_root);
+	if (IS_ERR(dev->bitstream_debugfs))
+		return PTR_ERR(dev->bitstream_debugfs);
+
+	return 0;
+}
+
+void visl_trace_bitstream(struct visl_ctx *ctx, struct visl_run *run)
+{
+	u8 *vaddr = vb2_plane_vaddr(&run->src->vb2_buf, 0);
+	struct visl_blob *blob;
+	size_t data_sz = vb2_get_plane_payload(&run->src->vb2_buf, 0);
+	struct dentry *dentry;
+	char name[32];
+
+	blob  = kzalloc(sizeof(*blob), GFP_KERNEL);
+	if (!blob)
+		return;
+
+	blob->blob.data = vzalloc(data_sz);
+	if (!blob->blob.data)
+		goto err_vmalloc;
+
+	blob->blob.size = data_sz;
+	snprintf(name, 32, "bitstream%d", run->src->sequence);
+
+	memcpy(blob->blob.data, vaddr, data_sz);
+
+	dentry = debugfs_create_blob(name, 0444, ctx->dev->bitstream_debugfs,
+				     &blob->blob);
+	if (IS_ERR(dentry))
+		goto err_debugfs;
+
+	blob->dentry = dentry;
+
+	mutex_lock(&ctx->dev->bitstream_lock);
+	list_add_tail(&blob->list, &ctx->dev->bitstream_blobs);
+	mutex_unlock(&ctx->dev->bitstream_lock);
+
+	return;
+
+err_debugfs:
+	vfree(blob->blob.data);
+err_vmalloc:
+	kfree(blob);
+}
+
+void visl_debugfs_clear_bitstream(struct visl_dev *dev)
+{
+	struct visl_blob *blob;
+	struct visl_blob *tmp;
+
+	mutex_lock(&dev->bitstream_lock);
+	if (list_empty(&dev->bitstream_blobs))
+		goto unlock;
+
+	list_for_each_entry_safe(blob, tmp, &dev->bitstream_blobs, list) {
+		list_del(&blob->list);
+		debugfs_remove(blob->dentry);
+		vfree(blob->blob.data);
+		kfree(blob);
+	}
+
+unlock:
+	mutex_unlock(&dev->bitstream_lock);
+}
+
+void visl_debugfs_bitstream_deinit(struct visl_dev *dev)
+{
+	visl_debugfs_clear_bitstream(dev);
+	debugfs_remove_recursive(dev->bitstream_debugfs);
+	dev->bitstream_debugfs = NULL;
+}
+
+void visl_debugfs_deinit(struct visl_dev *dev)
+{
+	visl_debugfs_bitstream_deinit(dev);
+	debugfs_remove_recursive(dev->debugfs_root);
+	dev->debugfs_root = NULL;
+}

drivers/media/test-drivers/visl/visl-debugfs.h

+/* SPDX-License-Identifier: GPL-2.0+ */
+/*
+ * Debugfs tracing for bitstream buffers. This is similar to VA-API's
+ * LIBVA_TRACE_BUFDATA in that the raw bitstream can be dumped as a debugging
+ * aid.
+ *
+ * Produces one file per OUTPUT buffer. Files are automatically cleared on
+ * STREAMOFF unless the module parameter "keep_bitstream_buffers" is set.
+ */
+
+#include "visl.h"
+#include "visl-dec.h"
+
+#ifdef CONFIG_VISL_DEBUGFS
+
+int visl_debugfs_init(struct visl_dev *dev);
+int visl_debugfs_bitstream_init(struct visl_dev *dev);
+void visl_trace_bitstream(struct visl_ctx *ctx, struct visl_run *run);
+void visl_debugfs_clear_bitstream(struct visl_dev *dev);
+void visl_debugfs_bitstream_deinit(struct visl_dev *dev);
+void visl_debugfs_deinit(struct visl_dev *dev);
+
+#else
+
+static inline int visl_debugfs_init(struct visl_dev *dev)
+{
+	return 0;
+}
+
+static inline int visl_debugfs_bitstream_init(struct visl_dev *dev)
+{
+	return 0;
+}
+
+static inline void visl_trace_bitstream(struct visl_ctx *ctx, struct visl_run *run) {}
+static inline void visl_debugfs_clear_bitstream(struct visl_dev *dev) {}
+static inline void visl_debugfs_bitstream_deinit(struct visl_dev *dev) {}
+static inline void visl_debugfs_deinit(struct visl_dev *dev) {}
+
+#endif

drivers/media/test-drivers/visl/visl-dec.h

+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Contains the virtual decoder logic. The functions here control the
+ * tracing/TPG on a per-frame basis
+ */
+
+#ifndef _VISL_DEC_H_
+#define _VISL_DEC_H_
+
+#include "visl.h"
+
+struct visl_fwht_run {
+	const struct v4l2_ctrl_fwht_params *params;
+};
+
+struct visl_mpeg2_run {
+	const struct v4l2_ctrl_mpeg2_sequence *seq;
+	const struct v4l2_ctrl_mpeg2_picture *pic;
+	const struct v4l2_ctrl_mpeg2_quantisation *quant;
+};
+
+struct visl_vp8_run {
+	const struct v4l2_ctrl_vp8_frame *frame;
+};
+
+struct visl_vp9_run {
+	const struct v4l2_ctrl_vp9_frame *frame;
+	const struct v4l2_ctrl_vp9_compressed_hdr *probs;
+};
+
+struct visl_h264_run {
+	const struct v4l2_ctrl_h264_sps *sps;
+	const struct v4l2_ctrl_h264_pps *pps;
+	const struct v4l2_ctrl_h264_scaling_matrix *sm;
+	const struct v4l2_ctrl_h264_slice_params *spram;
+	const struct v4l2_ctrl_h264_decode_params *dpram;
+	const struct v4l2_ctrl_h264_pred_weights *pwht;
+};
+
+struct visl_hevc_run {
+	const struct v4l2_ctrl_hevc_sps *sps;
+	const struct v4l2_ctrl_hevc_pps *pps;
+	const struct v4l2_ctrl_hevc_slice_params *spram;
+	const struct v4l2_ctrl_hevc_scaling_matrix *sm;
+	const struct v4l2_ctrl_hevc_decode_params *dpram;
+};
+
+struct visl_run {
+	struct vb2_v4l2_buffer	*src;
+	struct vb2_v4l2_buffer	*dst;
+
+	union {
+		struct visl_fwht_run	fwht;
+		struct visl_mpeg2_run	mpeg2;
+		struct visl_vp8_run	vp8;
+		struct visl_vp9_run	vp9;
+		struct visl_h264_run	h264;
+		struct visl_hevc_run	hevc;
+	};
+};
+
+int visl_dec_start(struct visl_ctx *ctx);
+int visl_dec_stop(struct visl_ctx *ctx);
+int visl_job_ready(void *priv);
+void visl_device_run(void *priv);
+
+#endif /* _VISL_DEC_H_ */

drivers/media/test-drivers/visl/visl-trace-fwht.h

+/* SPDX-License-Identifier: GPL-2.0 */
+#if !defined(_VISL_TRACE_FWHT_H_) || defined(TRACE_HEADER_MULTI_READ)
+#define _VISL_TRACE_FWHT_H_
+
+#include <linux/tracepoint.h>
+#include "visl.h"
+
+#undef TRACE_SYSTEM
+#define TRACE_SYSTEM visl_fwht_controls
+
+DECLARE_EVENT_CLASS(v4l2_ctrl_fwht_params_tmpl,
+	TP_PROTO(const struct v4l2_ctrl_fwht_params *p),
+	TP_ARGS(p),
+	TP_STRUCT__entry(
+			 __field(u64, backward_ref_ts)
+			 __field(u32, version)
+			 __field(u32, width)
+			 __field(u32, height)
+			 __field(u32, flags)
+			 __field(u32, colorspace)
+			 __field(u32, xfer_func)
+			 __field(u32, ycbcr_enc)
+			 __field(u32, quantization)
+			 ),
+	TP_fast_assign(
+		       __entry->backward_ref_ts = p->backward_ref_ts;
+		       __entry->version = p->version;
+		       __entry->width = p->width;
+		       __entry->height = p->height;
+		       __entry->flags = p->flags;
+		       __entry->colorspace = p->colorspace;
+		       __entry->xfer_func = p->xfer_func;
+		       __entry->ycbcr_enc = p->ycbcr_enc;
+		       __entry->quantization = p->quantization;
+		       ),
+	TP_printk("backward_ref_ts %llu version %u width %u height %u flags %s colorspace %u xfer_func %u ycbcr_enc %u quantization %u",
+		  __entry->backward_ref_ts, __entry->version, __entry->width, __entry->height,
+		  __print_flags(__entry->flags, "|",
+		  {V4L2_FWHT_FL_IS_INTERLACED, "IS_INTERLACED"},
+		  {V4L2_FWHT_FL_IS_BOTTOM_FIRST, "IS_BOTTOM_FIRST"},
+		  {V4L2_FWHT_FL_IS_ALTERNATE, "IS_ALTERNATE"},
+		  {V4L2_FWHT_FL_IS_BOTTOM_FIELD, "IS_BOTTOM_FIELD"},
+		  {V4L2_FWHT_FL_LUMA_IS_UNCOMPRESSED, "LUMA_IS_UNCOMPRESSED"},
+		  {V4L2_FWHT_FL_CB_IS_UNCOMPRESSED, "CB_IS_UNCOMPRESSED"},
+		  {V4L2_FWHT_FL_CR_IS_UNCOMPRESSED, "CR_IS_UNCOMPRESSED"},
+		  {V4L2_FWHT_FL_ALPHA_IS_UNCOMPRESSED, "ALPHA_IS_UNCOMPRESSED"},
+		  {V4L2_FWHT_FL_I_FRAME, "I_FRAME"},
+		  {V4L2_FWHT_FL_PIXENC_HSV, "PIXENC_HSV"},
+		  {V4L2_FWHT_FL_PIXENC_RGB, "PIXENC_RGB"},
+		  {V4L2_FWHT_FL_PIXENC_YUV, "PIXENC_YUV"}),
+		  __entry->colorspace, __entry->xfer_func, __entry->ycbcr_enc,
+		  __entry->quantization)
+);
+
+DEFINE_EVENT(v4l2_ctrl_fwht_params_tmpl, v4l2_ctrl_fwht_params,
+	TP_PROTO(const struct v4l2_ctrl_fwht_params *p),
+	TP_ARGS(p)
+);
+
+#endif
+
+#undef TRACE_INCLUDE_PATH
+#undef TRACE_INCLUDE_FILE
+#define TRACE_INCLUDE_PATH ../../drivers/media/test-drivers/visl
+#define TRACE_INCLUDE_FILE visl-trace-fwht
+#include <trace/define_trace.h>

drivers/media/test-drivers/visl/visl-trace-mpeg2.h

+/* SPDX-License-Identifier: GPL-2.0 */
+#if !defined(_VISL_TRACE_MPEG2_H_) || defined(TRACE_HEADER_MULTI_READ)
+#define _VISL_TRACE_MPEG2_H_
+
+#include <linux/tracepoint.h>
+#include "visl.h"
+
+#undef TRACE_SYSTEM
+#define TRACE_SYSTEM visl_mpeg2_controls
+
+DECLARE_EVENT_CLASS(v4l2_ctrl_mpeg2_seq_tmpl,
+	TP_PROTO(const struct v4l2_ctrl_mpeg2_sequence *s),
+	TP_ARGS(s),
+	TP_STRUCT__entry(__field_struct(struct v4l2_ctrl_mpeg2_sequence, s)),
+	TP_fast_assign(__entry->s = *s;),
+	TP_printk("\nhorizontal_size %u\nvertical_size %u\nvbv_buffer_size %u\n"
+		  "profile_and_level_indication %u\nchroma_format %u\nflags %s\n",
+		  __entry->s.horizontal_size,
+		  __entry->s.vertical_size,
+		  __entry->s.vbv_buffer_size,
+		  __entry->s.profile_and_level_indication,
+		  __entry->s.chroma_format,
+		  __print_flags(__entry->s.flags, "|",
+		  {V4L2_MPEG2_SEQ_FLAG_PROGRESSIVE, "PROGRESSIVE"})
+	)
+);
+
+DECLARE_EVENT_CLASS(v4l2_ctrl_mpeg2_pic_tmpl,
+	TP_PROTO(const struct v4l2_ctrl_mpeg2_picture *p),
+	TP_ARGS(p),
+	TP_STRUCT__entry(__field_struct(struct v4l2_ctrl_mpeg2_picture, p)),
+	TP_fast_assign(__entry->p = *p;),
+	TP_printk("\nbackward_ref_ts %llu\nforward_ref_ts %llu\nflags %s\nf_code {%s}\n"
+		  "picture_coding_type: %u\npicture_structure %u\nintra_dc_precision %u\n",
+		  __entry->p.backward_ref_ts,
+		  __entry->p.forward_ref_ts,
+		  __print_flags(__entry->p.flags, "|",
+		  {V4L2_MPEG2_PIC_FLAG_TOP_FIELD_FIRST, "TOP_FIELD_FIRST"},
+		  {V4L2_MPEG2_PIC_FLAG_FRAME_PRED_DCT, "FRAME_PRED_DCT"},
+		  {V4L2_MPEG2_PIC_FLAG_CONCEALMENT_MV, "CONCEALMENT_MV"},
+		  {V4L2_MPEG2_PIC_FLAG_Q_SCALE_TYPE, "Q_SCALE_TYPE"},
+		  {V4L2_MPEG2_PIC_FLAG_INTRA_VLC, "INTA_VLC"},
+		  {V4L2_MPEG2_PIC_FLAG_ALT_SCAN, "ALT_SCAN"},
+		  {V4L2_MPEG2_PIC_FLAG_REPEAT_FIRST, "REPEAT_FIRST"},
+		  {V4L2_MPEG2_PIC_FLAG_PROGRESSIVE, "PROGRESSIVE"}),
+		  __print_hex_dump("", DUMP_PREFIX_NONE, 32, 1,
+				   __entry->p.f_code,
+				   sizeof(__entry->p.f_code),
+				   false),
+		  __entry->p.picture_coding_type,
+		  __entry->p.picture_structure,
+		  __entry->p.intra_dc_precision
+	)
+);
+
+DECLARE_EVENT_CLASS(v4l2_ctrl_mpeg2_quant_tmpl,
+	TP_PROTO(const struct v4l2_ctrl_mpeg2_quantisation *q),
+	TP_ARGS(q),
+	TP_STRUCT__entry(__field_struct(struct v4l2_ctrl_mpeg2_quantisation, q)),
+	TP_fast_assign(__entry->q = *q;),
+	TP_printk("\nintra_quantiser_matrix %s\nnon_intra_quantiser_matrix %s\n"
+		  "chroma_intra_quantiser_matrix %s\nchroma_non_intra_quantiser_matrix %s\n",
+		  __print_array(__entry->q.intra_quantiser_matrix,
+				ARRAY_SIZE(__entry->q.intra_quantiser_matrix),
+				sizeof(__entry->q.intra_quantiser_matrix[0])),
+		  __print_array(__entry->q.non_intra_quantiser_matrix,
+				ARRAY_SIZE(__entry->q.non_intra_quantiser_matrix),
+				sizeof(__entry->q.non_intra_quantiser_matrix[0])),
+		  __print_array(__entry->q.chroma_intra_quantiser_matrix,
+				ARRAY_SIZE(__entry->q.chroma_intra_quantiser_matrix),
+				sizeof(__entry->q.chroma_intra_quantiser_matrix[0])),
+		  __print_array(__entry->q.chroma_non_intra_quantiser_matrix,
+				ARRAY_SIZE(__entry->q.chroma_non_intra_quantiser_matrix),
+				sizeof(__entry->q.chroma_non_intra_quantiser_matrix[0]))
+		  )
+)
+
+DEFINE_EVENT(v4l2_ctrl_mpeg2_seq_tmpl, v4l2_ctrl_mpeg2_sequence,
+	TP_PROTO(const struct v4l2_ctrl_mpeg2_sequence *s),
+	TP_ARGS(s)
+);
+
+DEFINE_EVENT(v4l2_ctrl_mpeg2_pic_tmpl, v4l2_ctrl_mpeg2_picture,
+	TP_PROTO(const struct v4l2_ctrl_mpeg2_picture *p),
+	TP_ARGS(p)
+);
+
+DEFINE_EVENT(v4l2_ctrl_mpeg2_quant_tmpl, v4l2_ctrl_mpeg2_quantisation,
+	TP_PROTO(const struct v4l2_ctrl_mpeg2_quantisation *q),
+	TP_ARGS(q)
+);
+
+#endif
+
+#undef TRACE_INCLUDE_PATH
+#undef TRACE_INCLUDE_FILE
+#define TRACE_INCLUDE_PATH ../../drivers/media/test-drivers/visl
+#define TRACE_INCLUDE_FILE visl-trace-mpeg2
+#include <trace/define_trace.h>

drivers/media/test-drivers/visl/visl-trace-points.c

+// SPDX-License-Identifier: GPL-2.0
+#include "visl.h"
+
+#define CREATE_TRACE_POINTS
+#include "visl-trace-fwht.h"
+#include "visl-trace-mpeg2.h"
+#include "visl-trace-vp8.h"
+#include "visl-trace-vp9.h"
+#include "visl-trace-h264.h"
+#include "visl-trace-hevc.h"

drivers/media/test-drivers/visl/visl-trace-vp8.h

+/* SPDX-License-Identifier: GPL-2.0 */
+#if !defined(_VISL_TRACE_VP8_H_) || defined(TRACE_HEADER_MULTI_READ)
+#define _VISL_TRACE_VP8_H_
+
+#include <linux/tracepoint.h>
+#include "visl.h"
+
+#undef TRACE_SYSTEM
+#define TRACE_SYSTEM visl_vp8_controls
+
+DECLARE_EVENT_CLASS(v4l2_ctrl_vp8_entropy_tmpl,
+	TP_PROTO(const struct v4l2_ctrl_vp8_frame *f),
+	TP_ARGS(f),
+	TP_STRUCT__entry(__field_struct(struct v4l2_ctrl_vp8_frame, f)),
+	TP_fast_assign(__entry->f = *f;),
+	TP_printk("\nentropy.coeff_probs {%s}\n"
+		  "entropy.y_mode_probs %s\n"
+		  "entropy.uv_mode_probs %s\n"
+		  "entropy.mv_probs {%s}",
+		  __print_hex_dump("", DUMP_PREFIX_NONE, 32, 1,
+				   __entry->f.entropy.coeff_probs,
+				   sizeof(__entry->f.entropy.coeff_probs),
+				   false),
+		  __print_array(__entry->f.entropy.y_mode_probs,
+				ARRAY_SIZE(__entry->f.entropy.y_mode_probs),
+				sizeof(__entry->f.entropy.y_mode_probs[0])),
+		  __print_array(__entry->f.entropy.uv_mode_probs,
+				ARRAY_SIZE(__entry->f.entropy.uv_mode_probs),
+				sizeof(__entry->f.entropy.uv_mode_probs[0])),
+		  __print_hex_dump("", DUMP_PREFIX_NONE, 32, 1,
+				   __entry->f.entropy.mv_probs,
+				   sizeof(__entry->f.entropy.mv_probs),
+				   false)
+		  )
+)
+
+DECLARE_EVENT_CLASS(v4l2_ctrl_vp8_frame_tmpl,
+	TP_PROTO(const struct v4l2_ctrl_vp8_frame *f),
+	TP_ARGS(f),
+	TP_STRUCT__entry(__field_struct(struct v4l2_ctrl_vp8_frame, f)),
+	TP_fast_assign(__entry->f = *f;),
+	TP_printk("\nsegment.quant_update %s\n"
+		  "segment.lf_update %s\n"
+		  "segment.segment_probs %s\n"
+		  "segment.flags %s\n"
+		  "lf.ref_frm_delta %s\n"
+		  "lf.mb_mode_delta %s\n"
+		  "lf.sharpness_level %u\n"
+		  "lf.level %u\n"
+		  "lf.flags %s\n"
+		  "quant.y_ac_qi %u\n"
+		  "quant.y_dc_delta %d\n"
+		  "quant.y2_dc_delta %d\n"
+		  "quant.y2_ac_delta %d\n"
+		  "quant.uv_dc_delta %d\n"
+		  "quant.uv_ac_delta %d\n"
+		  "coder_state.range %u\n"
+		  "coder_state.value %u\n"
+		  "coder_state.bit_count %u\n"
+		  "width %u\n"
+		  "height %u\n"
+		  "horizontal_scale %u\n"
+		  "vertical_scale %u\n"
+		  "version %u\n"
+		  "prob_skip_false %u\n"
+		  "prob_intra %u\n"
+		  "prob_last %u\n"
+		  "prob_gf %u\n"
+		  "num_dct_parts %u\n"
+		  "first_part_size %u\n"
+		  "first_part_header_bits %u\n"
+		  "dct_part_sizes %s\n"
+		  "last_frame_ts %llu\n"
+		  "golden_frame_ts %llu\n"
+		  "alt_frame_ts %llu\n"
+		  "flags %s",
+		  __print_array(__entry->f.segment.quant_update,
+				ARRAY_SIZE(__entry->f.segment.quant_update),
+				sizeof(__entry->f.segment.quant_update[0])),
+		  __print_array(__entry->f.segment.lf_update,
+				ARRAY_SIZE(__entry->f.segment.lf_update),
+				sizeof(__entry->f.segment.lf_update[0])),
+		  __print_array(__entry->f.segment.segment_probs,
+				ARRAY_SIZE(__entry->f.segment.segment_probs),
+				sizeof(__entry->f.segment.segment_probs[0])),
+		  __print_flags(__entry->f.segment.flags, "|",
+		  {V4L2_VP8_SEGMENT_FLAG_ENABLED, "SEGMENT_ENABLED"},
+		  {V4L2_VP8_SEGMENT_FLAG_UPDATE_MAP, "SEGMENT_UPDATE_MAP"},
+		  {V4L2_VP8_SEGMENT_FLAG_UPDATE_FEATURE_DATA, "SEGMENT_UPDATE_FEATURE_DATA"},
+		  {V4L2_VP8_SEGMENT_FLAG_DELTA_VALUE_MODE, "SEGMENT_DELTA_VALUE_MODE"}),
+		  __print_array(__entry->f.lf.ref_frm_delta,
+				ARRAY_SIZE(__entry->f.lf.ref_frm_delta),
+				sizeof(__entry->f.lf.ref_frm_delta[0])),
+		  __print_array(__entry->f.lf.mb_mode_delta,
+				ARRAY_SIZE(__entry->f.lf.mb_mode_delta),
+				sizeof(__entry->f.lf.mb_mode_delta[0])),
+		  __entry->f.lf.sharpness_level,
+		  __entry->f.lf.level,
+		  __print_flags(__entry->f.lf.flags, "|",
+		  {V4L2_VP8_LF_ADJ_ENABLE, "LF_ADJ_ENABLED"},
+		  {V4L2_VP8_LF_DELTA_UPDATE, "LF_DELTA_UPDATE"},
+		  {V4L2_VP8_LF_FILTER_TYPE_SIMPLE, "LF_FILTER_TYPE_SIMPLE"}),
+		  __entry->f.quant.y_ac_qi,
+		  __entry->f.quant.y_dc_delta,
+		  __entry->f.quant.y2_dc_delta,
+		  __entry->f.quant.y2_ac_delta,
+		  __entry->f.quant.uv_dc_delta,
+		  __entry->f.quant.uv_ac_delta,
+		  __entry->f.coder_state.range,
+		  __entry->f.coder_state.value,
+		  __entry->f.coder_state.bit_count,
+		  __entry->f.width,
+		  __entry->f.height,
+		  __entry->f.horizontal_scale,
+		  __entry->f.vertical_scale,
+		  __entry->f.version,
+		  __entry->f.prob_skip_false,
+		  __entry->f.prob_intra,
+		  __entry->f.prob_last,
+		  __entry->f.prob_gf,
+		  __entry->f.num_dct_parts,
+		  __entry->f.first_part_size,
+		  __entry->f.first_part_header_bits,
+		  __print_array(__entry->f.dct_part_sizes,
+				ARRAY_SIZE(__entry->f.dct_part_sizes),
+				sizeof(__entry->f.dct_part_sizes[0])),
+		  __entry->f.last_frame_ts,
+		  __entry->f.golden_frame_ts,
+		  __entry->f.alt_frame_ts,
+		  __print_flags(__entry->f.flags, "|",
+		  {V4L2_VP8_FRAME_FLAG_KEY_FRAME, "KEY_FRAME"},
+		  {V4L2_VP8_FRAME_FLAG_EXPERIMENTAL, "EXPERIMENTAL"},
+		  {V4L2_VP8_FRAME_FLAG_SHOW_FRAME, "SHOW_FRAME"},
+		  {V4L2_VP8_FRAME_FLAG_MB_NO_SKIP_COEFF, "MB_NO_SKIP_COEFF"},
+		  {V4L2_VP8_FRAME_FLAG_SIGN_BIAS_GOLDEN, "SIGN_BIAS_GOLDEN"},
+		  {V4L2_VP8_FRAME_FLAG_SIGN_BIAS_ALT, "SIGN_BIAS_ALT"})
+		  )
+);
+
+DEFINE_EVENT(v4l2_ctrl_vp8_frame_tmpl, v4l2_ctrl_vp8_frame,
+	TP_PROTO(const struct v4l2_ctrl_vp8_frame *f),
+	TP_ARGS(f)
+);
+
+DEFINE_EVENT(v4l2_ctrl_vp8_entropy_tmpl, v4l2_ctrl_vp8_entropy,
+	TP_PROTO(const struct v4l2_ctrl_vp8_frame *f),
+	TP_ARGS(f)
+);
+
+#endif
+
+#undef TRACE_INCLUDE_PATH
+#undef TRACE_INCLUDE_FILE
+#define TRACE_INCLUDE_PATH ../../drivers/media/test-drivers/visl
+#define TRACE_INCLUDE_FILE visl-trace-vp8
+#include <trace/define_trace.h>

drivers/media/test-drivers/visl/visl-video.h

+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Contains the driver implementation for the V4L2 stateless interface.
+ */
+
+#ifndef _VISL_VIDEO_H_
+#define _VISL_VIDEO_H_
+#include <media/v4l2-mem2mem.h>
+
+#include "visl.h"
+
+extern const struct v4l2_ioctl_ops visl_ioctl_ops;
+
+extern const struct visl_ctrls visl_fwht_ctrls;
+extern const struct visl_ctrls visl_mpeg2_ctrls;
+extern const struct visl_ctrls visl_vp8_ctrls;
+extern const struct visl_ctrls visl_vp9_ctrls;
+extern const struct visl_ctrls visl_h264_ctrls;
+extern const struct visl_ctrls visl_hevc_ctrls;
+
+int visl_queue_init(void *priv, struct vb2_queue *src_vq,
+		    struct vb2_queue *dst_vq);
+
+int visl_set_default_format(struct visl_ctx *ctx);
+int visl_request_validate(struct media_request *req);
+
+#endif /* _VISL_VIDEO_H_ */

drivers/media/test-drivers/visl/visl.h

+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * A virtual stateless device for stateless uAPI development purposes.
+ *
+ * This tool's objective is to help the development and testing of userspace
+ * applications that use the V4L2 stateless API to decode media.
+ *
+ * A userspace implementation can use visl to run a decoding loop even when no
+ * hardware is available or when the kernel uAPI for the codec has not been
+ * upstreamed yet. This can reveal bugs at an early stage.
+ *
+ * This driver can also trace the contents of the V4L2 controls submitted to it.
+ * It can also dump the contents of the vb2 buffers through a debugfs
+ * interface. This is in many ways similar to the tracing infrastructure
+ * available for other popular encode/decode APIs out there and can help develop
+ * a userspace application by using another (working) one as a reference.
+ *
+ * Note that no actual decoding of video frames is performed by visl. The V4L2
+ * test pattern generator is used to write various debug information to the
+ * capture buffers instead.
+ *
+ * Copyright (C) 2022 Collabora, Ltd.
+ *
+ * Based on the vim2m driver, that is:
+ *
+ * Copyright (c) 2009-2010 Samsung Electronics Co., Ltd.
+ * Pawel Osciak, <pawel@osciak.com>
+ * Marek Szyprowski, <m.szyprowski@samsung.com>
+ *
+ * Based on the vicodec driver, that is:
+ *
+ * Copyright 2018 Cisco Systems, Inc. and/or its affiliates. All rights reserved.
+ *
+ * Based on the Cedrus VPU driver, that is:
+ *
+ * Copyright (C) 2016 Florent Revest <florent.revest@free-electrons.com>
+ * Copyright (C) 2018 Paul Kocialkowski <paul.kocialkowski@bootlin.com>
+ * Copyright (C) 2018 Bootlin
+ */
+
+#ifndef _VISL_H_
+#define _VISL_H_
+
+#include <linux/debugfs.h>
+#include <linux/list.h>
+
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-device.h>
+#include <media/tpg/v4l2-tpg.h>
+
+#define VISL_NAME		"visl"
+#define VISL_M2M_NQUEUES	2
+
+#define TPG_STR_BUF_SZ		2048
+
+extern unsigned int visl_transtime_ms;
+
+struct visl_ctrls {
+	const struct visl_ctrl_desc *ctrls;
+	unsigned int num_ctrls;
+};
+
+struct visl_coded_format_desc {
+	u32 pixelformat;
+	struct v4l2_frmsize_stepwise frmsize;
+	const struct visl_ctrls *ctrls;
+	unsigned int num_decoded_fmts;
+	const u32 *decoded_fmts;
+};
+
+extern const struct visl_coded_format_desc visl_coded_fmts[];
+extern const size_t num_coded_fmts;
+
+enum {
+	V4L2_M2M_SRC = 0,
+	V4L2_M2M_DST = 1,
+};
+
+extern unsigned int visl_debug;
+#define dprintk(dev, fmt, arg...) \
+	v4l2_dbg(1, visl_debug, &(dev)->v4l2_dev, "%s: " fmt, __func__, ## arg)
+
+extern int visl_dprintk_frame_start;
+extern unsigned int visl_dprintk_nframes;
+extern bool keep_bitstream_buffers;
+extern int bitstream_trace_frame_start;
+extern unsigned int bitstream_trace_nframes;
+
+#define frame_dprintk(dev, current, fmt, arg...) \
+	do { \
+		if (visl_dprintk_frame_start > -1 && \
+		    (current) >= visl_dprintk_frame_start && \
+		    (current) < visl_dprintk_frame_start + visl_dprintk_nframes) \
+			dprintk(dev, fmt, ## arg); \
+	} while (0) \
+
+struct visl_q_data {
+	unsigned int		sequence;
+};
+
+struct visl_dev {
+	struct v4l2_device	v4l2_dev;
+	struct video_device	vfd;
+#ifdef CONFIG_MEDIA_CONTROLLER
+	struct media_device	mdev;
+#endif
+
+	struct mutex		dev_mutex;
+
+	struct v4l2_m2m_dev	*m2m_dev;
+
+#ifdef CONFIG_VISL_DEBUGFS
+	struct dentry		*debugfs_root;
+	struct dentry		*bitstream_debugfs;
+	struct list_head	bitstream_blobs;
+
+	/* Protects the "blob" list */
+	struct mutex		bitstream_lock;
+#endif
+};
+
+enum visl_codec {
+	VISL_CODEC_NONE,
+	VISL_CODEC_FWHT,
+	VISL_CODEC_MPEG2,
+	VISL_CODEC_VP8,
+	VISL_CODEC_VP9,
+	VISL_CODEC_H264,
+	VISL_CODEC_HEVC,
+};
+
+struct visl_blob {
+	struct list_head list;
+	struct dentry *dentry;
+	struct debugfs_blob_wrapper blob;
+};
+
+struct visl_ctx {
+	struct v4l2_fh		fh;
+	struct visl_dev	*dev;
+	struct v4l2_ctrl_handler hdl;
+
+	struct mutex		vb_mutex;
+
+	struct visl_q_data	q_data[VISL_M2M_NQUEUES];
+	enum   visl_codec	current_codec;
+
+	const struct visl_coded_format_desc *coded_format_desc;
+
+	struct v4l2_format	coded_fmt;
+	struct v4l2_format	decoded_fmt;
+
+	struct tpg_data		tpg;
+	u64			capture_streamon_jiffies;
+	char			*tpg_str_buf;
+};
+
+struct visl_ctrl_desc {
+	struct v4l2_ctrl_config cfg;
+};
+
+static inline struct visl_ctx *visl_file_to_ctx(struct file *file)
+{
+	return container_of(file->private_data, struct visl_ctx, fh);
+}
+
+static inline struct visl_ctx *visl_v4l2fh_to_ctx(struct v4l2_fh *v4l2_fh)
+{
+	return container_of(v4l2_fh, struct visl_ctx, fh);
+}
+
+void *visl_find_control_data(struct visl_ctx *ctx, u32 id);
+struct v4l2_ctrl *visl_find_control(struct visl_ctx *ctx, u32 id);
+u32 visl_control_num_elems(struct visl_ctx *ctx, u32 id);
+
+#endif /* _VISL_H_ */