56 Commits

Author SHA1 Message Date
Daniel Morin
3eefb6c5db analytics: always add GstTensorMeta
Always add GstTensorMeta to buffer instead of re-using existing GstTensorMeta
when it's already present. The reason for the change is when we have cascaded
inference elements overwriting (tflite) or leaking (ONNX) GstTensorMeta when we
have cascaded inferences. We didn't create and API to append to GstTensorMeta
and I think it's more convenient to have a new GstTensorMeta for each
inferences. It's also more clear that a group of tensor was produced by one
inference but doesn't limit tensordecoders from using tensors produced by
multiple inference. I don't see much value in having all tensors data always
inside one GstTensorMeta since appending would mean re-allocation of the tensors
array anyway.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/9564>
2025-08-22 23:10:31 +00:00
Olivier Crête
561e2b28af tensor: Print tensor name in debug name
It makes it easier to understand which one is rejected.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/9419>
2025-07-21 14:49:42 +00:00
Olivier Crête
2c3f2747d6 tensormeta: Check dimensions when retrieving tensor
Modify the API to retrieve the tensor meta to check for the dimensions
as well.

Also fix an API mistake, the buffer whose dimensions should be checheck
is the one inside the GstTensor, not another buffer some outside.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/9411>
2025-07-17 17:43:32 +00:00
Olivier Crête
5f52eaae83 tensormeta: Don't crash on invalid tensor name
It's a valid case to check for an existing tensor.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/9411>
2025-07-17 17:43:32 +00:00
Sebastian Dröge
d255ed96f0 tensor: Clarify meaning of the dimensions array in the docs
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/9325>
2025-07-17 11:25:59 +00:00
Olivier Crête
b11ccc5cdf analyticsmeta: Remove incorrect check
The value can be NULL which is the wildcard

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/9413>
2025-07-16 18:55:55 -04:00
Sebastian Dröge
6ce4637aab analytics: Include new batch meta in the single-include header
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/9391>
2025-07-14 15:07:37 +03:00
Sebastian Dröge
3a51f42ab0 analytics: Add GstAnalyticsBatchMeta for batches of buffers from one or more streams
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/9282>
2025-07-13 18:25:26 +02:00
Sebastian Dröge
8711294ea0 analytics: Fix docs of gst_tensor_check_type()
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/9311>
2025-06-30 11:13:24 +00:00
Sebastian Dröge
bebec6d965 analytics: Fix transfer annotations of gst_tensor_check_type()
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/9293>
2025-06-30 08:00:17 +00:00
Daniel Morin
9bd3a3be74 gsttensor: adding new datatypes
- Adding datatype for string, boolean, complex numbers and special floating
  point numbers.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/9172>
2025-06-10 09:04:48 +00:00
raghu447
4b6c732fdd analytics: add a convenient API to retrieve tensor
use the API in facedetector tensor decoding

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/9162>
2025-06-09 10:06:43 +05:30
Olivier Crête
5c188d90c0 tensor: Add helper function to stringify a tensor data type
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8523>
2025-05-20 16:05:51 +00:00
Santosh Mahto
700b665bf7 gstanalytics: Add transform function to copy the tensor meta
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8825>
2025-05-10 20:58:52 +00:00
Daniel Morin
93af941ce7 analytics: add more convenient API to retrieve tensor
`gst_tensor_meta_get_by_id (meta,id)' is more convenient then
retrieving the tensor index using `gst_tensor_meta_get_index_from_id()` followed
by `gst_tensor_meta_get ()`.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8848>
2025-04-15 16:17:47 -04:00
Daniel Morin
e887b2e20e analytics: Move IoU calculation to gstanalytics lib
Calculating intersection-of-union (IoU) is a very common operation used by
tensor-decoder handling tensors from vision models. Having this in a library
will improve maintainability and ease of writing tensor-decoder.

- Post-fix _uint: We might eventually want to handle different datatype that we
woule post-fix with _type

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8805>
2025-04-14 23:15:11 +00:00
Jan Tojnar
bc93bbf5c8 gst-analytics: Add gst-video to Requires in pkg-config
`gst/analytics/analytics.h` includes `gst/analytics/gstanalyticssegmentationmtd.h`,
which in turn `gst/video/video-info.h` but `gst-video-1.0` was only listed
in `Requires.private` field of `gst-analytics-1.0.pc`.

This would cause projects linking against `gst-analytics-1.0.pc` to fail to find
the headers when using alternative interpretation of pkg-config specification
that only considers private dependencies for include path during static builds,
such as the case e.g. on Nix.

https://gitlab.freedesktop.org/pkg-config/pkg-config/-/issues/28

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8661>
2025-03-26 23:40:04 +00:00
Olivier Crête
3dc6abbe68 analyticsmeta: Make output struct annotation more explicit
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8489>
2025-02-17 15:25:51 +00:00
Olivier Crête
7b6ba90416 analyticsmeta: Avoid crash when adding Mtd with NULL Mtd structure
It's documented that you don't need to get the position of the Mtd
when adding it.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8489>
2025-02-17 15:25:50 +00:00
Mathieu Duponchelle
7983ecff1c docs: generate hotdoc configs for libraries with our helper script
With this patch, configure time is identical no matter whether doc is
enabled or not.

The configuration files also now contain explicitly-listed sources with
no wildcards.

For the four libraries where hotdoc needs to use clang to generate the
documentation (as opposed to the rest of the libraries where hotdoc uses
the gir), the script will call pkg-config to determine the appropriate
C flags.

This means a side effect of this patch is that pkg-config files are now
generated for the gstadaptivedemux and gstopencv libraries.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8312>
2025-01-17 20:36:06 +01:00
Daniel Morin
5919b20e8b gst-analytics: add missing mtd segmentation API
- add gst_analytics_segmentation_mtd_get_mtd_type() which is required to
  retrieve the concrete type of a generic mtd (GstAnalyticsMtd).

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8307>
2025-01-15 17:27:15 -05:00
Olivier Crête
33259e7ea4 analytics: Tensor dimensions are always row-major or col-major
Simplify by removing the extra fields, as this is what all
frameworks give us.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8250>
2025-01-08 21:47:52 +00:00
Daniel Morin
0a343fa738 analytics: remove batch-size
- Batch-size will be the outer-most dimension. Presence of batch dimension can
  be identified using `dims` and `id`.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8191>
2025-01-03 23:14:39 +00:00
Oskar Fiedot
a3a7acdd0a analytics: add rotation to object detection mtd
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7938>
2024-12-11 11:54:15 +01:00
Daniel Morin
a1e623d200 analytics: add _N_TO_N relation type
This relation type define relations between each components of two groups.
First component of first group relate to first component of second group,
Second component of second group relate to second component of second group,
and so on. It's a denser way to express relations in this context.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8087>
2024-12-09 18:18:30 +00:00
Olivier Crête
d934ea3f7a tensormeta: Add APIs to create and access GstTensorMeta contents
Also document those APIs better.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6000>
2024-11-08 14:58:49 +00:00
Olivier Crête
5e73e8e1b3 tensor: Add APIs to create and access GstTensor contents
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6000>
2024-11-08 14:58:49 +00:00
Olivier Crête
4295386804 tensors: Use full GstTensorDataType type name in type members
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6000>
2024-11-08 14:58:49 +00:00
Olivier Crête
e01a3b1d79 analytics: Add APIs to add or get a GstTensorMeta
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6000>
2024-11-08 14:58:49 +00:00
Daniel Morin
6a5a63f051 analytics: Adding abstraction on tensor dims
Tensor can be row or col major, but it's also possible that the order by we need
to read the tensor with more than two dimension need to be described. The
reserved field in GstTensorDim is there for this purpose. If we need this we
can add  GST_TENSOR_DIM_ORDER_INDEXED, and follow an index defining order for
each dimension.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6000>
2024-11-08 14:58:49 +00:00
Daniel Morin
8169863f01 analytics: Make GstTensor more suitable for inline allocation
GstTensor contained two fields (data, dims) that were dynamicallay allocated. For
data it's for a GstBuffer and we have pool for efficient memory management. For
dims it's a small array to store the dimension of the tensor. The dims field
can be allocated inplace by moving it at the end of the structure. This will
allow a better memory management when GstTensor is stored in an analytics meta
which will take advantage of the _clear interface for re-use.

- New api to allocate and free GstTensor
To continue to support use-cases where GstTensor is not stored in an
analytics-meta we provide gst_tensor_alloc, gst_tensor_alloc_n and
gst_tensor_free that will facilitate memory management.
- Make GstTensor a boxed type

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6000>
2024-11-08 14:58:49 +00:00
Daniel Morin
7c925eae61 analytics: Move batch to GstTensor
- batch_size is required to interpret the tensor depending on the tensor format
the batch are not necessarily memory plane therefore it's preferable to keep it
inside GstTensor.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6000>
2024-11-08 14:58:49 +00:00
Daniel Morin
43c7e524ce analytics: Decouple GstTensor from GstTensorMeta
- To support transporting tensor as GstMeta, Analytics-Meta and Media we need to
  decouple GstTensor from GstTensorMeta.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6000>
2024-11-08 14:58:49 +00:00
Olivier Crête
03fd6fadbc analytics: Move tensor meta to the analytics library
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6000>
2024-11-08 14:58:49 +00:00
Daniel Morin
6db6d44ff5 analytics: add segmentation analytics-meta
- Add a new analytics-meta to store segmentation analysis result.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6026>
2024-10-17 18:13:03 +00:00
Daniel Morin
98653aa43a analytics: Allow specific analytics-meta (Mtd) to handle their clear
- Add mtd_meta_clear to allow specific analytics-meta to handle their clear
  operation specific to their type.

- Clear mtd's attached when analytic-meta is freed. When the buffer where
  analytics-meta is attached is not from a buffer pool
  gst_analytics_relation_meta_clear will not be called unless we explicitly call
  it in _free. This important otherwise _mtd_clear are not called and lead to
  leak if embedded mtd's allocated memory
- Un-ref in transform if it's a copy

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6026>
2024-10-17 18:13:03 +00:00
Oskar Fiedot
327df9766d analytics: Change pointers in getters to const
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7403>
2024-09-02 12:27:37 +00:00
Thibault Saunier
5f50b93f0a meson: Remove duplicated library definitions for gstmse and gstanalytics
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7109>
2024-06-30 15:43:27 +00:00
Daniel Morin
55b12a80d6 analytics: Add validation on classification analytics-meta
- Add valiation on parameters passed to gst_analytics_cls_add_cls_mtd.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7046>
2024-06-18 19:13:10 +00:00
Tim-Philipp Müller
9ec99ac6f2 analyticsmeta: fix g-ir-scanner warnings
Fix

 gstanalyticsmeta.c:134: Warning: GstAnalytics: "@instance"
                         parameter unexpected at this location

warning (caused by the extraneous empty line in the doc chunk)
and align function arguments with documentation and header file
(handle -> instance).

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6585>
2024-04-10 00:46:42 +00:00
Mathieu Duponchelle
f1e2c7918e analytics: whitespace matters for gtk-doc syntax
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6253>
2024-03-04 17:33:00 +00:00
Olivier Crête
caac280466 analytics: Add documentation to hotdoc build
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6253>
2024-03-04 17:33:00 +00:00
Olivier Crête
7a14b48dad analytics: Add missing documentation
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6253>
2024-03-04 17:33:00 +00:00
Olivier Crête
0aecef9b63 analytics: Fix various typos in the documentation
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6253>
2024-03-04 09:59:12 -05:00
Sebastian Dröge
77a09c1781 analytics: Allow getting a related meta with ANY type
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6068>
2024-02-07 14:07:52 +00:00
Olivier Crête
351f823704 analytics: Always return a string from mtd_type_get_name()
It makes it easier to use in printf() style strings without worrying
about getting a NULL.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6035>
2024-02-02 15:49:56 +00:00
Daniel Morin
8122f623f6 analytics: Change indexes type from i32 -> gsize
- gst_analytics_cls_mtd_get_length() return a gsize, this type dicated index
  type for gst_analytics_cls_mtd_get_quark() and
  gst_analytics_cls_mtd_get_level().
- Minor cleanup/improvement on index validation.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6018>
2024-01-31 15:22:43 +00:00
Daniel Morin
42227d7de3 analytics: add api on OD to retrieve bbox confidence
- Add a specific api to retrieve bbox confidence level without having to
  retrieve it's location.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6017>
2024-01-31 14:40:56 +00:00
Olivier Crête
745197d386 analytics: Implement scale meta transform for Object detection meta
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5863>
2024-01-10 18:35:17 -05:00
Olivier Crête
5d3035553c analytics: Replace type quark with opaque struct pointer
This way, we can add function pointers in there for things like
transformations.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5863>
2024-01-10 18:34:53 -05:00