238 Commits

Author SHA1 Message Date
Stéphane Cerveau
b2eeb6d6d3 ci: enable vulkan tests in validate
As CI now supports llvm 18 and mesa 24.1 which
allow to use properly lavapipe in the CI, the vulkan
ci tests have been removed from the validate blacklist.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/9416>
2025-08-27 10:50:56 +00:00
Thibault Saunier
efdc9b1aea validate: scenario: Clear wait_message_action on stop
When executing a stop action, ensure that any pending wait_message_action
is cleared and unreffed to prevent it from being executed after the
scenario has been stopped. This prevents potential crashes or unexpected
behavior when stopping a scenario that has a pending wait action.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/9013>
2025-08-19 17:11:51 +00:00
Thibault Saunier
fe6ad6f370 validate: scenario: Fix NULL context usage in action done callback
Use the local context variable that was retrieved earlier instead of
action->priv->context which is always NULL at this point since we
reset it right before. This ensures we invoke the callback on the
correct context.

And make sure that the wait message still has a reference when setting done
otherwise it is unref before calling the context invoke function

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/9013>
2025-08-19 17:11:51 +00:00
Thibault Saunier
f99bf2abee validate: Add support for overrides field in meta structure
Add support for an 'overrides' field in validatetest meta structures that
allows changing issue severity levels during test execution. This enables
tests to pass when encountering known issues by downgrading their severity.

The overrides field accepts an array of change-severity structures with:
- issue-id: The issue ID to override
- new-severity: New severity level (critical, warning, issue, ignore)

Currently only change-severity overrides are supported. The feature follows
the same pattern as expected-issues and is only available in .validatetest
files, not .scenario files.

Includes comprehensive documentation and a test case demonstrating the
functionality.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/9013>
2025-08-19 17:11:50 +00:00
Thibault Saunier
48a2be6d6a validate: http-actions: Replace GUri with GstURI for GLib 2.64 compatibility
GUri was added in GLib 2.66, but GStreamer claims to build with 2.64.
Replace GUri usage with equivalent GstURI functions to maintain
backward compatibility.

Fixes #4607

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/9572>
2025-08-19 06:49:41 +00:00
Doug Nazar
49a81965fc validate: Ignore error msgs when executing next action
When processing an expected error msg, the test for if the next action
was a message would trigger and we'd never execute the next action
causing a timeout.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/9118>
2025-06-20 00:27:57 +00:00
Doug Nazar
1fb195e08e validate: Fix memory leaks
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/9118>
2025-06-20 00:27:57 +00:00
Doug Nazar
198498d3d2 validate: Pass correct argument size to va_arg function
sizeof(int) != sizeof (GType) and can cause an invalid access.
Also one function call was missing the terminator altogether.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/9118>
2025-06-20 00:27:57 +00:00
Doug Nazar
0f32647aae validate: Fix a memory leak
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/9118>
2025-06-20 00:27:57 +00:00
Doug Nazar
83be566ca7 validate: Reset mutex to NUL to allow re-init
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/9118>
2025-06-20 00:27:57 +00:00
Doug Nazar
76fc1d0acf validate: tests: use fixtures to init/deinit and fix leaks
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/9118>
2025-06-20 00:27:57 +00:00
Doug Nazar
79321deb27 validate: cleanup several items on shutdown
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/9118>
2025-06-20 00:27:57 +00:00
Doug Nazar
b0885d2fc3 validate: Fix various memory leaks
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/9118>
2025-06-20 00:27:57 +00:00
Doug Nazar
2f5ecdc911 validate: ensure we shutdown execute_actions source
Actions are no longer always called directly from the main loop
so we can't depend on G_SOURCE_REMOVE to remove it. It's also
possible while returning up the call chain after running stop
to try to re-add it.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/9118>
2025-06-20 00:27:57 +00:00
Víctor Manuel Jáquez Leal
82a71a7739 gstreamer-vaapi: remove subproject
It's almost superseded by va plugin in gst-plugins-bad.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/9200>
2025-06-10 17:32:46 +00:00
Thibault Saunier
c0d8d1b942 validate: launcher: Downgrade non-critical log messages from ERROR to INFO
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/9165>
2025-06-03 10:31:39 +02:00
Thibault Saunier
07bac2b40a validate: launcher: Remove log files for passing tests by default
Adding an option to keep them no matter what.
Log files are often pretty large and keeping them around can be annoying,
usually people won't look at logs files for passing tests, and we do not
even print them out.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/9127>
2025-05-31 07:48:46 +00:00
Thibault Saunier
b51dadf6f1 validate: launcher: Allow using debug log colors when those are being redirected
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/9127>
2025-05-31 07:48:46 +00:00
Thibault Saunier
ca36315dab validate: Do not check that timestamp are in input range for encoders
As some encoder offset the segment range to handle bframes, that check doesn't make sense.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/9127>
2025-05-31 07:48:46 +00:00
Thibault Saunier
313435f6cb general: Stop checking G_HAVE_GNUC_VARARGS now that we depend on c99
Cleaning up a bit the code now that we can rely on C99 which specifies
varargs for macros.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8990>
2025-05-30 15:30:36 +00:00
Doug Nazar
feb6e1602a validate: Escape '%' in Valgrind log filenames
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/9042>
2025-05-28 16:01:44 +00:00
Doug Nazar
822bef51f0 validate: Fix marking actions done
Don't take extra ref during calling done() from 'stream-selection'
Mark as done actions that are completed immediately

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/9042>
2025-05-28 16:01:44 +00:00
Doug Nazar
5c4bf60c94 validate: Free log files
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/9042>
2025-05-28 16:01:44 +00:00
Doug Nazar
09cf44aa13 validate: Various minor cleanups for lost memory or use after free
Includes several missing unref(), clear() or free() calls.
Reset current_seek since we just cleared all the seeks.
Reset all_configs to NULL to prevent double clearing.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/9042>
2025-05-28 16:01:44 +00:00
Thibault Saunier
d0b2e6cb68 validate: scenario: Fix race condition when ignoring EOS
The part of the code that is commented with:

```
  /* gst_validate_action_set_done() does not finish the action
    * immediately. Instead, it posts a task to the main thread to do most
    * of the work in _action_set_done().
    *
    * While the EOS handling lock guarantees that if an action had to call
    * gst_validate_action_set_done() it has done so, it does not guarantee
    * that _action_set_done() has been called.
    *
    * Is it possible that this handler is run before _action_set_done(), so
    * we check at this point for actions that have a pending_set_done and
    * call it before continuing. */
```

was not being executed in the case where the scenario was 'ignoring EOS'
while it was also required.

Also fix potential use after free in that specific code path.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/9014>
2025-05-27 06:59:36 +00:00
Thibault Saunier
6fc32cbe0e validate: Do not list test files that are not autogenerated in .testlist
It was useless, adding the .validatetest to the git repo should be enough

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/9035>
2025-05-20 21:43:14 +00:00
Doug Nazar
08143e9967 validate: baseclasses: Reset Test timeouts between iterations
Several options (valgrind, gdb, rr) increase the timeout each time
the tests start. Eventually reaching inf and causing a conversion
to integer to throw an exception.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/9019>
2025-05-19 17:09:43 +00:00
Alexander Slobodeniuk
a03c4de48f elements: use set_static_metadata when it's allowed
Those strings are nice but CPU doesn't want to copy them

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8905>
2025-04-26 19:30:15 +02:00
Jan Tojnar
13c0f44dd5 validate: Add gst-video to Requires in pkg-config
`gst/validate/validate.h` includes `gst/validate/gst-validate-media-info.h`,
which in turn includes `gst/pbutils/pbutils.h` but `gstreamer-pbutils-1.0`
was only listed in `Requires.private` field of `gstreamer-validate-1.0.pc`.

This would cause projects linking against `gstreamer-validate-1.0.pc` to fail to find
the headers when using alternative interpretation of pkg-config specification
that only considers private dependencies for include path during static builds,
such as the case e.g. on Nix.

https://gitlab.freedesktop.org/pkg-config/pkg-config/-/issues/28

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8661>
2025-03-26 23:40:04 +00:00
Thibault Saunier
e845b596a2 validate: baseclasses: Reset Test.extra_logfiles when copying to start an iteration
Otherwise we get a big mixup with all iteration pointing to all the extra_logfiles

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8640>
2025-03-14 12:50:34 +00:00
Nicolas Dufresne
356c1ff8e3 validate: Do not use G_URI_FLAGS_NONE symbol from 2.66
The GStreamer project currently requires GLib 2.64,
so just avoid this new API for now.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8577>
2025-03-01 00:03:55 +00:00
Jochen Henneberg
30a00b6a97 validate: Don't print to stdout on bail out
Otherwise, if the output is multiline like from expected/actual file
comparison mismatch, meson will parse the lines for TAP formatted
strings and report an error. In that case the tests is accidently
SKIPPED instead of FAIL.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8376>
2025-02-24 19:00:16 +00:00
Jochen Henneberg
8a5ca84612 validate: Get proper error message on diff error instead of assert
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8376>
2025-02-24 19:00:16 +00:00
Jochen Henneberg
61fc65475e validate: 'bat' is optional
If 'bat' is not installed G_IS_SUBPROCESS (subprocess) fails and the
error message is misleading.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8376>
2025-02-24 19:00:16 +00:00
Thibault Saunier
294f1165fa validate: Implement a 'http-request' action type
Which is useable with our own HTTP server

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8128>
2025-02-24 16:57:23 +00:00
Thibault Saunier
7e1fd3b069 validate: scenario: Reset the pipeline on expected ERROR messages while executing actions
Otherwise the scenario gets into an inconsistent state and users won't
be able to properly recover.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8128>
2025-02-24 16:57:23 +00:00
Thibault Saunier
6ddaa9f56a validate: scenario: Better log expected Error messages on the bus
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8128>
2025-02-24 16:57:23 +00:00
Thibault Saunier
78cf4706d9 validate: launcher: add HTTP server control endpoints
Add administrative REST API endpoints to RangeHTTPServer allowing tests to:

- Configure HTTP return codes for specific paths:
  * PUT /admin/status-rules to set rules
  * Support time-based expiry with "during" parameter
  * Support count-based expiry with "repeat" parameter
  * DELETE /admin/status-rules/<path> to remove rules

- Track failure statistics:
  * PUT /admin/failure-counts/start to begin monitoring
  * GET /admin/failure-counts/<path> to get current count

Useful for testing HTTP retry mechanisms, error handling and failure
recovery behaviors.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8128>
2025-02-24 16:57:22 +00:00
Thibault Saunier
0259d46fd3 validate: scenario: Add an action type to start the http scenario
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8128>
2025-02-24 16:57:22 +00:00
Alicia Boya García
482c029c53 validate: miscellaneous Python cleanups
I spent too much time trying to navigate validate Python code to figure
out where a test was coming from. Hoping that it's slightly easier for
the next person, this patch:

* Adds type annotations to setup_tests(), for the sake of code
  navigation.
* Adds comments matching each test generator with the patterns of test
  names it produces.
* Removes an if statement in `register_default_scenarios()` where both
  branches have the same exact code with the same exact very long list.
* Removes NamedDic [sic] and replaces it with SimpleNamespace from the
  standard library (3.3+) which has the same purpose and API.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8090>
2025-02-22 10:30:35 +00:00
Mathieu Duponchelle
3de86b2b97 docs: port plugins to explicit sources
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8273>
2025-01-13 19:17:13 +01:00
Edward Hervey
f20ecfa920 validate: Add more warning flags
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8229>
2025-01-13 12:51:46 +00:00
Edward Hervey
5c2c1bce6c validate: Fix old style definition
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8229>
2025-01-13 12:51:46 +00:00
Thibault Saunier
2c9a642b66 meson: Give the same name for api_version in all modules
There were 2 version of it, apiversion and api_version, I chose the one
with most occurencies: `api_version`

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8178>
2024-12-20 20:19:28 +00:00
Alicia Boya García
fbad7b593d validate-scenario: Fix busy waiting, missing lock
execute_next_action_full() logs that it removes the source when an
action returns ASYNC, but the code for that was incomplete, as it was
setting source_id to zero but not actually removing the source.

This lead to execute_next_action_full() being run continuously, only
alliviated by the default 10ms interval in the GSource from
`scenario->priv->action_execution_interval`.

This patch fixes that. As a drive-by fix it also adds locking to one
remaining unlocked usage of `priv->execute_actions_source_id`.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8014>
2024-12-02 14:36:59 +00:00
Matthew Waters
c7ee7c65ae validate/gapplication: silence a maybe-unitialized warning
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7875>
2024-11-18 12:10:58 +11:00
Thibault Saunier
932f0a6175 validate: flow: Fix logging upstream events
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7700>
2024-11-07 20:52:46 +00:00
Thibault Saunier
56000b91b7 validate: Add a way to force monitoring all pipelines in a .validatetest file
See documentation for more details

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7700>
2024-11-07 20:52:45 +00:00
Thibault Saunier
302797a965 validate: Add a deep-property-path parameter to the wait signal
Allowing wait actions to wait on any property of any element in the pipeline,
even for elements that might be added later in the pipeline. This also
works for pads which can be pretty useful

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7700>
2024-11-07 20:52:45 +00:00
Thibault Saunier
2c88bbf07f validate: Allow overriding features rank early in testfiles meta
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7700>
2024-11-07 20:52:45 +00:00