218 Commits

Author SHA1 Message Date
Doug Nazar
feb6e1602a validate: Escape '%' in Valgrind log filenames
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/9042>
2025-05-28 16:01:44 +00:00
Doug Nazar
822bef51f0 validate: Fix marking actions done
Don't take extra ref during calling done() from 'stream-selection'
Mark as done actions that are completed immediately

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/9042>
2025-05-28 16:01:44 +00:00
Doug Nazar
5c4bf60c94 validate: Free log files
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/9042>
2025-05-28 16:01:44 +00:00
Doug Nazar
09cf44aa13 validate: Various minor cleanups for lost memory or use after free
Includes several missing unref(), clear() or free() calls.
Reset current_seek since we just cleared all the seeks.
Reset all_configs to NULL to prevent double clearing.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/9042>
2025-05-28 16:01:44 +00:00
Thibault Saunier
d0b2e6cb68 validate: scenario: Fix race condition when ignoring EOS
The part of the code that is commented with:

```
  /* gst_validate_action_set_done() does not finish the action
    * immediately. Instead, it posts a task to the main thread to do most
    * of the work in _action_set_done().
    *
    * While the EOS handling lock guarantees that if an action had to call
    * gst_validate_action_set_done() it has done so, it does not guarantee
    * that _action_set_done() has been called.
    *
    * Is it possible that this handler is run before _action_set_done(), so
    * we check at this point for actions that have a pending_set_done and
    * call it before continuing. */
```

was not being executed in the case where the scenario was 'ignoring EOS'
while it was also required.

Also fix potential use after free in that specific code path.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/9014>
2025-05-27 06:59:36 +00:00
Thibault Saunier
6fc32cbe0e validate: Do not list test files that are not autogenerated in .testlist
It was useless, adding the .validatetest to the git repo should be enough

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/9035>
2025-05-20 21:43:14 +00:00
Doug Nazar
08143e9967 validate: baseclasses: Reset Test timeouts between iterations
Several options (valgrind, gdb, rr) increase the timeout each time
the tests start. Eventually reaching inf and causing a conversion
to integer to throw an exception.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/9019>
2025-05-19 17:09:43 +00:00
Alexander Slobodeniuk
a03c4de48f elements: use set_static_metadata when it's allowed
Those strings are nice but CPU doesn't want to copy them

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8905>
2025-04-26 19:30:15 +02:00
Jan Tojnar
13c0f44dd5 validate: Add gst-video to Requires in pkg-config
`gst/validate/validate.h` includes `gst/validate/gst-validate-media-info.h`,
which in turn includes `gst/pbutils/pbutils.h` but `gstreamer-pbutils-1.0`
was only listed in `Requires.private` field of `gstreamer-validate-1.0.pc`.

This would cause projects linking against `gstreamer-validate-1.0.pc` to fail to find
the headers when using alternative interpretation of pkg-config specification
that only considers private dependencies for include path during static builds,
such as the case e.g. on Nix.

https://gitlab.freedesktop.org/pkg-config/pkg-config/-/issues/28

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8661>
2025-03-26 23:40:04 +00:00
Thibault Saunier
e845b596a2 validate: baseclasses: Reset Test.extra_logfiles when copying to start an iteration
Otherwise we get a big mixup with all iteration pointing to all the extra_logfiles

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8640>
2025-03-14 12:50:34 +00:00
Nicolas Dufresne
356c1ff8e3 validate: Do not use G_URI_FLAGS_NONE symbol from 2.66
The GStreamer project currently requires GLib 2.64,
so just avoid this new API for now.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8577>
2025-03-01 00:03:55 +00:00
Jochen Henneberg
30a00b6a97 validate: Don't print to stdout on bail out
Otherwise, if the output is multiline like from expected/actual file
comparison mismatch, meson will parse the lines for TAP formatted
strings and report an error. In that case the tests is accidently
SKIPPED instead of FAIL.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8376>
2025-02-24 19:00:16 +00:00
Jochen Henneberg
8a5ca84612 validate: Get proper error message on diff error instead of assert
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8376>
2025-02-24 19:00:16 +00:00
Jochen Henneberg
61fc65475e validate: 'bat' is optional
If 'bat' is not installed G_IS_SUBPROCESS (subprocess) fails and the
error message is misleading.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8376>
2025-02-24 19:00:16 +00:00
Thibault Saunier
294f1165fa validate: Implement a 'http-request' action type
Which is useable with our own HTTP server

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8128>
2025-02-24 16:57:23 +00:00
Thibault Saunier
7e1fd3b069 validate: scenario: Reset the pipeline on expected ERROR messages while executing actions
Otherwise the scenario gets into an inconsistent state and users won't
be able to properly recover.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8128>
2025-02-24 16:57:23 +00:00
Thibault Saunier
6ddaa9f56a validate: scenario: Better log expected Error messages on the bus
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8128>
2025-02-24 16:57:23 +00:00
Thibault Saunier
78cf4706d9 validate: launcher: add HTTP server control endpoints
Add administrative REST API endpoints to RangeHTTPServer allowing tests to:

- Configure HTTP return codes for specific paths:
  * PUT /admin/status-rules to set rules
  * Support time-based expiry with "during" parameter
  * Support count-based expiry with "repeat" parameter
  * DELETE /admin/status-rules/<path> to remove rules

- Track failure statistics:
  * PUT /admin/failure-counts/start to begin monitoring
  * GET /admin/failure-counts/<path> to get current count

Useful for testing HTTP retry mechanisms, error handling and failure
recovery behaviors.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8128>
2025-02-24 16:57:22 +00:00
Thibault Saunier
0259d46fd3 validate: scenario: Add an action type to start the http scenario
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8128>
2025-02-24 16:57:22 +00:00
Alicia Boya García
482c029c53 validate: miscellaneous Python cleanups
I spent too much time trying to navigate validate Python code to figure
out where a test was coming from. Hoping that it's slightly easier for
the next person, this patch:

* Adds type annotations to setup_tests(), for the sake of code
  navigation.
* Adds comments matching each test generator with the patterns of test
  names it produces.
* Removes an if statement in `register_default_scenarios()` where both
  branches have the same exact code with the same exact very long list.
* Removes NamedDic [sic] and replaces it with SimpleNamespace from the
  standard library (3.3+) which has the same purpose and API.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8090>
2025-02-22 10:30:35 +00:00
Mathieu Duponchelle
3de86b2b97 docs: port plugins to explicit sources
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8273>
2025-01-13 19:17:13 +01:00
Edward Hervey
f20ecfa920 validate: Add more warning flags
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8229>
2025-01-13 12:51:46 +00:00
Edward Hervey
5c2c1bce6c validate: Fix old style definition
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8229>
2025-01-13 12:51:46 +00:00
Thibault Saunier
2c9a642b66 meson: Give the same name for api_version in all modules
There were 2 version of it, apiversion and api_version, I chose the one
with most occurencies: `api_version`

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8178>
2024-12-20 20:19:28 +00:00
Alicia Boya García
fbad7b593d validate-scenario: Fix busy waiting, missing lock
execute_next_action_full() logs that it removes the source when an
action returns ASYNC, but the code for that was incomplete, as it was
setting source_id to zero but not actually removing the source.

This lead to execute_next_action_full() being run continuously, only
alliviated by the default 10ms interval in the GSource from
`scenario->priv->action_execution_interval`.

This patch fixes that. As a drive-by fix it also adds locking to one
remaining unlocked usage of `priv->execute_actions_source_id`.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8014>
2024-12-02 14:36:59 +00:00
Matthew Waters
c7ee7c65ae validate/gapplication: silence a maybe-unitialized warning
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7875>
2024-11-18 12:10:58 +11:00
Thibault Saunier
932f0a6175 validate: flow: Fix logging upstream events
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7700>
2024-11-07 20:52:46 +00:00
Thibault Saunier
56000b91b7 validate: Add a way to force monitoring all pipelines in a .validatetest file
See documentation for more details

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7700>
2024-11-07 20:52:45 +00:00
Thibault Saunier
302797a965 validate: Add a deep-property-path parameter to the wait signal
Allowing wait actions to wait on any property of any element in the pipeline,
even for elements that might be added later in the pipeline. This also
works for pads which can be pretty useful

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7700>
2024-11-07 20:52:45 +00:00
Thibault Saunier
2c88bbf07f validate: Allow overriding features rank early in testfiles meta
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7700>
2024-11-07 20:52:45 +00:00
Thibault Saunier
f5b0f91d33 validate: launcher: Remove log files for passing tests by default
Adding an option to keep them no matter what.
Log files are often pretty large and keeping them around can be annoying,
usually people won't look at logs files for passing tests, and we do not
even print them out.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7700>
2024-11-07 20:52:45 +00:00
Thibault Saunier
0cdd22f176 validate: Remove some regex related python 3.12 warnings
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7700>
2024-11-07 20:52:45 +00:00
Thibault Saunier
dfd5357214 validate: scenario: Add a way to make the select-streams run several times
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7700>
2024-11-07 20:52:45 +00:00
Thibault Saunier
19060b2ce0 validate: Add a way to retrieve HTTP server port in .validatetest files
By setting a `$(http_server_port)s` variable in a dedicated config file
and making sure that file can always be imported in `.validatetest`
files.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7700>
2024-11-07 20:52:45 +00:00
Edward Hervey
36b32de831 validate: Fix leaks in ssim components
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7782>
2024-10-31 01:39:19 +00:00
Tim-Philipp Müller
b3245eb7bd validate: skip curlhttpsrc test_get_range when running in valgrind
Seems to reliably fail (timeout) when running in valgrind
on the post-F40 CI.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7753>
2024-10-26 02:10:37 +01:00
Tim-Philipp Müller
bf5b0283a2 validate: add srtp test_roc to valgrind exclude list
Consistently fails on CI after F40 upgrade.

See #3939

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7753>
2024-10-26 02:09:44 +01:00
Tim-Philipp Müller
f64cf9998f validate: launcher: add --fair-sched=try to valgrind args
Enables fairer scheduling of threads in valgrind, possibly
at cost of performance though.

Hopefully helps with valgrind core/base jobs deadlocking or
timing out when runners are not under load.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7753>
2024-10-26 00:23:01 +01:00
Edward Hervey
5ac43063b6 tests: Blacklist more netsim test
They are know to be racy/failing

See https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/792

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7736>
2024-10-25 10:44:24 +00:00
Corentin Damman
339e6e0069 validate: use real_main instead of main in run_test_from_file
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7704>
2024-10-23 13:43:03 +00:00
Thibault Saunier
14a9138a88 validate: flow: Display diffs properly
When called after writing the report we can't see them

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5035>
2024-10-19 10:30:05 +00:00
Sebastian Dröge
6233eb0ff3 common: Stop using GQuark-based GstStructure field name API
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7432>
2024-09-26 19:21:29 +03:00
Thibault Saunier
9f898f839c validate: pad-monitor: Fix remaining pad functions data handling
That case was missed in a5717530899836cdc32e17edc8f8e47d2c14420d

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7477>
2024-09-10 15:50:52 +00:00
Thibault Saunier
0d24821167 validate: launcher: Add support for lldb
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7452>
2024-09-10 11:23:02 +00:00
Thibault Saunier
a571753089 validate: pad-monitor: Fix pad function data properly
Until now we were overriding pad functions forgetting about the function
data (that are set using the _full variant of the functions setters), meaning
that the data was lost and any user of that feature would get empty data when
the wrapped function were called.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7466>
2024-09-08 12:02:25 +00:00
Thibault Saunier
9f410a8714 validate: scenario: Handle the fact that structs field names don't have a static lifetime anymore
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7453>
2024-09-05 18:58:24 +00:00
Sebastian Dröge
ec11c337c3 validate: Properly use errors = 'replace' for handling invalid utf8 characters
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7453>
2024-09-05 18:58:24 +00:00
Thibault Saunier
87c69e5174 ci: Fail tests if we forget to checkout expectation files
And add missing expectation files

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7400>
2024-08-21 17:53:38 +00:00
Thibault Saunier
8fdd59f9d5 validate: flow: Allow logging upstream events
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7382>
2024-08-21 13:55:15 +00:00
Sebastian Dröge
417c5e19b7 validate: Copy action structure before retrieving strings from it
The returned strings are only valid for as long as the structure is valid.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7333>
2024-08-09 10:26:30 +00:00