summaryrefslogtreecommitdiffstats
path: root/util
Commit message (Collapse)AuthorAgeFilesLines
* qt-testrunner: exit(1) instead of exit(3) when asserts triggerDimitrios Apostolou5 days1-1/+7
| | | | | | | | | | | | Asserts triggering signify programming errors in the testrunner. For the same reason, change also a pre-existing AssertionError exception raised with bad XML input file to a dedicated exception. Pick-to: 6.11 6.10 6.8 Change-Id: I5e5cc746df60075e02964efd97d1a37dc401925f Reviewed-by: Axel Spoerl <axel.spoerl@qt.io> Reviewed-by: Frederic Lefebvre <frederic.lefebvre@qt.io>
* qt-testrunner: check validity of the XML file on test function re-runsDimitrios Apostolou5 days1-2/+13
| | | | | | | | | | | | | | | | | | | | | Tests on certain platforms run with specific wrappers, that might have trouble reporting back the proper exit code. For example, on Android each test is transfered to the device/emulator and is executed not as a separate process, but as an activity. The equivalent of "exit code" is caught and returned by a special wrapper script for the platform. It happens sometimes that these wrapper scripts fail to report back correctly, and report zero (0) despite failed tests. For that reason we now parse the test XML log on individual test re-runs too, and reporting inconsistencies as CRASH, like we do with the main test execution. Task-number: QTQAINFRA-7349 Task-number: QTQAINFRA-7378 Pick-to: 6.11 6.10 6.8 Change-Id: I27525f22331d44141be8825786a6f71e89543e92 Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
* Test qt-testrunner as the first test that CTest runsDimitrios Apostolou5 days5-844/+3
| | | | | | | | | | | | | Only for inside the Coin CI, as it is Coin that uses qt-testrunner by exporting TESTRUNNER=qt-testrunner. Also rename the test executable from tst_testrunner to tst_qt_testrunner to avoid confusion with other testrunners, or generic $TESTRUNNER testing. Pick-to: 6.11 6.10 6.8 Change-Id: I607f8c2affec2ca5dd38b4a333abb3a324d2078c Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
* qt_testrunner: fix execution of qt_mock_test.py under testing on windowsDimitrios Apostolou5 days2-0/+7
| | | | | | | | | | | | | | | | | | tst_qt_testrunner.py invokes qt_testrunner.py which invokes qt_mock_test.py. On Windows a python script is not directly executable. It needs to be executed as "python.exe script.py". To achieve that when testing, the tests set the env var QT_TESTRUNNER_TESTING. When qt-testrunner detects that, it prepends the command line with sys.executable which is the currently used python interpreter. Doing so also ensures that all sub-scripts are executed with the same Python version. Pick-to: 6.11 6.10 6.8 Change-Id: I4786e4015ded328077fd9cc7b443f4b4d9d69de7 Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io> Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
* tst_qt_testrunner: make it runnable on windowsDimitrios Apostolou5 days1-7/+10
| | | | | | | | | | | | | | | | | | | By making sure it invokes other python executables using the same python interpreter. Also on the tests that write to an XML file, close the file handle that mkstemp() generates, otherwise Windows refuse to open and write to the filename. Side change: + For some tests, include all the environment instead of selecting specific env vars, in order to reduce verbosity. Pick-to: 6.11 6.10 6.8 Change-Id: If4ba68b5f9bf64824c888b99e9525d4bdcf42037 Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io> Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
* tst_qt_testrunner:Avoid parameter root_dir in glob()Dimitrios Apostolou5 days1-27/+20
| | | | | | | | because it's only available since Python 3.10. Pick-to: 6.11 6.10 6.8 Change-Id: I81170b9cbbd7819fbfe3a13c8caa33e21dcae9b3 Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
* tst_testrunner.py: small fix to make it run with python<3.9Dimitrios Apostolou5 days1-1/+1
| | | | | | Pick-to: 6.11 6.10 6.8 Change-Id: I382e8c8b766c0c36e484ecb16e74a0010ea4fe32 Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
* Test qt-testrunner's functionality of repeating a non-QTest testDimitrios Apostolou5 days1-1/+25
| | | | | | | | | | In that case, it re-runs the whole executable once, since the test does not output a QTest XML log and does not accept selective running through QTest command-line arguments. Pick-to: 6.11 6.10 6.8 Change-Id: I35b255aff10a0f2a79851889c8daeca277c605ed Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
* tst_testrunner: split up a long line to stop gerrit from complainingDimitrios Apostolou5 days1-4/+10
| | | | | | | | | | | The line had been copied verbatim from the CI logs, but gerrit refuses to do any syntax highlighting in the file. So break it down. Also remove some old commented-out code. Pick-to: 6.11 6.10 6.8 Change-Id: Ibde01161edb099eb8b1c9d7d6357a7a8b36e6819 Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
* qt-testrunner: if a failed test re-run crashes, then exit(3) immediatelyDimitrios Apostolou5 days4-14/+46
| | | | | | | | | Instead of continuing the re-runs of the failed function multiple times, quit immediately with exit code 3 which signifies crash. Pick-to: 6.11 6.10 6.8 Change-Id: I9997d6fefe946f719019dad85852609528d9a30e Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
* qt-testrunner: comment updatesDimitrios Apostolou5 days2-4/+21
| | | | | | Pick-to: 6.11 6.10 6.8 Change-Id: Ia72fa89b47d99a90d2c669f5b889ee8fba461cf2 Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
* qt-testrunner: add testcase that verifies a crashed test re-runs fullyDimitrios Apostolou5 days2-3/+46
| | | | | | | | | | | | | Even if it has written a proper, non-corrupt XML file, with no trace of crash. Fixed also a small typo in a comment because it referred to the wrong environment variable. (changed QT_MOCK_FAIL_LIST to QT_MOCK_RUN_LIST). Task-number: QTQAINFRA-5226 Pick-to: 6.11 6.10 6.8 Change-Id: I2e0c443edcaeba053927f4e3fbb325a13f34d238 Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
* qt-testrunner: consider exit codes >=128 as CRASHDimitrios Apostolou5 days2-13/+31
| | | | | | | | | | | | | | | | | | | Previously it was only < 0, which was valid because Python set those on POSIX platforms to represent signal number. By checking >=128 we also accommodate for crashes on windows and other unpredictable situations (since QTest is limiting itself to [0,127]). Also change the exit code of the mock test used for qt_testrunner testing. 130 is what is shown by bash when a process is interrupted by Ctrl-C (SIGINT), while 131 is what is shown for SIGQUIT and does not happen that easily. Completely insignificant change for the python programs like qt-testrunner and its tests, since subprocess.run() returncode is negative in case of a signal. Only visible if qt_mock_test is invoked in the shell for debugging purposes. Pick-to: 6.11 6.10 6.8 Change-Id: I4759bc0e74ca2fb1d1594c71e541feb5363bb7ea Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
* qt-testrunner: skip some of the tests on non-POSIX platformsDimitrios Apostolou5 days1-0/+4
| | | | | | Pick-to: 6.11 6.10 6.8 Change-Id: Ib48d1a2e98423218097405fe679c6d68d0e208c7 Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
* qt-testrunner: fix two pre-existing testsDimitrios Apostolou5 days1-0/+2
| | | | | | | | | | | Previously the tests were not testing what they were supposed to. Instead of an empty file, qt-testrunner was finding no file. By using the newly introduced hidden knob, qt-testrunner will read the XML file that the test provides. Pick-to: 6.11 6.10 6.8 Change-Id: I8bd356f4f770a478aabaaee8eb449162abed18f1 Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
* qt-testrunner: if a test logs a qFatal message, then treat it as crashedDimitrios Apostolou5 days3-22/+153
| | | | | | | | | | | | | | | | | | QTest can, depending on the platform, clean-up properly after a qFatal. In that case the test <Incident> is marked simply as FAIL, no further tests are run, and the XML logfile is closed properly. The only way to figure out that something went bad and the testrun was aborted, is the qFatal message. This is needed primarily for qt-testrunner to *not* consider re-running individual failed test functions/datatags, to avoid missing the ones that did not start at all. Fixes: QTQAINFRA-7349 Fixes: QTQAINFRA-7378 Pick-to: 6.11 6.10 6.8 Change-Id: I77276f37e0e6668048c78563560af0e15101be0e Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
* qsimd/x86: Remove detection of RDRND and RDSEEDThiago Macieira2025-11-181-2/+2
| | | | | | | | | | | | | | | | The previous commit removed their use from QRandomGenerator, which was the only user. This removes their detection too, which means QtCore will not refuse to load when compiled with an -march= flag that specifies the feature should be present, but is missing at runtime. See https://lore.kernel.org/all/9a27f2e6-4f62-45a6-a527-c09983b8dce4@cachyos.org/ Task-number: QTBUG-69423 Task-number: QTBUG-129193 Pick-to: 6.10 6.8 6.5 Change-Id: Ib67da873cc42acfedaeefffd56db0414bb7ea6b0 Reviewed-by: Jason A. Donenfeld <Jason@zx2c4.com> Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
* Include new languages for CLDR v48Edward Welbourne2025-11-182-0/+45
| | | | | | | | | | | | | | | | Include two new languages, Ladin and Shan, and document the various languages and scripts that show up in the cldr2qlocalexml.py output, that I have checked and seen to contain inadequate information. This may make it easier for future updaters to spot new unknown codes when they show up. These are not picked back to past versions because they're naturally documented as [since 6.11] and picking would involve each past branch getting a minor version as its since. Fixes: QTBUG-141949 Change-Id: If0cb3e3b33cd3ce636fd29e904a9ddd617940314 Reviewed-by: Mate Barany <mate.barany@qt.io>
* wasm: enable gui tests for the test runnerMorten Sørvig2025-11-186-12/+66
| | | | | | | | | | Create a container element where the test can place windows, pass it to the test instance. Change-Id: Ie12c96ae8382047897d7e76a5a5a2beff06d0899 Reviewed-by: Even Oscar Andersen <even.oscar.andersen@qt.io> Reviewed-by: Lorn Potter <lorn.potter@qt.io> Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
* wasm: run the current test by defaultMorten Sørvig2025-11-183-4/+16
| | | | | | | | | Make the test runner run the current test by default in non-batched mode, without requiring a testname parameter. Change-Id: Ia74f5a3db4a5c4a8d9f6a41073520653781d0f17 Reviewed-by: Lorn Potter <lorn.potter@qt.io> Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
* Update CLDR to v48Edward Welbourne2025-11-171-2/+12
| | | | | | | | | | | | | This is just the plain upgrade without adding new languages. It required two changes to expected data in tests. [ChangeLog][QtCore][Third-Party Code] QLocale now uses v48 of the Unicode Consortium's Common Locale Data Repository (CLDR). Pick-to: 6.10 6.8 6.5 Task-number: QTBUG-141949 Change-Id: I8d3a299c602e6cd5ea76c6bc479028aeda8b8e50 Reviewed-by: Mate Barany <mate.barany@qt.io>
* CMake: Add initial Cyclone DX v1.6 SBOM generation supportAlexandru Croitor2025-11-057-0/+1636
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This change adds initial Cyclone DX v1.6 SBOM generation support for CMake-based Qt projects. Cyclone DX generation is enabled by default for all Qt repos, as long as the required Python dependencies are found. If needed, it can be explicitly enabled by configuring the qt repo project with: -sbom -sbom-cyclonedx-v1_6 or a generic CMake project with -DQT_GENERATE_SBOM=ON -DQT_SBOM_GENERATE_CYDX_V1_6=ON. Disabling can be done with -no-sbom-cyclonedx-v1_6 or -DQT_SBOM_GENERATE_CYDX_V1_6=OFF respectively. Note that if the required python dependencies are not found, the generation will be silently skipped. To ensure cmake configuration fails if SBOM dependencies are missing, pass: -sbom-cyclonedx-v1_6-required or -DQT_SBOM_REQUIRE_GENERATE_CYDX_V1_6=ON To ensure the generated document is valid according to the Cyclone DX schema, pass: -sbom-cyclonedx-v1_6-verify-required or -DQT_SBOM_REQUIRE_VERIFY_CYDX_V1_6=ON Cyclone DX generation requires a Python 3.9 interpreter in PATH and the `cyclonedx-python-lib` and 'tomli' Python packages to be installed. The packages can be installed via pip: pip install 'cyclonedx-python-lib[json-validation]' tomli ideally in a Python virtual environment. If using Python 3.11, tomli is not required. If using Python 3.9 or 3.10, and pip is available, the code will try to use the toml library bundled with pip, in which case `tomli` is not required to be installed separately. Generated Cyclone DX documents are installed in the same `sbom` directory under the prefix. For example when building and installing qtbase, the generated Cyclone DX document will be installed to: $qt_prefix/sbom/qtbase-$version.cdx.json The commit title mentions "minimal" Cyclone DX support. Minimal support means that not all features of the equivalent SPDX generation are implemented. List of implemented features: - CMake target information included as Cyclone DX components with information like name, version, supplier, download location, CPE and PURL - attribution information - license information (ids and texts, without OR/AND structure) - dependencies between components in the same document - dependencies between components in different documents List of missing features: - partial license information (pending v1.7 spec support, currently any license expression with a LicenseRef- has its structure lost, so OR and AND gets removed, and the result is just a flat list of license ids and texts) - binary file, source file, and custom file information (unclear if there's a good way to represent these in the spec) - checksums for components, files and sbom documents (also unclear how to represent, because the spec has a 'hashes' key, but it doesn't say what does it apply to) - relationship information (the Cyclone DX spec doesn't support relationships like DEPENDS_ON or CONTAINS, only dependencies) - detailed component types (only application and library are supported currently) - custom added relationship / dependency info - qtwebengine / Chromium enablers - probably other things Implementation details. Cyclone DX doesn't have a tag:value format like SPDX v2.3, so there isn't an easy way to generate Cyclone DX directly in CMake. Instead, an intermediate TOML file is generated in CMake, which is then fed to a custom Python script that generates the final Cyclone DX document in JSON format. TOML is a much easier format to generate in CMake compared to JSON, and is easy to read for humans and to parse in Python. Note that some of the TOML fields are wrapped in literal multi-line strings aka ''' foo ''' rather than non-literal ones, aka """ foo """. That's to avoid the need for escaping backslashes for content like configure arguments in the description field (Windows is notorious for trailing backslashes in paths). The python script is located in util/sbom/cyclonedx/qt_cyclonedx_generator. It is installed into $qt_prefix/libexec so it can be used for other qt repos and projects. Development of the script is done using the `uv` tool and a pyproject.toml project, but `uv` is not required for installation and usage of the script itself. The script parses the TOML file and serializes the Cyclone DX document into the JSON format, while also syntactically validating it. Cyclone DX and SPDX differences. Cyclone DX has the concept of `bom-ref` which is similar to SPDX IDs and SPDX references. We reuse the SPDX ids like SPDXRef-Package-Foo as bom-refs to allow cross-referencing between repo BOM documents, but it also allows easier comparison with the SPDX documents. SPDX mostly operates on the concept of packages and files, whereas Cyclone DX uses components. We map CMake targets to Cyclone DX components. CycloneDX has one notable difference to SPDX. External dependencies whose full BOM is located in a different document, must still be represented as components in the current document, but with an external BOM link, which is why we need to track such components. This is achieved by exporting the spdx id, bom serial number, etc as CMake target properties, to then partially recreate the components in the currently processed document. Note that the QT_GENERATE_SBOM option is now a main toggle for SBOM generation, instead of being specific to SPDX generation. Various other variables that were SPDX v2.3 specific but had generic names have been deprecated, and new ones introduced to make it clearer that they are SPDX specific, and to allow further evolution for SPDX v3 and Cyclone DX 1.7 while being backwards compatible. Adjusted tests to generate SPDX and Cyclone DX SBOMs together, separately, or not at all. And to check for the existence of the installed sbom files. As well as the computed generation options in case dependencies are missing. [ChangeLog][Build System] A new -sbom-cyclonedx-v1_6 configure option can be used to generate and install a CycloneDX v1.6 SBOM (Software Bill of Materials) file for each built Qt repository. Task-number: QTBUG-129598 Change-Id: If20a45083a70b01668f52023053b75f520129381 Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
* QUnicodeTools: don't look up surrogate line-break propertiesMarc Mutz2025-10-281-0/+2
| | | | | | | | | | | | | | We know they're SG, so don't go through the properties trie, hard-code the result. As a defense against changes, add checks to the generator and tst_QUnicodeTools. This is in preparation of porting getLineBreaks() to QStringIterator. Pick-to: 6.10 6.8 6.5 Change-Id: Ib3567398ba56f7ad3ce6fbca81f6b0f40379ee7d Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
* QUnicodeTables: separate Properties::cases from the restMarc Mutz2025-10-271-20/+87
| | | | | | | | | | | | | | | | | | | | | | | | | | | | These entries are quite repetitive, esp. the all-zero entry for uncased characters (but not only: there are also 137 non-zero duplicates), and each one takes 8 bytes of the total 20 bytes of sizeof(Properties). Make a new array with these entries and only store an index into it in Properties. The new array happens to have a size of 448 entries (down from 3372 unique Properties), so 9 bits would suffice for the index, but a sizeof(Properties) == 14 is probably rather pointless, so add a reserved field to prop the struct up to 16. That sounds like the ideal size for rapid indexing and probably improves qGetProp() performance, esp. if case information is not needed. Theoretically, this should save 3372 * 4 - 448 * 8 = 9904 bytes. The TEXT size of libQtCore, however shrinks by a bit more, 10596 bytes, on optimized Linux AMD64 Clang 19 builds. Picking to all active branches, because the Unicode tables are still maintained in all of them. Fixes: QTBUG-139427 Pick-to: 6.10 6.9 6.8 6.5 Change-Id: If4dc47ef06c674ad0263f0623ec408a25b977b3a Reviewed-by: Edward Welbourne <edward.welbourne@qt.io> Reviewed-by: Ahmad Samir <a.samirh78@gmail.com>
* util/{unicode,xkbdatagen}: fix build with `REUSE-IgnoreStart`Ahmad Samir2025-10-262-4/+4
| | | | | | | | | | | | | | | | | Building util/unicode I get: main.cpp:3368:7: error: invalid preprocessing directive #REUSE 3368 | # REUSE-IgnoreStart | ^~~~~ main.cpp:3374:7: error: invalid preprocessing directive #REUSE 3374 | # REUSE-IgnoreEnd | ^~~~~ Amends a53be8b935964836b643f1b40d56fe16a35cf9c4. Pick-to: 6.8 6.10 Change-Id: I21c5c6ecc0a8095d74c3721acf77274a1cc78d60 Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io> Reviewed-by: Marc Mutz <marc.mutz@qt.io>
* Adapt to reuse version 6Lucie Gérard2025-10-132-1/+4
| | | | | | | Task-number: QTBUG-140916 Pick-to: 6.8 6.10 Change-Id: I5f865a54efc505312ba4172398efea6d78f7a9a3 Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
* qt-testrunner: detect test filename for androidtestrunner --aabDimitrios Apostolou2025-10-062-16/+99
| | | | | | | | | | | | | | | | | Tests on android are wrapped by androidtestrunner. Previously the test was passed to it with --apk testname.apk. A new way was recently added: --aab testname.aab. qt-testrunner now supports these two ways to detect the filename. Additionally, it now exits with error if the testname is undetected. This way we'll avoid all tests being recorded as "androidtestrunner" in the future. Fixes: QTQAINFRA-7351 Pick-to: 6.10 6.9 6.8 Change-Id: I571b617b28cdb4ea51d0b61e629d9bc8ec8e1917 Reviewed-by: Edward Welbourne <edward.welbourne@qt.io> Reviewed-by: Assam Boudjelthia <assam.boudjelthia@qt.io>
* util/unicode: use prinft-style syntax with qDebug()Ahmad Samir2025-10-041-12/+11
| | | | | | | | | | | | The QDebug streaming operators would expand to more code. We can also remove the qdebug.h include (with all the other transitive includes that brings) which may make compiling faster. Explicitly include qmap.h, which used to be included transitively. Pick-to: 6.10 6.8 6.5 Change-Id: Ic2039e7b18801876c42e043d56c34e2220f3a46f Reviewed-by: Marc Mutz <marc.mutz@qt.io>
* util/unicode: DRY terminating when unable to open a fileAhmad Samir2025-10-041-3/+9
| | | | | | Pick-to: 6.10 6.8 6.5 Change-Id: I1e85b7bf5124338abe1abbfd80182e2f2ddf2f8d Reviewed-by: Marc Mutz <marc.mutz@qt.io>
* util/unicode: format debug messages as <relative-path>:<lineNo>: <Msg>Ahmad Samir2025-10-041-105/+106
| | | | | | | | | | Like Linux standard tools, e.g. grep and gcc. It's a more familiar pattern, which makes it easier to read, also easier for tools to parse the output (some tools can turn these into a clickable link). Pick-to: 6.10 6.8 6.5 Change-Id: I28e16f1a3ac91e703bba270da36edc27b9457fa8 Reviewed-by: Marc Mutz <marc.mutz@qt.io>
* Change C locale's short date format to use two-digit monthEdward Welbourne2025-09-301-1/+1
| | | | | | | | | | | | | | | | | | It was previously the only locale using short month name in the short date format, where all others used two-digit or unpadded month number. Since this anomalous format was used by no other locale and wasn't even a substring of some locale's long format, but was one of the first in date_format_data[], most other locales' date formats have changed position within this block of character data, causing trivial changes to two columns of the main locale_data[] table. [ChangeLog][QtCore][QLocale] The C locale now, in line with all locales derived from CLDR data, uses numeric month rather than the abbreviated month name. Change-Id: I5451036fa170236d88a46bef93a37c299de53a63 Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
* Remove seconds from C locale short time formatEdward Welbourne2025-09-301-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | This is for consistency with other locales, none of which include seconds in their short formats. Until Qt 5.0.0's update to CLDR 1.4 (commit 3bfb72b90151339d85af3d064839beb40a82f933 in the git-p4 history, 2006-09-15) the other locales did include seconds, but since then they have not. However, the C locale retained its seconds. Since the C locale is used as fall-back for locales for which we have no data, this affects all such locales as well. Since the resulting short time format is, like its prior longer form, a sub-string of the long time format, the change doesn't affect the time_format_data[] or trigger any consequent changes to other rows of the main locale_data[] table. Only the C locale's own row changes, just shortening the substring it reuses. [ChangeLog][QtCore][QLocale] The C locale now, in line with all locales derived from CLDR data, omits seconds from its short time format. Fixes: QTBUG-140643 Change-Id: Ic5f181dc581c70a28afdd3f8f32b78190e818131 Reviewed-by: Thiago Macieira <thiago.macieira@intel.com> Reviewed-by: Ahmad Samir <a.samirh78@gmail.com>
* Mark QUnicodeTables as security-significant (= default)Marc Mutz2025-09-261-0/+1
| | | | | | | | | | | | | | | | | | | | This component does not process external data, so it is not security-critical. Yes, the characters presented to its functions may come from external sources, but that's not different from, say, a QRect being parsed from -geometry. The fact that there is code that parses a -geometry into a QRect doesn't make QRect a data-parser, or security-critical. It's just container for the result, and so is QUnicodeTables: a container for char32_t-indexed properties. This accompanying qt_attribution.json confirms that this component is not security-critical. Task-number: QTBUG-135195 Pick-to: 6.10 6.8 Change-Id: I565bd885220c0282ce7fb801411f12a80052465f Reviewed-by: Ivan Solovev <ivan.solovev@qt.io>
* QUnicodeTables: remove "wasm 64 packing trick"Marc Mutz2025-09-181-6/+0
| | | | | | | | | | | | | | | | | | | | | | | It is expected that this is no longer needed. The upstream issue¹ is closed since 2016-09-05 (nine years ago at this point; even though the work-around was added to Qt still two years after upstream was fixed). Amends e59ba35f1b1954062266164f7b802076dc152c7b (for the generator) and a2efb8d94bb8e860d8616f56c4f40e7c41eb3810 (for the generated file; squashed into 29c0377f07f4942f9957ea87d59c252148dc9e5b for the linear history) ¹ https://github.com/emscripten-core/emscripten/issues/4540 Task-number: QTBUG-66621 Task-number: QTBUG-69421 Task-number: QTBUG-74511 Pick-to: 6.10 6.9 6.8 6.5 Change-Id: I9b08792aa4f5f18240d9425c44fa7a88e63e27da Reviewed-by: Thiago Macieira <thiago.macieira@intel.com> Reviewed-by: Lorn Potter <lorn.potter@qt.io>
* QUnicodeTables: abstract access to Properties::casesMarc Mutz2025-09-181-10/+20
| | | | | | | | | | | | | | | | | | | This is in preparation for storing this information in a separate array to save space by removing the many duplicates in there. Pick to all active branches to have the same internal API going forward, even if we don't pick the storage optimization back as far. Qt 6.5 doesn't have QSpan, yet (not even as private API), but returning a reference to const std::array<.,4> will be an adequate replacement. To enable that without casting, convert `cases` from a C array to a std::array. For some reason, this requires extra parentheses, so add them. Task-number: QTBUG-139427 Pick-to: 6.10 6.9 6.8 6.5 Change-Id: I5331fd6d71a6a447b0445d8235b5eb8e38178e2e Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
* util/unicode: remove properties(char16_t) declaration here, tooMarc Mutz2025-09-161-2/+0
| | | | | | | | | | | | Amends 772b62c91e27d357048a1d2ae1962b5901862e76. The amended commit removed the definition from both the generator and the generated code, but only removed the declaration from the generated code, not the generator. Pick-to: 6.10.0 6.10 Change-Id: I2f41aad9777a8c27f80edb9b7ef7a97e1871ffbb Reviewed-by: Ahmad Samir <a.samirh78@gmail.com>
* util/unicode: split and trim in readUnicodeFile() to save repetitionMarc Mutz2025-09-091-51/+45
| | | | | | | | | | | | | | | | | ... in all the lambdas, which now receive a span<QByteArrayView> instead. Of course, this isn't a 1:1 move, but ports to QStringTokenizer and QVarLengthArray as a drive-by. Since everything is now QByteArrayView, we're now implicitly depending on QHash heterogeneous lookup, a Qt 6.8 feature, so mention that in the QT_VERSION check comment (just FYI; we've already required 6.9, anyway). Pick-to: 6.10 6.9 6.8 6.5 Change-Id: Ide6bcce5e1cd28c42f0091b5bcefb89d6278b6a9 Reviewed-by: Ahmad Samir <a.samirh78@gmail.com>
* util/unicode: prepare qFatal()s for being passed viewsMarc Mutz2025-09-091-25/+47
| | | | | | | | | | | | | | | | ... instead of QByteArrays. Define a new qPrinableView(x) macro that expands to x.size(), x.data(), suitably cast so "%.*s" will accept it, then apply it to all strings printed in qFatal()s that might become views soon. As a drive-by, improve the (touched) messages by mentioning the line number and using "" to delimit variable output. Pick-to: 6.10 6.9 6.8 6.5 Change-Id: I82434f6c8522a84daf18367a8ab5cafb74453f1c Reviewed-by: Ahmad Samir <a.samirh78@gmail.com> Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
* util/unicode: fix typoAhmad Samir2025-09-071-1/+1
| | | | | | | | Amends 44da6f996a6d687caa2d1692e4b6125f330ee3e5. Pick-to: 6.10 6.9 6.8 6.5 Change-Id: I175f1db4ef8ee40d83dabd9e352167df5ec840c4 Reviewed-by: Marc Mutz <marc.mutz@qt.io>
* QChar: micro-optimization: force inlining of qGetProp()Thiago Macieira2025-09-071-13/+2
| | | | | | | | | | | | | | | | | | | | | | | | Clang was already doing it, but GCC (at least in LTO mode) wasn't and was repeatedly calling qGetProp(). This has the benefit that, in most cases, the input character whose property we seek is UTF-16, so dead code-elimination removes the extra branch - this can happen when QString functions go through the QChar front-end, like QChar::isSpace() or isSymbol(), which route through the char32_t overload. This forced inling allows us to remove the UCS2 overloads of qGetProp() and properties(), because the same const-propagation will apply to all but one of the places where UTF-16 code units were being compared. The 16-bit qGetProp() was only used in qstring.cpp's convertCase_helper(), whose 16-bit overload was only used in foldCase(). The one exception to this is qtextengine.cpp's QBidiAlgorithm::resolveN0(): const QUnicodeTables::Properties *p = QUnicodeTables::properties(char16_t{text[pos].unicode()}); This will now call the full UTF-32 overload. Pick-to: 6.10 Change-Id: Ifa4f2d77475877f26be2fffd9a987ff994dc8ef1 Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
* util/unicode: add missing assertions on field countsMarc Mutz2025-09-051-0/+2
| | | | | | | | | | | | | | | Two functions indexed into the split() fields without first checking whether there are sufficiently many fields to index into. Add the missing Q_ASSERT()s. Yes, a qFatal() with line-number would be nicer, but all other functions do it like this, so I'm asking for forgiveness that I do it here, too. Amends the start of the public history. Pick-to: 6.10 6.9 6.8 6.5 Change-Id: Ief054796f2e058a331037540bc2a633ec7f64f2c Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
* util/unicode: remove replace() calls from remaining read*() functionsMarc Mutz2025-09-051-30/+15
| | | | | | | | | | | | | | | | | | | | | | | Some users of the split()ed value handled intervening whitespace already: - the first field is piped through parseHexRange(), which does - the second field was missing the trimmed() call before lookup. Added. All looked-up values are space-free (cf. resp. init*() functions), so that's enough, too. As a consequence, we can accept the lines by reference to const QByteArray now and, now that all lambdas have the same signature, change readUnicodeFile() from a template to a regular function taking qxp::function_ref callbacks. Amends a794c5e287381bd056008b20ae55f9b1e0acf138 (readEmojiData()) and the start of the public history (rest). Pick-to: 6.10 6.9 6.8 6.5 Change-Id: I442855a183552aa90d24810023793e6464b18162 Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
* util/unicode: createNormalizationCorrections(): remove replace() callMarc Mutz2025-09-051-3/+2
| | | | | | | | | | | | | | | | | | | | Some users of the split()ed value handled intervening whitespace already: - fields[0..1] are piped through parseHex(), which does - fields[2] is unused - fields[3] needs to be trimmed, so do that. As a consequence, we can accept the line by reference to const QByteArray now. Amends the start of the public history. Pick-to: 6.10 6.9 6.8 6.5 Change-Id: I60371820cd143b980c81a1077d9c3e34528f1830 Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
* util/unicode: readEastAsianWidth(): remove simplified() callMarc Mutz2025-09-051-3/+1
| | | | | | | | | | | | | | | | | | | All users of the split()ed value handle intervening whitespace already: - fields[0] is piped through parseHexRange(), which does - fields[1] has trimmed() called on it before lookup and all idnaStatusMap values are space-free (cf. initIdnaStatusMap()) As a consequence, we can accept the line by reference to const QByteArray now. Amends 838a7a01f388673c45af2dd60bc38992de1a2a05. Pick-to: 6.10 6.9 6.8 6.5 Change-Id: I53247332c624a192fcaca6009a3f20cb8c65786a Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
* util/unicode: readArabicShaping(): remove replace() callMarc Mutz2025-09-051-3/+1
| | | | | | | | | | | | | | | | | | | All users of the split()ed value handle intervening whitespace already: - fields[0] is piped through parseHexRange(), which does - fields[1] has trimmed() called on it before lookup and all age_map values are space-free (cf. initAgeMap()) As a consequence, we can accept the line by reference to const QByteArray now. Amends the start of the public history. Pick-to: 6.10 6.9 6.8 6.5 Change-Id: I1d371af33bc0b1c1b2bf28bbd3cbaf6820f8b4e8 Reviewed-by: Ahmad Samir <a.samirh78@gmail.com>
* util/unicode: remove replace('_', "") from readScripts()Marc Mutz2025-09-051-50/+49
| | | | | | | | | | | | | | | | | | | | | | | For some reason, the code stored the official Unicode script tags without their intervening underscores, removing underscores from the input before attempting to match, which works, as long as Unicode stays consistent in spelling properties "Like_This". Relying on that is brittle, though, seeing as a tag without intervening underscore (SignWriting) already slipped into the database, potentially matching a sought Sign_Writing. It's highly unlikely that Unicode will start to use property names that differ only by their use of underscore, but why risk it, and why confuse readers of code by using a different sought string, compared to what's in the files? Fix by storing the tags unaltered and leaving the underscores in the input alone, too. Amends the start of the public history. Pick-to: 6.10 6.9 6.8 6.5 Change-Id: I5870a35812cb3fc0b28888cb09e9f42661684a26 Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
* util/unicode: use new parseHexList() in readIdnaMappingTable()Marc Mutz2025-09-031-12/+3
| | | | | | | | | | | | | | | | | | | | This one is straight-forward. As a drive-by, use QString::append(QStringView) instead of iterating over the result of QChar::fromUcs4(). It may not be faster, in fact, I expect it to be slower, but it's much nicer to read, and this tool doesn't need to be optimized. Since every field is now clearly handled by functions that can handle extra whitespace (the values looked up unsurprisingly are space-free, too), we can drop the simplified() call and take the QByteArray 'line' by cref. Amends 2afe1a3c19239da0a7bf4dd578b334d8ff6903b3. Pick-to: 6.10 6.9 6.8 6.5 Change-Id: Id8900367d774ec4a6dccb89f6be73984caac2701 Reviewed-by: Ahmad Samir <a.samirh78@gmail.com>
* util/unicode: assert no mirrored pairs exist outside BMPMarc Mutz2025-09-021-0/+6
| | | | | | | | | | | | | | | QTextEngine implicitly assumes this (it's looking up mirrored characters in UTF-16 space, without first decoding surrogates). Add a comment there, too. Amends 7f504283ef44c35dfca7198a80742063a940fabd. Fixes: QTBUG-139456 Pick-to: 6.10 6.9 6.8 6.5 Change-Id: Ie79b33907e71cc455434127c1752898c40b128f9 Reviewed-by: Edward Welbourne <edward.welbourne@qt.io> Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
* util/unicode: error out with a meaningful message if Qt is too oldMarc Mutz2025-09-021-0/+5
| | | | | | | | | | | | | | | | | The Qt version against which this tool is build need not be the same as the Qt version for which this tool generates code. The advantage is that we can use the latest Qt features in this tool without having to worry about compat with older Qt versions. We might also use C++20 here in the future. Instead of greeting prospective users of the tool with random compile errors, check the Qt version and #error out with a descritive message instead. Pick-to: 6.10 6.9 6.8 6.5 Change-Id: I2a153ee4eb6ca1a1ea7ece39c9872f3f6d746fcd Reviewed-by: Thiago Macieira <thiago.macieira@intel.com> Reviewed-by: Ahmad Samir <a.samirh78@gmail.com>
* util/unicode: Extract Method parseHexRange()Marc Mutz2025-09-021-130/+29
| | | | | | | | | | | | Wrapping parseHexList(), which gets extended to support QLatin1StringView separators, add parseHexRange() and use it around the code to parse HHHHH[..HHHHH] hex ranges. Amends the start of the public history. Pick-to: 6.10 6.9 6.8 6.5 Change-Id: I0372e5c239642988f0e920d95108657e276b19dd Reviewed-by: Ahmad Samir <a.samirh78@gmail.com>
generated by cgit v1.2.3 (git 2.25.1) at 2025-12-17 00:50:09 +0000