aboutsummaryrefslogtreecommitdiffstats
path: root/src
Commit message (Collapse)AuthorAgeFilesLines
...
* Reset countDown each run of the testRobin Burchell2017-04-251-1/+3
| | | | | | | | This may contribute to some test instability, although I see no real difference locally. Change-Id: I078029e4abec08aaaf1347a2e028d38d2c43ac70 Reviewed-by: Gunnar Sletta <gunnar@crimson.no>
* qmlbench: Switch to using std::cerr for output for debugRobin Burchell2017-04-244-31/+26
| | | | | | | | This way, we can enable it while still outputting json to stdout, which means we have feedback on what is going on. Change-Id: I1fd65464c410ed61077aa664789a86e84dc57d96 Reviewed-by: Gunnar Sletta <gunnar@crimson.no>
* change default run mode to be 'frame-count'Gunnar Sletta2017-04-241-3/+1
| | | | | Change-Id: I424e7fd0c2b93541c37f7857a2a00cfe098b4aef Reviewed-by: Robin Burchell <robin.burchell@crimson.no>
* Don't pass benchmark to BM runnerRobin Burchell2017-04-043-10/+14
| | | | | | | | It can work it out itself. This will make subsequent timing changes cleaner. Change-Id: Iff7a6abc080952180d51ec39576688b203be1fcc Reviewed-by: Gunnar Sletta <gunnar@crimson.no>
* Skip manual tests unless they are explicitly runRobin Burchell2017-03-301-0/+5
| | | | | | | These are not useful for automation purposes Change-Id: I5948814c9c73aa7d21216c3081a7bbf7bcc3b254 Reviewed-by: Gunnar Sletta <gunnar@crimson.no>
* Add a flag to mark benchmarks as having a "boolean" resultRobin Burchell2017-03-242-5/+15
| | | | | | | | | | | | | Some of our benchmarks don't have a meaningful performance indicator (in that they won't ever really improve or regress on the percentage figure), but are still of interest in that they let us know if a particular feature is totally broken or not. This flag lets us easily mark such tests for ease of discovery when looking into results / implementation of benchmarks. Change-Id: Icda0981ba32d9abe84a835c0ca6ccb8a01be374d Reviewed-by: Gunnar Sletta <gunnar@crimson.no>
* Add a base Benchmark and CreationBenchmark typeRobin Burchell2017-03-244-0/+104
| | | | | | | | These will allow us to make benchmarks more succinct without adding too much complexity. Change-Id: I5b38dfc388d1d56780eb107d36060d3e1c23cdea Reviewed-by: Gunnar Sletta <gunnar@crimson.no>
* Move host process away from using QGuiApplicationRobin Burchell2017-03-242-9/+31
| | | | | | | | | This makes it more robust from window system or OpenGL issues, and mean that it may stand a chance of actually reporting failures if a benchmark locks up due to a serious problem in either of those. Change-Id: I6d7372a9161f0f3565822f6990d621af34b41425 Reviewed-by: Gunnar Sletta <gunnar@crimson.no>
* Move result recorder stuff to result recorderRobin Burchell2017-03-243-10/+7
| | | | | | | | | | | And at the same time, make subprocesses capable of overriding JSON output from the host. This is the last remaining step before removing QGuiApplication from the host. Change-Id: I1e8f9798e691b85441295619ca2d2c2b2238dc9c Reviewed-by: Gunnar Sletta <gunnar@crimson.no>
* Split host and process mode out of mainRobin Burchell2017-03-241-90/+105
| | | | | | | | This is mostly a tidyup, but it will also be followed by making host mode use QCoreApplication to be resilient to GPU/window system issues. Change-Id: I098035b9bfdd12668b7c1e87bb83453bbba352b0 Reviewed-by: Gunnar Sletta <gunnar@crimson.no>
* Compile with Qt 5.6Eskil Abrahamsen Blomfeldt2017-03-241-0/+1
| | | | | Change-Id: Ida278c76c1c268f01d5b070033f9c61f8ea2526c Reviewed-by: Robin Burchell <robin.burchell@crimson.no>
* Split BenchmarkRunner out into a separate fileRobin Burchell2017-03-234-177/+258
| | | | | | | | No functional changes. Change-Id: I2317331b28620320b5fa6946442341f8f442108d Reviewed-by: Eskil Abrahamsen Blomfeldt <eskil.abrahamsen-blomfeldt@qt.io> Reviewed-by: Gunnar Sletta <gunnar@crimson.no>
* Split ResultRecorder out to a separate fileRobin Burchell2017-03-234-156/+250
| | | | | | | | Cleanup only, no functional changes. Change-Id: Ie9f6cfea744300b47551776ea622726a73727278 Reviewed-by: Eskil Abrahamsen Blomfeldt <eskil.abrahamsen-blomfeldt@qt.io> Reviewed-by: Gunnar Sletta <gunnar@crimson.no>
* Clean up BenchmarkRunner a bitRobin Burchell2017-03-233-55/+73
| | | | | | | | | | | | | After the process split, BenchmarkRunner no longer runs multiple benchmarks, so a lot of the code here was dead - so we can remove it. While we're at it, move the benchmarks to Options - as that's where they really belong. This removes the final tie from BenchmarkRunner to the host mode process, which makes things a little tidier still. Change-Id: I180fd61d1eb7267d68f06642d78383d8c0afc55c Reviewed-by: Gunnar Sletta <gunnar@crimson.no>
* BenchmarkRunner: Merge redundant methodsRobin Burchell2017-03-231-6/+0
| | | | | | | No need to have two methods we run on completion Change-Id: Iaeb3f1ec9c45e395c973ebfb205448762695f494 Reviewed-by: Gunnar Sletta <gunnar@crimson.no>
* Benchmark: Remove redundant variableRobin Burchell2017-03-231-3/+0
| | | | | Change-Id: Iedc7b9a35f574734480bcb4edbcda520bbb9a042 Reviewed-by: Gunnar Sletta <gunnar@crimson.no>
* Alter standard deviation calculationRobin Burchell2017-03-221-1/+4
| | | | | | | | | | | | | | | | | | | | | | | | As with many things mathematical, there is more than one way to skin a cat. In the case of standard deviation, there is "uncorrected sample standard deviation" (or "population standard deviation"), and there is "corrected sample standard deviation". There's probably more, but the former is what we're using now, and the latter is what this change introduces. Quoting Erik: the "issue" is that the sqrt biases toward small numbers because it's a non-linear function. so the uncorrected one will give smaller x_i - avg a higher "weight" than the outliers where x_i-avg is bigger. Note: This will cause an increase in CoV values (since they are based off of stddev). This is probably acceptable due to the increased "precision" gained though. Change-Id: I0e9c8bd245de47fe7099baefc2966b744ec78028 Reported-by: Erik Verbruggen <erik.verbruggen@qt.io> Reviewed-by: Gunnar Sletta <gunnar@crimson.no> Reviewed-by: Erik Verbruggen <erik.verbruggen@qt.io>
* Add some edge case handlingRobin Burchell2017-03-221-4/+20
| | | | | | | | | | | | | | | | | | | First and most importantly, don't let a benchmark run indefinitely. At least on TX1, it seems that things can occasionally lock up (hard). After running fine for ~1.5 days, my TX1 "hung" on a benchmark, seemingly for GPU related reasons (at least, so I am informed in dmesg). To handle this, we wait only so long before giving up, logging the failure, and not attempting to start any other tests. We do this as a precaution, as if we run into this condition, it likely means either that someone is developing, or the harwdare needs a reset. Such hardware reset is left to the harness running qmlbench. In addition, we also log crashes a bit more noisily, on the offchance that they are interesting to know about. Change-Id: Id8d9eb50a29eb295669e743169a8d3e69c737776 Reviewed-by: Eskil Abrahamsen Blomfeldt <eskil.abrahamsen-blomfeldt@qt.io>
* Move average() to a functionRobin Burchell2017-03-211-4/+10
| | | | | | | | Fits well alongside stddev, and lets us test this stuff easier in the future if wanted/needed. Change-Id: I5d57103f94d0ba001826a3116cef9a069d3ad9a9 Reviewed-by: Gunnar Sletta <gunnar@crimson.no>
* Rip out bucketingRobin Burchell2017-03-212-50/+13
| | | | | | | | | | | This was an attempt at stabilising the results from the "default" shell. However, as we are not using this for other shells, and on the limited known-good HW the default shell is happy with the results are quite stable, this seems to be an unnecessary complication. Change-Id: I683f294d8a698de86ccda09f62bd61276fc09e6a Reviewed-by: Eskil Abrahamsen Blomfeldt <eskil.abrahamsen-blomfeldt@qt.io> Reviewed-by: Gunnar Sletta <gunnar@crimson.no>
* Compile with Qt 5.7 and belowEskil Abrahamsen Blomfeldt2017-03-171-0/+6
| | | | | | | The setFlags() method was added in Qt 5.8. Change-Id: I1c656e026429ded9e391fd0962d1f7032532d09a Reviewed-by: Robin Burchell <robin.burchell@crimson.no>
* Split main into smaller piecesRobin Burchell2017-03-171-30/+37
| | | | | | | | One for handling command line arguments (easy, now we stuck everything in Options), and one for setting up the surface format if required. Change-Id: I32942db53c08d38b5e504319b95869feb846edde Reviewed-by: Gunnar Sletta <gunnar@crimson.no>
* Move isSubProcess to the Options structRobin Burchell2017-03-172-6/+8
| | | | | | | Will make way for some tidyups Change-Id: I565a7a40a960930a44f6e9f6085e98736d97fd3f Reviewed-by: Gunnar Sletta <gunnar@crimson.no>
* Move id to Options structRobin Burchell2017-03-172-1/+3
| | | | | | | Preparing for cleanup of argument handling. Change-Id: I767c34a0e6f8378e6f23300c7f1375bfc6f3474b Reviewed-by: Gunnar Sletta <gunnar@crimson.no>
* Move onlyPrintJson bool to the Options structRobin Burchell2017-03-172-95/+57
| | | | | Change-Id: I854734628eeaf7bc846e7e0b41a72de7104e4be2 Reviewed-by: Gunnar Sletta <gunnar@crimson.no>
* Make options a static classRobin Burchell2017-03-172-0/+75
| | | | | | | This will allow us to clean up option parsing a bit. Change-Id: I69756f81a19293e75be103245d18e4178673b2ae Reviewed-by: Gunnar Sletta <gunnar@crimson.no>
* Split qmlbench into two "process" modesRobin Burchell2017-03-171-7/+96
| | | | | | | | | | | The host mode (what you actually run) no longer runs benchmarks. It exists to proxy output from the subprocess which runs the test itself. This may help to avoid "contamination" of global state (caches and similar) on multiple levels (drivers, Qt/QML/Quick itself, etc). Change-Id: Ifc46f16cb5b02c93e48c691bc99658b9831d19d1 Reviewed-by: Gunnar Sletta <gunnar@crimson.no>
* Split decidefps out to a separate toolRobin Burchell2017-03-151-108/+0
| | | | | | | | This isn't used constantly, so having it as a separate tool cleans things up a little. Change-Id: I710b57161337af26c018fa42cf8362a100df5cdd Reviewed-by: Gunnar Sletta <gunnar@crimson.no>
* Rearrange source treeRobin Burchell2017-03-1510-0/+2887
Move qmlbench binary to src/, leaving the top directory clean. Change-Id: If0113d503dba5f567a52dfb94c76fa1b2692d992 Reviewed-by: Gunnar Sletta <gunnar@crimson.no>
generated by cgit v1.2.3 (git 2.25.1) at 2025-12-16 22:31:15 +0000