Tweet
In release V11 build 1002 we made some changes to the database test to make the test score more consistent over long and short test runs. We are hoping it might also fix up (or at least improve) the behaviour reported above.
The internals of the test runs various SQL commands. CREATE, INSERT, SELECT & DROP being the main ones. And the database tables are fairly large, so as to use a lot of RAM. The INSERT and SELECT steps are the slow steps. CREATE and DROP are fast and don't count for much. This is done in a loop for the duration of the test. The test duration varies with the selections made in the preferences window.
What we noticed after a bunch of careful measurements on higher core count machines was that the first loop was a lot slower than subsequent loops. This barely matters for a long test, where many loops are done. But it matters a lot for short tests, where only a single test loop might be done in the worst case.
Our theory about this is that there is some blocking occurring in SQLLite during the INSERT step when there is a large number of threads all trying to allocate RAM at the same time. i.e the RAM allocation process regresses to being single threaded. After the 1st loop however, it gets a lot faster, as we suspect SQLLite holds on to the allocated RAM even after the table is DROPped.
So what we are trying out is to do a pre-test "warm up loop", which is excluded from the benchmark time measurement. It will make the whole test slightly longer, but should deliver more consistent results, especially for short test runs.
The internals of the test runs various SQL commands. CREATE, INSERT, SELECT & DROP being the main ones. And the database tables are fairly large, so as to use a lot of RAM. The INSERT and SELECT steps are the slow steps. CREATE and DROP are fast and don't count for much. This is done in a loop for the duration of the test. The test duration varies with the selections made in the preferences window.
What we noticed after a bunch of careful measurements on higher core count machines was that the first loop was a lot slower than subsequent loops. This barely matters for a long test, where many loops are done. But it matters a lot for short tests, where only a single test loop might be done in the worst case.
Our theory about this is that there is some blocking occurring in SQLLite during the INSERT step when there is a large number of threads all trying to allocate RAM at the same time. i.e the RAM allocation process regresses to being single threaded. After the 1st loop however, it gets a lot faster, as we suspect SQLLite holds on to the allocated RAM even after the table is DROPped.
So what we are trying out is to do a pre-test "warm up loop", which is excluded from the benchmark time measurement. It will make the whole test slightly longer, but should deliver more consistent results, especially for short test runs.
Comment