Announcement

Collapse
No announcement yet.

Why is the file size so big? I got 10.2 to work from a 32MB (not a typo) MMC card...

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Why is the file size so big? I got 10.2 to work from a 32MB (not a typo) MMC card...

    One thing I couldn't help but notice is that the disk space required by MemTest86 seems a bit excessive for how much it actually uses, and I managed to prove this to myself by being able to get it to work from a 32MB (not a typo) MMC card.

    From there, I was able to dd the image in Linux to then have a small file size that can be easily launched via Ventoy - a situation where a difference of a full gigabyte can be the difference between whether you're able to fit some ISOs or not (in my case, it was Foxclone Edge which is around 800MB).

    The trick is to create a ramdisk using something like Romex Primo Ramdisk and configure it as a SCSI disk that does not automatically create the TEMP folder and has a ramdisk size of 513MB (not a typo). From there, use Rufus v3.1.1320 (newer versions seem to fail?), press Ctrl+Alt+F and select the ramdisk as your destination drive, then select the normal memtest86-usb.img file that is provided in the ZIP download.

    Now from there, you can use a program like EaseUS Partition Master to clone the ramdisk onto a smaller-capacity flash disk, whether a USB drive, SD/MMC card, etc (make sure the destination flash disk is using GPT - I know that at least EaseUS Partition Master can convert MBR to GPT). Then, after that, connect the disk in Linux and run the following command to create a .img of it that will "just work" in Ventoy:

    sudo dd if=/dev/sd* status=progress of=~/Downloads/MemTest86.img


    If you want to be more fancy-pants, you can shrink and move the two FAT16 partitions to their smallest sizes (I found 9MB to be the smallest but, at least in EaseUS Partition Master, I actually had to first resize to 32MB and then resize to 9MB; also, I couldn't resize the second FAT16 partition until I first manually assigned a drive letter via diskpart.exe due to it being a hidden partition or something), then trimming off the end of your resulting .img file and then subsequently writing a recreated backup portion of the GPT partition table that's normally located at the end.

  • #2
    An 8GB flash drive is $3 on NewEegg (not a typo). This includes free shipping. So a new 8GB drive is near free.

    We are stating in system requirements that a 1GB USB drive is required (and smallest drive available today is 8GB at NewEgg).

    So advantages of size optimisation are pretty much non-existent.

    The technical reason for the size are three fold.
    1) Some BIOS have quirks and two partitions gave better boot compatibility.
    2) We write log files, benchmark report and test reports to the flash drive during the test run. So some free space is required for these files. Especially if you are doing 100s of runs.
    3) The RAM DMA (Direct Memory Access), Test #14, test writes data to the flash drive. This test is off by default. But if turned on you need a reasonable amount of free disk space. So your small build would break this test.

    Comment


    • #3
      Unfortunately it seems that, because I'm a new user, I cannot edit my opening post to include various corrections and-or additional information... But it sounds like you're basically saying that the likes of Ventoy are simply not a consideration and neither is the use of older drives that one already has laying around that may otherwise be considered "ewaste".

      Regardless, if you're feeling daring to try a .img file made by a random user like myself, I just want to prove the point that I was able to make MemTest86 Free v10.2 fit within a mere 17MB .img file - and this includes having both of the two aforementioned partitions (file will be automatically deleted in around a year):And for file integrety to prove that I've not tampered with it (other than optimizing the mt86.png file to be even smaller), you can hash at the SHA256 checksums of the files inside of the 3 .img files within the actual 17MB .img and compare them to those found within the official free memtest86-usb.img
      Last edited by NM64; Mar-24-2023, 12:03 AM.

      Comment


      • #4
        But it sounds like you're basically saying that the likes of Ventoy are simply not a consideration​.
        We haven't tested with Ventoy, but it seems someone did, as they are listing Memtest86 as being compatible.

        the use of older drives that one already has laying around
        Our suggest min size is 1GB, this covers pretty much all brand name flash drives made in the last 10 years.
        And to be honest you don't really want to use a USB1 or USB2 drive anyway, boot will be much slower and writing of logs will also be slower (so test time extended).
        If you look through the forums you will also find numerous people complaining about MemTest86 bugs (e.g. very slow testing, DMA test failure) that turned out to be failures of older flash drives.

        And for file integrety to prove that I've not tampered with it
        Main binary executable file is verified and code signed by Microsoft. This required for the secure boot functionality in UEFI BIOS.
        So any modification to the main binary will break secure boot (if you have it enabled in BIOS) and the drive won't boot.

        Comment


        • #5
          Originally posted by David (PassMark) View Post
          We haven't tested with Ventoy, but it seems someone did, as they are listing Memtest86 as being compatible.
          If it means anything, I've been using it with Ventoy for the last few days in-combination with my manually-shrunken .img file. My point was that, when you're putting a bunch of files on a single flash disk so that you only need to carry one disk, then file size matters again.

          Originally posted by David (PassMark) View Post
          Our suggest min size is 1GB, this covers pretty much all brand name flash drives made in the last 10 years.
          FYI, the bundled image writer won't work on actual 1GB USB drives (yes, I tried).

          Originally posted by David (PassMark) View Post
          And to be honest you don't really want to use a USB1 or USB2 drive anyway, boot will be much slower and writing of logs will also be slower
          USB1 predates USB flash drives altogether, so that's a non-issue.

          USB2 is fine and it's what I've been using, unless you think a boot time of 1 to 2 minutes is too slow, and logs are a non-issue if using Ventoy (unless, of course, you want to actually read the logs after-the-fact, but I care not for them). If speed were an issue then I'd be booting from one of my backup SSDs that double as a Ventoy disk (what I do is give every PC a second disk that is for nothing but backups, but I also install Ventoy onto it before-hand and throw on a couple of ISOs and utilities such as whatever the corresponding live ISO is for the distro installed). And, for reference, Ventoy not only works from SATA but you can put the same SATA SSD into a USB enclosure and Ventoy will still work (or you can just use eSATA on systems without USB 3.0...though I've found that you can't boot from native built-in eSATA on AMD systems annoyingly enough)

          Originally posted by David (PassMark) View Post
          MemTest86 bugs (e.g. very slow testing, DMA test failure) that turned out to be failures of older flash drives.
          Old flash disks have more reliable NAND flash though due to predating TLC and possibly even MLC depending on their age, and they're not only manufactured on considerably larger node sizes (which additionally improves lifetime of NAND flash) but, because SSDs and especially enterprise SSDs weren't really a thing yet, they also aren't getting the cheapest and lowest-binned NAND.

          Considering the above, non-SSD flash disks with best reliability should be those from the late 2000s if not mid 2000s which should also additionally have wear-leveling as well as enough general disk capacity for everyday use so that you're not constantly overwriting all of the memory cells. It should be worth noting that, of the flash disks I've had become problematic, they were all ones made in the 2010s, yet all the ones I have made in the 2000s still work to this day.


          Originally posted by David (PassMark) View Post
          Main binary executable file is verified and code signed by Microsoft. This required for the secure boot functionality in UEFI BIOS.
          So any modification to the main binary will break secure boot (if you have it enabled in BIOS) and the drive won't boot.
          Yeah uh, I'm a home Linux user, so secure boot and the like are not exactly things that I fawn over

          Also, my affinity for ECC, integrated graphics, finer-tuned fan control, and CPU underclocking+undervolting (to farther reduce fan noise and also heat output) means that I've generally been stuck with AM3+ socket and older AMD hardware until recently when Pro APU AM4 chips have become more readily available (also AM5 is still $$$), not including a Asus Z87-Pro that I recently "revived" which I was given for free from a friend many years ago since it "supposedly had a problem" (thus far I've not found any issues, but the first thing I did was update the BIOS, so maybe that fixed it?).
          Last edited by NM64; Mar-24-2023, 12:56 AM.

          Comment


          • #6
            Image size is 1,073,741,824 bytes. Which is exactly 1GB (2^30). So I am guessing your old 1GB drive was actually not 1GB in capacity.

            US patent for a USB flash drive was filed in April 1999 by Israeli firm M-Systems, the same year USB flash drives were released. This was before USB2.0 was in use.

            Here is a photo of M-Systems “DiskOnKey" 8MB flash drive from 2000. Interface was USB1.1 (12Mbit/sec)

            There was one available on Ebay for $7500
            Click image for larger version

Name:	First-Flash-Drive.webp
Views:	580
Size:	5.6 KB
ID:	54695

            Old flash disks have more reliable NAND flash
            Depends entirely on the brand.
            We've been selling software on flash drives for ~15 years. The cheap ones have an insanely high failure rate. We had 30 fail from 100 units tested (old USB2 2GB units).
            After switching to brand name drives, Kingston in our case, failure rate is more like 0.1%. Too low to measure. This was despite a x16 capacity increase and x5 speed increase.

            So old doesn't equal good.

            Comment


            • #7
              Regardless of debating semantics, I still disagree with making the image 1GB under the premise of "storage is cheap" but I'm not surprised by such a stance considering that the last version that supported ECC polling on non-UEFI systems was v3.5b which itself is only accessible via archive.org...

              (but that's also not a surprise when you only recommend USB3 anyway which was a thing right around the same time as UEFI - you could sometimes get boards around then with UEFI but no USB3 like the Thinkpad T420 or sometimes USB3 but no UEFI like the Gigabyte GA-880GMA-USB3)


              Originally posted by David (PassMark) View Post
              Image size is 1,073,741,824 bytes. Which is exactly 1GB (2^30). So I am guessing your old 1GB drive was actually not 1GB in capacity.
              Spoiler alert - most drives aren't exactly exactly their rated capacity down to the last byte. I have access to three different 1GB flash disks and they're all a teeny bit less than exactly 1GB (and, to be clear, this isn't unique to specifically 1GB):
              • SolidWorks-branded disk (USB drive): 1,047,003,136 bytes
              • SanDisk Ultra II (CF card): 1,024,966,656 bytes
              • Verbatim Store 'n' Go (USB drive): 1,010,826,752 bytes



              Originally posted by David (PassMark) View Post
              Depends entirely on the brand.
              We've been selling software on flash drives for ~15 years. The cheap ones have an insanely high failure rate. We had 30 fail from 100 units tested (old USB2 2GB units).
              After switching to brand name drives, Kingston in our case, failure rate is more like 0.1%. Too low to measure. This was despite a x16 capacity increase and x5 speed increase.

              So old doesn't equal good.
              But Kingston doesn't actually make any of their NAND... I was talking about the NAND flath chips themselves.

              Thing is, there are controller failures and there are NAND flash failures, and I was only referring to the latter while it's the former (controller failures) that is something that newer devices can definitely be better at (I've experienced this myself with SATA SSDs whereby the only SSD failures I've had are controller-related and only occurred on older drives). Controller failures are more common when higher speeds are involved which lines up with your recommendation of USB3, and controller failure is also the most common mode of failure for cheap devices like you mention since there are only a few companies in the world that actually make NAND flash.

              One thing is that, back in the 2000s, there were much fewer players in the flash memory world so you didn't really have the whole super-cheap bulk items that you can find nowadays. Also, I cannot help but think that USB2 is likely slow enough that it puts much less stress on the controller which is going to alleviate risk of controller failure.
              Last edited by NM64; Mar-24-2023, 03:26 AM.

              Comment


              • #8
                I still disagree with making the image 1GB under the premise of "storage is cheap"
                That wasn't the main motivation. See my first post for details.

                the last version that supported ECC polling on non-UEFI systems
                ECC support never fully worked in V3 and V4.
                See the release notes from Version 4.0 28/Mar/2011
                This was added and removed before we took over the project. So it had nothing to do with us.
                We started to add support back in once we took over maintancence and did V5 in 2013, but it is never ending.

                most drives aren't exactly exactly their rated capacity down to the last byte​
                Yes, 1GB of data won't fit on a 0.9GB drive.



                Comment


                • #9
                  Originally posted by David (PassMark) View Post
                  That wasn't the main motivation. See my first post for details
                  It seems that I need to break down that first post more individually, so I suppose let's go through it then...

                  Originally posted by David (PassMark) View Post
                  An 8GB flash drive is $3 on NewEegg (not a typo). This includes free shipping. So a new 8GB drive is near free.

                  We are stating in system requirements that a 1GB USB drive is required (and smallest drive available today is 8GB at NewEgg).
                  ...aka "storage is cheap", no?


                  Originally posted by David (PassMark) View Post
                  So advantages of size optimisation are pretty much non-existent.
                  In a pre-Ventoy world where you had to use individual flash disks anyway, sure. But the same is not ture in a post-Ventoy world where disk images are competing for space with other disk images, and I've not found one person that didn't find Ventoy to be kind of brilliant (heck I've started even installing Ventoy onto my flash disks that are so small that are really only used to just boot a single thing so that I can much more easily retrieve the corresponding disk image, like when needing to access the plain ISO to boot it in VirtualBox or the like)

                  Originally posted by David (PassMark) View Post
                  1) Some BIOS have quirks and two partitions gave better boot compatibility.
                  Even my example 17MB .img file contains those two partitions (both of which only had around 8MB of actual used data even in the unmodified memtest86-usb.img)

                  Originally posted by David (PassMark) View Post
                  2) We write log files, benchmark report and test reports to the flash drive during the test run. So some free space is required for these files. Especially if you are doing 100s of runs.
                  I've already re-reformatted my previous 8GB flash drive with Ventoy that originally had MemTest86 installed to it in the traditional way (using the disk image writer that you provide) so I can't check, but do log files really need 200MB of disk space per each of the two aforementioned partitions?

                  In particular, I help with a fan-made translation & HD enhancement patch of the visual novel "Fate/stay night" which is known to have more story text than the Lord of the Rings trilogy and The Hobbit combined, and the file size of the entire story in uncompressed plain text is a mere 4MB (and that's including various krkrz-engine scripting code as well).

                  Originally posted by David (PassMark) View Post
                  3) The RAM DMA (Direct Memory Access), Test #14, test writes data to the flash drive. This test is off by default. But if turned on you need a reasonable amount of free disk space. So your small build would break this test.
                  Understood

                  (at least in my case, the DMA test always gave errors anyway, so I had minimal use for it but obviously your mileage may vary in this regard)
                  Last edited by NM64; Mar-24-2023, 04:33 AM.

                  Comment


                  • #10
                    D'oh! Apparently there's a time limit on editing... Well, I thought of a better example regarding the disk space that might be required for logs:

                    I have a checksum file formatted as plain text that contains all of the SHA hashes and file name of every file that was located on my OS partition before my last disk backup and the file size is "only" 124MB despite it containing a ridiculous 130 million characters of text (specifically 130212566 characters spread out across 971288 lines with it being one line per file hash)

                    Amusingly, 124 * 2 = 248 which is exactly 8MB less than 256, and those ~256MB partitions had ~8MB of data on them. This means that, by default, MemTest86 provisions enough disk space to store plain text logs that contain over 260 million characters...

                    Comment


                    • #11
                      DMA test always gave errors anyway
                      So either you old flash drive is bad, or you have broken the software.

                      It is your choice in the end how you install it.

                      We've made a decision based on what we think is optimal for the majority of users based on the technical requirements of the software.
                      Optimal = Easy, fast, cheap, fully functional.

                      There might be future scope to remove one of the partitions as the motherboards with the quirks requiring this are getting old & rare now.

                      Comment


                      • #12
                        Originally posted by David (PassMark) View Post

                        So either you old flash drive is bad, or you have broken the software.
                        Sorry, I meant that it just gives constant ECC errors regardless of actual RAM stability. It may be worth noting that my go-to ECC testing motherboard is noted my MemTest86 as having known issues with setting the CPU mode to parallel, so issues with something experimental would make sense, no?


                        Originally posted by David (PassMark) View Post
                        We've made a decision based on what we think is optimal for the majority of users based on the technical requirements of the software.
                        Optimal = Easy, fast, cheap, fully functional.
                        I still at least think it'd make more sense to make it be the teeniest bit smaller so that it can actually fit on most 1GB flash disks rather than those that are exactly exactly 1GB.

                        I also still think that some sort of acknowledgement of the Ventoy ecosystem (if you can call it that) would be ideal for end-users going forward as it gains in popularity, but I suppose I'm not exactly surprised that you've chosen not to considering your stance on everything else.

                        Comment

                        Working...
                        X