Announcement

Collapse
No announcement yet.

OSFMount GetFinalPathNameByHandleW fails with ERROR_INVALID_FUNCTION

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • OSFMount GetFinalPathNameByHandleW fails with ERROR_INVALID_FUNCTION

    When using Python functions os.path.realpath() or pathlib.Path.resolve(strict=True) on a drive created by OSFMount, the operation fails with OSError: [WinError 1] Incorrect function. This error does not occur on standard physical NTFS drives.

    This issue is critical for modern Python applications and web servers (e.g., ComfyUI, aiohttp) that rely on these functions to resolve canonical file paths.
    ​​
    Steps to Reproduce:
    1. Create a RAM disk (NTFS) using OSFMount (e.g., drive R:)
    2. Save and run the following Python test script
    3. Observe the output
    Test Script​:

    Code:
    import os
    import pathlib
    
    def test_realpath_on_drive(drive_letter: str):
        test_dir = f"{drive_letter}:\\test_osfmount_{os.getpid()}"
        
        os.makedirs(test_dir, exist_ok=True)
        dir_exists = os.path.exists(test_dir) and os.path.isdir(test_dir)
        
        print(f"Testing on {drive_letter}:\\")
        print(f"  Directory exists: {dir_exists} ({test_dir})")
        
        try:
            result = pathlib.Path(test_dir).resolve(strict=True)
            print(f"  pathlib.Path.resolve(strict=True): SUCCESS -> {result}")
        except OSError as e:
            print(f"  pathlib.Path.resolve(strict=True): FAILED -> {e}")
        
        try:
            result = os.path.realpath(test_dir, strict=True)
            print(f"  os.path.realpath(strict=True): SUCCESS -> {result}")
        except OSError as e:
            print(f"  os.path.realpath(strict=True): FAILED -> {e}")
        
        os.rmdir(test_dir)
        print()
    
    print("Python version:", os.sys.version)
    print("\nTesting realpath behavior:\n")
    
    test_realpath_on_drive("R")
    test_realpath_on_drive("D")​


    Expected Result:
    Both functions should return the canonical path without errors on all drives.

    Actual Result:
    On the OSFMount RAM disk (R:), both functions fail with:
    OSError: [WinError 1] Incorrect function: 'R:\\test_folder'

    On a local physical NTFS drive (D:), both functions succeed.


    Environment Information:
    • OSFMount Version: 3.1.1003.0
    • OS: Windows 10/11
    • Python Version: 3.12.10, 3.9.11
    • RAM Disk Configuration: Type: RAM Disk, File System: NTFS
    Additional:
    The underlying Windows API function GetFinalPathNameByHandleW appears to return the error ERROR_INVALID_FUNCTION (1) when called on files located on the OSFMount virtual drive. This breaks standard library functionality in Python and likely affects other applications that rely on this system call.

    This issue blocks the operation of several modern software packages. A fix or workaround guidance would be greatly appreciated.
    ​​

  • #2
    Before we spend any time on this can you confirm the type of drive you are creating.
    Logical volume or physical disk emulation?

    Volumes are missing a bunch of functionality as they function more like remote network shares.

    And are you sure the Windows API function GetFinalPathNameByHandleW() actually fails if called directly from C / C++ ?
    (if you haven't done this we can)


    Click image for larger version

Name:	image.png
Views:	54
Size:	42.6 KB
ID:	60122

    Comment


    • #3
      I'm not using GUI
      OSFMount.com -a -t vm -s 96G -o format:ntfs:"RAMDRIVE_96GB" -m R:

      Comment


      • #4
        After your question I've started GUI and see that it is a logical drive.
        Then run --help and found -o logical|physical option.
        So by default it is a logical...​

        Comment


        • #5
          Sorry for three posts in line. Can't edit even unapproved post.
          No, I'm not sure about C/C++. I'm faced it when using python.

          Comment


          • #6
            Sorry about the no edit rule. We had too many spammers (they come back a few days like and edit past posts to add porn or gambling links).

            Can you try with Physical disk emulation.

            It might also be a Python bug. There has been this bug report open since 2017
            https://bugs.python.org/issue31842

            Comment


            • #7
              Yes, I have tried using the Physical disk emulation mode. In this mode, that specific error is absent, but this mode is unsuitable for several reasons:

              1) I cannot assign a fixed drive letter immediately upon creation, as is possible with a logical disk.
              2) Assigning the desired drive name requires numerous additional manipulations - finding the mount point of this physical disk, locating the assigned drive name (if one was even assigned, which isn't guaranteed), and then changing it. Using some utilities for this is risky because something could accidentally go wrong and affect one of the real physical disks.
              3) Saving images and then loading them is also not a viable option.
              4) Using a temporary file in %TEMP% is a very poor idea as well.

              In principle, creating a disk should not involve any extra write operations or figuring out "where the disk is." Creating a disk with a specified drive letter is the ideal solution​. This ideal solution (a logical disk with a pre-assigned drive letter) is precisely where the GetFinalPathNameByHandleW API failure occurs, which is the core bug I am reporting.


              The entire point of automation and using the command line solely with osfmount itself is completely lost.​

              Comment


              • #8
                Thank you for the reference to bpo-31842. I have reviewed the discussion, and while the symptoms appear similar, I believe the core context and implication of my report are different.

                1. Different Scope of Failure: the bpo-31842 discussion largely revolves around how Python's pathlib should handle certain Windows errors (like ERROR_INVALID_FUNCTION) in its non-strict mode (strict=False). My issue demonstrates that the error occurs in the strict mode (strict=True) as well, which is designed to call the underlying system API and is expected to fail only on truly invalid paths. This indicates a more fundamental compatibility issue.

                2. Root Cause is an Open Question: I cannot definitively state the root cause is within the OSFMount driver. My evidence is the observed behavior:
                • The GetFinalPathNameByHandleW API call fails with ERROR_INVALID_FUNCTION on an OSFMount logical volume.
                • The same call succeeds on standard drives and on an OSFMount physical disk emulation volume.
                • This failure propagates up and breaks standard Python functions (os.path.realpath, Path.resolve).
                The precise origin of this API failure (driver, system layer, specific configuration) would need to be diagnosed by your team or someone with deep Windows kernel expertise.

                3. The Practical Problem Remains: regardless of the ultimate cause, the result is that the default and most automated way to use OSFMount (logical volume with a fixed drive letter) is incompatible with a standard Windows API call. This, in turn, breaks a wide array of modern Python software and frameworks that rely on this API for correct path resolution.

                Therefore, my report is about a functional incompatibility between OSFMount's logical volume mode and a common system path resolution workflow. The workaround of using physical disk emulation is, as I detailed, cumbersome and defeats the purpose of automation.

                I hope this clarifies why I believe this is a distinct issue that warrants attention.​

                Comment


                • #9
                  With a bit of scripting it should be possible to use physical drive emulation.

                  Here is a Powershell script to find a raw physical drive and partition it and assign a drive letter (hard coded to Z: at the moment).

                  Code:
                  # 1. Get the first available 'Raw' disk that matches the specific name pattern
                  $disk = Get-Disk | Where-Object {
                  $_.PartitionStyle -eq 'Raw' -and
                  $_.IsSystem -eq $false -and
                  $_.FriendlyName -match "osfdisk"
                  } | Select-Object -First 1
                  
                  if ($disk) {
                  Write-Host "Found matching disk: Number $($disk.Number) - FriendlyName: $($disk.FriendlyName)" -ForegroundColor Green
                  
                  # 2. Initialize the disk
                  Write-Host "Initializing Disk $($disk.Number)..."
                  Initialize-Disk -Number $disk.Number -PartitionStyle GPT -PassThru |
                  
                  # 3. Create a new partition using max size and FORCE drive letter Z
                  # Note: This will fail if Z: is already in use by a network drive or USB stick.
                  New-Partition -DriveLetter Z -UseMaximumSize |
                  
                  # 4. Format the volume
                  Format-Volume -FileSystem NTFS -NewFileSystemLabel "OSF_Data" -Confirm:$false
                  
                  Write-Host "Drive setup complete! Volume mounted at Z:" -ForegroundColor Green
                  } else {
                  Write-Host "No uninitialized disks found with 'osfdisk' in the name." -ForegroundColor Yellow
                  }​
                  I also wrote some C++ code (attached) to check how GetFinalPathNameByHandleW() works with RAM drive volumes.
                  Conclusion from this was that GetFinalPathNameByHandleW is supported and does work, but only with some of the allowed flags.

                  So "VOLUME_NAME_NT​" and "VOLUME_NAME_NONE​" both work. But "VOLUME_NAME_DOS"​ does not work.

                  So for any code directly using the Windows API, using VOLUME_NAME_NONE​ should be an easy work-around.

                  Seems like Python might be hard coded to use the "VOLUME_NAME_DOS" flag however
                  It also seems like this started happening in Python V3.8. In V3.7 and below it was alias for os.path.abspath and didn't call that Windows API.

                  So yes, it seems there is a problem with the VOLUME_NAME_DOS flag. Likely because we by-pass the Windows Volume Mount Manager​.

                  And after a bit more searching it seems this problem has been documented 5 years ago.

                  So I don't think this is a easy fix. If enough people encounter the problem we can take a look at it (or someone funds the engineering time).
                  Attached Files

                  Comment


                  • #10
                    Thank you David for clarification. Understood.

                    Comment

                    Working...
                    X