# OpenVR Library Modifications for Raw IMU Data Access

This document outlines the necessary modifications to the OpenVR library to enable continuous access to raw IMU data, even when optical tracking is lost.

## Problem Statement

The current OpenVR API only provides IMU data through the `TrackedDevicePose_t` structure, which becomes invalid when optical tracking is lost. This means applications cannot access raw IMU data during tracking loss, which is problematic for applications that need continuous IMU data.

## Solution Overview

After examining the OpenVR codebase at "C:\Users\decid\Documents\projects\openvr-resilient", we've identified that OpenVR already has structures for IMU data (`ImuSample_t`) but doesn't expose a direct API for accessing this data continuously. We need to modify the OpenVR library to:

1. Expose a new interface for accessing raw IMU data
2. Implement this interface to provide access to the IMU data regardless of tracking state

## Detailed Modifications

### 1. Add a New Interface for IMU Data Access

In `headers/openvr.h`, add a new interface definition and version string:

```cpp
static const char* const IVRRawIMUData_Version = "IVRRawIMUData_001";

/** Interface for accessing raw IMU data */
class IVRRawIMUData
{
public:
    /** Get raw gyroscope data in radians per second */
    virtual bool GetRawGyroscopeData(float* x, float* y, float* z) = 0;
    
    /** Get raw accelerometer data in meters per second squared */
    virtual bool GetRawAccelerometerData(float* x, float* y, float* z) = 0;
    
    /** Get the timestamp of the last IMU sample in seconds */
    virtual bool GetIMUSampleTimestamp(double* timestamp) = 0;
    
    /** Get a full IMU sample containing both gyroscope and accelerometer data */
    virtual bool GetIMUSample(vr::ImuSample_t* pSample) = 0;
};
```

### 2. Implement the Interface in the OpenVR Runtime

In `src/vrcore/vrcore_impl.cpp` (or an appropriate implementation file), add the implementation of the new interface:

```cpp
class CVRRawIMUData : public IVRRawIMUData
{
public:
    CVRRawIMUData()
    {
        // Initialize any necessary resources
    }
    
    virtual ~CVRRawIMUData()
    {
        // Clean up resources
    }
    
    virtual bool GetRawGyroscopeData(float* x, float* y, float* z) override
    {
        // Get the HMD device
        vr::TrackedDeviceIndex_t hmdIndex = vr::k_unTrackedDeviceIndex_Hmd;
        
        // Check if the HMD is connected
        if (!vr::VRSystem()->IsTrackedDeviceConnected(hmdIndex))
            return false;
        
        // Try to get the latest IMU sample
        vr::ImuSample_t sample;
        if (GetIMUSample(&sample))
        {
            // Convert from double to float
            *x = (float)sample.vGyro.v[0];
            *y = (float)sample.vGyro.v[1];
            *z = (float)sample.vGyro.v[2];
            return true;
        }
        
        return false;
    }
    
    virtual bool GetRawAccelerometerData(float* x, float* y, float* z) override
    {
        // Get the HMD device
        vr::TrackedDeviceIndex_t hmdIndex = vr::k_unTrackedDeviceIndex_Hmd;
        
        // Check if the HMD is connected
        if (!vr::VRSystem()->IsTrackedDeviceConnected(hmdIndex))
            return false;
        
        // Try to get the latest IMU sample
        vr::ImuSample_t sample;
        if (GetIMUSample(&sample))
        {
            // Convert from double to float
            *x = (float)sample.vAccel.v[0];
            *y = (float)sample.vAccel.v[1];
            *z = (float)sample.vAccel.v[2];
            return true;
        }
        
        return false;
    }
    
    virtual bool GetIMUSampleTimestamp(double* timestamp) override
    {
        // Get the HMD device
        vr::TrackedDeviceIndex_t hmdIndex = vr::k_unTrackedDeviceIndex_Hmd;
        
        // Check if the HMD is connected
        if (!vr::VRSystem()->IsTrackedDeviceConnected(hmdIndex))
            return false;
        
        // Try to get the latest IMU sample
        vr::ImuSample_t sample;
        if (GetIMUSample(&sample))
        {
            *timestamp = sample.fSampleTime;
            return true;
        }
        
        return false;
    }
    
    virtual bool GetIMUSample(vr::ImuSample_t* pSample) override
    {
        // This is the key method that needs to be implemented to access the raw IMU data
        // The implementation will depend on the specific driver and hardware
        
        // Get the HMD device
        vr::TrackedDeviceIndex_t hmdIndex = vr::k_unTrackedDeviceIndex_Hmd;
        
        // Check if the HMD is connected
        if (!vr::VRSystem()->IsTrackedDeviceConnected(hmdIndex))
            return false;
        
        // Access the IMU data buffer
        // This is where we need to access the driver's internal IMU data buffer
        // The exact implementation will depend on the specific driver
        
        // For example, we might use the IVRIOBuffer interface to access the IMU data buffer:
        vr::IOBufferHandle_t imuBuffer = vr::k_ulInvalidIOBufferHandle;
        vr::EIOBufferError error = vr::VRIOBuffer()->Open(
            "/dev/imu/hmd", 
            vr::IOBufferMode_Read, 
            sizeof(vr::ImuSample_t), 
            1, 
            &imuBuffer);
        
        if (error != vr::IOBufferError_None || imuBuffer == vr::k_ulInvalidIOBufferHandle)
            return false;
        
        uint32_t bytesRead = 0;
        error = vr::VRIOBuffer()->Read(imuBuffer, pSample, sizeof(vr::ImuSample_t), &bytesRead);
        
        vr::VRIOBuffer()->Close(imuBuffer);
        
        return (error == vr::IOBufferError_None && bytesRead == sizeof(vr::ImuSample_t));
    }
};
```

### 3. Expose the Interface through the OpenVR API

In the appropriate factory method in the OpenVR implementation (likely in `src/vrcore/vrcore_impl.cpp`), add code to create and return the new interface:

```cpp
void* CVRSystem::GetGenericInterface(const char* pchInterfaceVersion, vr::EVRInitError* peError)
{
    // Existing code...
    
    // Check for our new interface
    if (strcmp(pchInterfaceVersion, vr::IVRRawIMUData_Version) == 0)
    {
        if (!m_pRawIMUData)
        {
            m_pRawIMUData = new CVRRawIMUData();
        }
        
        if (peError)
            *peError = vr::VRInitError_None;
            
        return m_pRawIMUData;
    }
    
    // Existing code...
}
```

### 4. Add the Interface Member to the CVRSystem Class

In the appropriate header file (likely `src/vrcore/vrcore_impl.h`), add a member for the new interface:

```cpp
class CVRSystem : public vr::IVRSystem
{
    // Existing code...
    
private:
    // Existing members...
    CVRRawIMUData* m_pRawIMUData;
};
```

## Building the Modified OpenVR Library

1. Clone the OpenVR repository: `git clone https://github.com/ValveSoftware/openvr.git`
2. Make the modifications described above
3. Build the library following the standard build instructions
4. Replace the existing OpenVR library with your modified version

## Using the Modified Library

Once the modified library is in place, you can access the raw IMU data using the following code:

```cpp
#include <openvr.h>

// Get the raw IMU data interface
vr::IVRRawIMUData* pRawIMUData = nullptr;
vr::EVRInitError error = vr::VRInitError_None;
pRawIMUData = (vr::IVRRawIMUData*)vr::VR_GetGenericInterface(vr::IVRRawIMUData_Version, &error);

if (error != vr::VRInitError_None || !pRawIMUData)
{
    // Handle error
    return false;
}

// Get the raw gyroscope data
float gyroX, gyroY, gyroZ;
if (pRawIMUData->GetRawGyroscopeData(&gyroX, &gyroY, &gyroZ))
{
    // Use the gyroscope data
    printf("Gyro: %f, %f, %f\n", gyroX, gyroY, gyroZ);
}

// Get the raw accelerometer data
float accelX, accelY, accelZ;
if (pRawIMUData->GetRawAccelerometerData(&accelX, &accelY, &accelZ))
{
    // Use the accelerometer data
    printf("Accel: %f, %f, %f\n", accelX, accelY, accelZ);
}

// Get a full IMU sample
vr::ImuSample_t sample;
if (pRawIMUData->GetIMUSample(&sample))
{
    // Use the IMU sample
    printf("IMU Sample: Time=%f, Gyro=(%f, %f, %f), Accel=(%f, %f, %f)\n",
        sample.fSampleTime,
        sample.vGyro.v[0], sample.vGyro.v[1], sample.vGyro.v[2],
        sample.vAccel.v[0], sample.vAccel.v[1], sample.vAccel.v[2]);
}
```

## Alternative Approaches

If modifying the OpenVR library is not feasible, consider these alternatives:

### 1. Use the IVRIOBuffer Interface

The OpenVR API already includes an `IVRIOBuffer` interface that might be used to access raw IMU data. This would require knowledge of the specific buffer paths used by the driver to store IMU data.

```cpp
vr::IVRIOBuffer* pIOBuffer = vr::VRIOBuffer();
vr::IOBufferHandle_t imuBuffer = vr::k_ulInvalidIOBufferHandle;
vr::EIOBufferError error = pIOBuffer->Open("/dev/imu/hmd", vr::IOBufferMode_Read, sizeof(vr::ImuSample_t), 1, &imuBuffer);

if (error == vr::IOBufferError_None && imuBuffer != vr::k_ulInvalidIOBufferHandle)
{
    vr::ImuSample_t sample;
    uint32_t bytesRead = 0;
    error = pIOBuffer->Read(imuBuffer, &sample, sizeof(vr::ImuSample_t), &bytesRead);
    
    if (error == vr::IOBufferError_None && bytesRead == sizeof(vr::ImuSample_t))
    {
        // Use the IMU sample
        printf("IMU Sample: Time=%f, Gyro=(%f, %f, %f), Accel=(%f, %f, %f)\n",
            sample.fSampleTime,
            sample.vGyro.v[0], sample.vGyro.v[1], sample.vGyro.v[2],
            sample.vAccel.v[0], sample.vAccel.v[1], sample.vAccel.v[2]);
    }
    
    pIOBuffer->Close(imuBuffer);
}
```

### 2. Create a Custom Driver

Create a custom OpenVR driver that wraps the existing driver and exposes the IMU data through a custom interface.

### 3. Use a Different VR SDK

Consider using a different VR SDK that provides direct access to IMU data, such as the Oculus SDK or the SteamVR Unity Plugin.

## Notes

- The exact implementation details may vary depending on the specific driver you're using (HTC Vive, Oculus, etc.)
- You may need to study the driver code to understand how to access the raw IMU data
- Some drivers may already have internal access to raw IMU data, but don't expose it through the public API