0

I am using Microsoft kinect v1 in MATLAB and want to get depth data on every pixel in meters.

I am not sure how to get that data because I am getting uint16 and as far as I have read is that it provides depth only in 13 bits, So how do I get those 13 bits and do some conversion to get depth exactly in meters.

I have searched a lot about it but could not get to any conclusion.


    Kinectinfo = imaqhwinfo('kinect');
    colorinfo = Kinectinfo.DeviceInfo(1);
    depthinfo = Kinectinfo.DeviceInfo(2);
    colorvid = videoinput('kinect',1);
    depthvid = videoinput('kinect',2);
    srcDepth = getselectedsource(depthvid);

    % Set the frames per trigger for both devices to 1.

    colorvid.FramesPerTrigger = 1;
    depthvid.FramesPerTrigger = 1;

    % Set the trigger repeat for both devices to 200, in order to acquire 201 frames from both the color sensor and the depth sensor.

    colorvid.TriggerRepeat = 200;
    depthvid.TriggerRepeat = 200;
    %Configure the camera for manual triggering for both sensors.

    triggerconfig([colorvid depthvid],'manual');
    % Start both video objects.

    start([colorvid depthvid]);

    %Trigger the devices, then get the acquired data.

    % Trigger 200 times to get the frames.

    for i = 1:200
        % Trigger both objects.
        trigger([colorvid depthvid])
        % Get the acquired frames and metadata.
        [imgColor, ts_color, metaData_Color] = getdata(colorvid);
        [imgDepth, ts_depth, metaData_Depth] = getdata(depthvid);
    end


[NYU Depth and RGB image][1]
[Histogram of swaped Raw Depth image][2]
[Histogram of Raw Depth Image][3]

I would like to have some code for conversion or any SDK that provides me with meters in matlab.

Thanks alot.

abdur rahman
  • 11
  • 4

1 Answers1

0
  1. According to several articles, eg. such as here and here, the first three bits are code for the player that the device has identified, but this is only activated if skeleton tracking is enabled. If you don't enable skeleton tracking, the three bits are set to zero.

Kinect v1 16 bit depth data

  1. If you are using matlab, the depth images are 16bit double images, and contain data that is already extracted from the 13bits (most probably because skeleton tracking is already disabled, hence 3 MSB are zero.) Therefore you dont need to convert/extract the 13 bit data in matlab or libfreenect etc becuase the bits be already zero.

  2. According to the Matlab help page, matlab ImAq toolbox requires the installation of Kinect for Windows Runtime; because it uses the Kinect SDK Kinect drivers to acquire data. And according to this SO answer, "using Microsoft SDK, so the values that are returned from kinect sensor are already real distances in mm."

Therefore, you dont really need to bitshift data in matlab. The only reason you would need to do it, would be if you are getting raw depth stream from the kinect drivers (?) in C++/C# and the 16bit data would include the 3 LSB bits.

Atif Anwer
  • 450
  • 5
  • 13
  • Thanks very much for very elaborate answer, Actually I have to use NYU Matlab toolbox [here](https://cs.nyu.edu/~silberman/datasets/nyu_depth_v2.html) for filling the raw depths on images I am getting from raw dataset and further processing. – abdur rahman May 30 '19 at 03:10
  • Actually I have to align my dataset according to theirs so that I use the same toolbox, I do not want to mess with the toolbox because it is working perfect on their raw dataset. Before further processing it they call [swapbytes.m](https://www.mathworks.com/help/matlab/ref/swapbytes.html) function to convert it to 11 bits I guess because their maximum value is 2047. – abdur rahman May 30 '19 at 03:18
  • I have two options right now either use 16 bits of information and use swapbytes as they did or I should take only 11 bits of data. – abdur rahman May 30 '19 at 03:19
  • I have used libfreenect and reading this I came to know that there is some option [available](https://stackoverflow.com/questions/12819599/freenect-depth-registered-has-no-effect-with-libfreenect) but I do not know how to call it. – abdur rahman May 30 '19 at 03:21
  • Have you tried using both the types of images (16 bit and 11bit using swapbytes) to the toolbox? I think the 16bit should work directly, without needing the 11bit, as per my answer. I cannot check the toolbox due to time constraints and lack of kinect with me. As for librenect, the answer you linked is 6 years old, so it might have been changed now. unfortunately im not well versed with it so i cannot comment on that. But i recommend testing/experimenting with the 16bit double images... hopefully they should work. – Atif Anwer May 31 '19 at 04:19