Finding the best acquisition parameters (D415)

[GitwellAnyohub]

Camera Model: D415
Firmware: librealsense version 2.9.1
OS: Linux (Ubuntu 16.04)

I'm currently trying to image a mouse in a bucket and I want to know which acquisition parameters I should play with to get the best depth resolution. The working distance is between 20 and 30 centimeters. None of the preset acquisition parameters suggested on the wiki page (/~https://github.com/IntelRealSense/librealsense/wiki/D400-Series-Visual-Presets) are suitable for my project and I don't have a good intuition on what to play with (partly because I don't know what most of the 100+ parameters mean). Does anyone have any suggestions?

Is there a place where I can find a description of each of the parameters and what they do? Which parameters should I change to change the working distance and get the best depth resolution?

[dorodnic]

Hi @GitwellAnyohub
For short-range applications one frequently overlooked parameter is the Disparity Shift (under Advanced Controls > Depth Table)
When used correctly, it allows you to see objects extremely close to the camera (15-20cm) at expense of the background.

[GitwellAnyohub]

@dorodnic thanks for your reply. In one set of the parameter sets I tried I did actually play with the disparity shift. I know that disparity accuracy is a relevant parameter for calculating depth resolution of stereo cameras ( depth resolution is approximately (distance^2 x disparity accuracy) / (focal length x distance between stereo cameras). So at a fixed distance, disparity accuracy seems to be the thing to change although I'm actually sure how disparity shift is related to disparity accuracy. Below, I'll post a video that uses the best set of parameters I have so far. One other thing I'm worried about is the "waviness" of the mouse body. It sort of looks like fat rolls. Is there a way to get rid of this artifact?

[agrunnet]

This is pretty awesome!
To get the best result, you the highest resolution and use the disparity shift to reduce the minimum operating distance (minZ).

The disparity shift parameter does not diminish the accuracy, it just shifts the search range from 0-126 disparities (which is infinity-to-minZ) to nearer range. As @dorodnic mentioned this should be used in particular when you dont have any objects that are farther away. Your example is a great usage. You have your camera pointed at a table with mice. If the camera is mounted 50cm above the table, then there will not be any distance longer than 50cm.

Returning to your question of depth accuracy, I suggest

1. Use 1280x720 for D415 and 848x480 for D435.
2. Explore using the post-processing in LibRS 2.9.1+. It has option to enhance depth by down-sampling, using temporal filter, and applying edge-preserving filters. If you have lots of light, operate at higher frame rate and use the temporal averaging.
3. Experiment with the different depth presets
4. You get better depth the closer you are to the object. Yes, the error scales as Z*Z. Use disparity shift to get even closer. Note however that if you use this, then the operating range goes from minZ-to-infinity, to minZnew-toMaxZnew.

[agrunnet]

One more comment - when you actually get really close you RMS error will go to less than 1mm. However, by default the "depth unit" is 1=1000um (1mm). You can change that under the advanced mode to be, for example, 1=100um. See the screenshot above. You now just need to make sure in software that you interpret your output depth as if each value is 100um, and not 1mm.

There will be a limit to how close you can get with the D415 and D435. Basically the lenses are focused at hyperfocal 50cm, but have good focused images to closer to 20cm. Closer than this and the image will blur, causing a degradation of depth. Furthermore, when you get this close with such a large baseline (distance between imagers), stereo triangulation starts to break down - If you are looking at an object from opposite sides for the Left and Right imagers then it will of course not find a match.

[GitwellAnyohub]

Thanks @argunnet and thanks again @dorodnic. I think I get the gist on what parameters to change to increase the depth resolution. Unfortunately there are a few other issues that I'm not sure now to tackle (not sure if I should start a new thread).

1. There seems to be ripples or waves that occur at the edge of objects (like the walls or the body of the mouse). This is the artifact that makes it seem like the mouse has constant "fat" rolls.

2. When the mouse approaches the edge of the arena, there is an artificial merging effect. Part of the mouse becomes indistinguishable from the wall.

I used to work with the xbox kinect2 cameras before deciding to switch to Intel, and many of these artifacts are new. I understand working with new hardware requires optimization and I appreciate any tips I can get (from anyone!) I've attached some videos that illustrate the problems I'm having.

[agrunnet]

More nice videos.
To reduce the merging of depth when it approaches the wall, please try either the “high accuracy” or “hand” depth preset.
To reduce ripples use the highest depth resolution of the sensor, and use the post processing edge-preserving filter.

[GitwellAnyohub]

@agrunnet
These are fantastic suggestions! I will try them as soon as possible and post back the results. I actually tried all of the presets before I was good at getting high depth resolution. The presets by themselves give horrible depth resolution at my working distance. However, if I start with the presets and change the parameters related to working distance and depth resolution (that I just learned about) I should have better luck.

[dorodnic]

I have added link to this thread to the preset Wiki as an immediate follow-up. @GitwellAnyohub, please feel free share more of your findings.

[amaanda]

@GitwellAnyohub I know this is an old issue, but I wonder if you could share your final .json file. I have a similar application, but using D435. Thanks!

[zhaihuiying]

I see that the depth resolution you adjusted is really great. Now I am also using D415, I'm faced with the problem of getting high depth resolution. Could you please share your acquisition parameters with me ? Iwould be very grateful.

[666tua]

@dorodnic感谢您的回复。在我尝试过的一组参数集中，我确实使用了视差变化。我知道视差精度是计算立体相机深度分辨率的相关参数（深度分辨率约为（距离^ 2 x视差精度）/（焦距x立体相机之间的距离）。所以在固定距离处，视差精度似乎是要改变的事情，尽管我实际上确定视差变化与视差准确性之间的关系。下面，我将发布一个使用迄今为止我拥有的最佳参数集的视频。我担心的另一件事是鼠标身体的“波纹”，有点像肥肉卷，有没有办法去掉这个神器？

Hello！I'm also currently trying to image a mouse in a bucket.Could you share the collection parameters you used? I placed my RealSense D455 about 1 meter above the bottom of the bucket to collect.

[666tua]

When the mice stood up against the wall of the tube,part of their body became the same color as the wall. How do you track mouse profiles in this case?