1

I have a generic 360 Camera, and when I convert its fisheye 235° output into equirectangular using FFMpeg, I don't get good results, the image appears squeezed at the bottom.

How could I determine the exact transformation needed to properly convert my images into equirectangular? FFMpeg has a "remap" filter, which uses 2 .pgm files for remapping... but how do I crate such files for my camera?

There is also lensfun and lenscorrection , but question is the same: how do I tune them for my camera?

I am using the generic filter v360 with these parameters:

ffmpeg -i input.jpg -vf v360=fisheye:e:ih_fov=235:iv_fov=235 -y output.jpg

These are 2 croppings on a detail of the equirectangular output:

cropped frames

Look at how much the car is squeezed!

This also cause 360 video stabilization to fail due to too much distortion.

Some explanatory images:

Distortion of my camera highlighting by shooting a chessboard:

distorted chessboard

Probably I should instead get something like this, i.e. squares getting bigger and bigger as they come closer:

chessboard2

The target:

chessboard

Additional images:

FOVs

Pitch, Yaw, FOV

jumpjack
  • 272
  • 1
  • 2
  • 9

1 Answers1

0

I'm a bit confused by the "fisheye 235º output" part of your question. Most 360º cameras give you either a 360ºx180º stitched pano, or two pre-stitch fisheye images. And fisheyes that goes much wider than 180º need a pretty funky design.

Regardless, if you only have 235º horizontal scene coverage and/or less than 180º vertical coverage (able to look straight up/down), then you cannot stretch the image to fill an equirectangular projection without warping the image, because you're missing scene coverage.

inkista
  • 52,065
  • 10
  • 88
  • 161
  • Yes, since some years they started to arrive on the market "super-fisheye" cameras with FOV greater than 180: initially 210, than 220, the one I purchased covers 235°, but then I also read of a 256° ( http://www.8klens.com/cn/product/Fisheye-Lens-16MP-Panoramic-CMOS-Image-Sensor-for-GoPro-Hero-3plus-4-Xiaomi-Yi-4K-Replaced.html ), and now you talk me about this impressive 270° lens! Look for "elenco obiettivi" in this article of mines for link to super-fisheye lens available for purchase: https://jumpjack.wordpress.com/ – jumpjack Aug 12 '21 at 08:05
  • But probably this is exactly the problem: maybe current fisheye-to-rect algorithms only work properly for FOV<180 – jumpjack Aug 12 '21 at 08:06
  • @jumpjack, but an equirectangular mapping is a 360ºx180º FoV: the entire sphere. There is no possible fisheye design that can encompass that in a single shot. You still have to stitch multiple shots to get something that wouldn't require warping or black areas of no image with an equirectangular without warping. Equirectangular is not rectilinear. – inkista Aug 12 '21 at 22:04
  • It's a very complec topic, I am trying to write an article about it, but the most difficult part is creating all the needed images to explain everything... See this image: https://i.imgur.com/RH3xD1G.png Covered Yaw/Pitch is different than FOV. And we are also talking about 3d angles, so also "degrees" are wrong, we should talk about "steradians" – jumpjack Aug 13 '21 at 08:36
  • I found another method to represent this "weird world": https://i.imgur.com/afzWpyh.png – jumpjack Aug 13 '21 at 10:37
  • @jumpjack And we are also talking about 3d angles, so also "degrees" are wrong, we should talk about "steradians". Steradians aren't necessary, unless the metric of interest is areal measurement that can't be approximated as planar. The geometry of the optical system is such that radial symmetry about the optical axis is assumed, so "clock angle" direction (when looking along the optical axis), plus angular deflection off-axis, are much more convenient and useful. And both of those are plane angle measurements. – scottbb Aug 13 '21 at 14:30
  • @scottbb But we were talking about coverage of whole sphere, and inkista supposed that 360°x235° is not possible because a sphere covers 360°x180°... Actually a sphere covers 4pi steradians, so probably* 360x235 translates to 2pi X 1.3 pi – jumpjack Aug 13 '21 at 15:33
  • Sphere/semisphere generator: https://threejs.org/docs/#api/en/geometries/SphereGeometry – jumpjack Aug 13 '21 at 16:01
  • @jumpjack I get what you're saying, but when we talk about projection formulas, we're talking about functions of rays entering a lens (or point of view) with respect to plane angles relative to the optical axis. Steradians are really only useful to talk about with respect to scalar quantities such as power per unit solid angle (that is, nondirectional differentials) and the like. – scottbb Aug 13 '21 at 17:10