ROVI-POLAR · WHITE PAPER

Polarization Camera White Paper

What a polarization camera sees, how its sensor is built, and how polarization reveals the surface tilt and depth of clear and shiny objects — explained with diagrams and real footage.

1. Summary

Besides brightness and colour, light carries another piece of information: the direction it vibrates in (polarization). Our eyes and ordinary cameras throw it away, but a polarization camera puts four polarizing filters over its pixels and records it in a single shot. When light reflects off a surface, its polarization changes with the tilt and material of that surface, so reading polarization reveals the shape of objects that other 3D sensors miss because they are clear or shiny.

RGB cameraPolarization camera
RecordsBrightness and colourBrightness and colour + strength (DoLP) and direction (AoLP) of polarization
Filters over pixelsColour filtersColour filters + 0° · 45° · 90° · 135° polarizers
Clear or glossy objectsBackground shows through, or only glarePolarization of reflected light reveals outline and tilt
ReflectionsCaptured as they areRemoving the polarized part also gives a reflection-reduced image
ResolutionFull pixels4 × 4 pixels per unit (5 MP sensor → 612 × 512 aligned image)

In one line: RGB tells you *how it looks*; polarization tells you *which way the surface faces*. ROVI-Polar is being developed to recover the depth and 6D pose of clear plastic parts from it.

2. Why clear and shiny objects are hard

Most industrial 3D sensors measure distance from light they send out and get back. Clear or mirror-like surfaces break that assumption.

MethodWhat happens on clear or glossy surfaces
ToF (time of flight)Light passes through to the background behind, or bounces off in another direction — depth goes missing or wrong
Active stereo, structured lightThe projected pattern does not land on the clear object or is distorted by reflections, so no matches are found
Depth from a single RGB image (AI)Mistakes the background seen through the object for the object — δ1 0.547 in transparent regions (reference)
6D pose modelsFail along with the depth — public models also state they do not work on transparent or glassy objects

So parts whose depth breaks, such as clear plastic parts, are hard to pick with the existing 3D vision pipeline. Polarization is information that actually gets stronger on such surfaces.

3. What polarization is

Light is a wave that vibrates across its direction of travel. Sunlight and lamp light vibrate in every direction — unpolarized. After a polarizing filter, light vibrates in one direction only — linearly polarized.

Reflection off a smooth surface also polarizes light. The component vibrating parallel to the surface reflects more strongly, so reflected light is partly polarized. How strongly (degree) and in which direction (angle) are set by the viewing angle, the direction the surface is tilted, and the material's refractive index — this is the Fresnel equations.

natural light — vibrates in every direction (unpolarized) polarizing filter vibrates in one direction (linearly polarized) clear or glossy surface reflected light vibrates more parallel to the surface (⊙, out of the page) → partly polarized as the surface tilts, the strength (DoLP) and direction (AoLP) change
Unpolarized light → polarizing filter → linearly polarized light (top). Light reflected from a surface becomes partly polarized, vibrating more parallel to the surface (⊙), with a strength and direction that depend on how the surface is tilted (bottom).

4. How a polarization camera is built

A polarization camera (the Sony IMX264MYR Polarsens used in ROVI-Polar) has very fine metal wire-grid polarizers built directly over its pixels. Each 2 × 2 group of neighbouring pixels has one filter at each of 0°, 45°, 90° and 135°, and in the colour sensor each such group sits under one R, G, G or B colour filter, so 4 × 4 pixels make one computing unit.

90° 45° 135° 0° 90° 45° 135° 0° 90° 45° 135° 0° 90° 45° 135° 0° 4 × 4 pixels = 1 unit 0° · 45° · 90° · 135° polarizers on every 2 × 2 pixels R · G · G · B colour filter on every 2 × 2 block computed from the four brightnesses S0 = brightness · S1 = I0 − I90 · S2 = I45 − I135 DoLP = √(S1² + S2²) / S0 · AoLP = ½ atan2(S2, S1) RGB colour image DoLP degree 0–1 AoLP angle 0–180°
Pixel layout of a colour polarization sensor — four polarizer directions on every 2 × 2 pixels (arrows show the direction light passes), and a colour filter on every 2 × 2 group. Stokes values computed from the four brightnesses give RGB, DoLP and AoLP.

From the brightness of the four directions, I0, I45, I90 and I135:

ValueComputed asMeaning
S0(I0 + I45 + I90 + I135) / 2Total brightness — the ordinary image
S1, S2I0 − I90, I45 − I135Horizontal and diagonal parts of the polarization
DoLP (degree)√(S1² + S2²) / S00 = unpolarized, 1 = fully linearly polarized
AoLP (angle)½ · atan2(S2, S1)Direction of polarization, 0–180°
I_min (reflection suppression)(S0 − √(S1² + S2²)) / 2Brightness with the polarized part (mostly reflections) removed

5. Reading surface tilt from polarization

For light reflected from a clear plastic (refractive index about 1.5), the degree of polarization (DoLP) follows a fixed curve against the viewing angle θ.

0° 30° 60° 90° 0.5 1.0 degree of polarization DoLP viewing angle θ (from the surface normal) Brewster angle ≈ 56° specular (reflected off the surface) diffuse (came out through the material) surface normal θ measure DoLP → solve back for the angle θ; AoLP → the direction of tilt
Degree of polarization against angle for a clear plastic (index 1.5) — orange: light reflected off the surface (peaks at the Brewster angle, about 56°); blue: light that came out through the material. As the surface on the right tilts, the viewing angle θ grows and DoLP changes.
  1. DoLP → tilt angle (zenith): putting the measured DoLP into the curve gives how far the surface is tilted from the camera (θ). The specular curve has two solutions either side of the Brewster angle; neighbouring information picks one.
  2. AoLP → tilt direction (azimuth): the direction of polarization tells which way the surface is tilted (its 180° ambiguity is resolved from neighbouring information).
  3. Normal → depth: joining up (integrating) the per-pixel surface directions (normals) gives relative depth.
  4. Refractive index: the target material's index sets the curve — PMMA (acrylic) 1.49, PET 1.57, PC 1.58, PS 1.59. Most clear plastics fall in this range.

PolarDepth (ICML 2026), which ROVI-Polar builds on, combines overall depth from a single RGB image with normals and depth decoded physically from polarization (index, zenith, azimuth) in transparent regions. The paper reports raising δ1 in transparent regions from 0.547 for the RGB-only model to 0.859 (reference). Simply adding polarization as extra image channels reached only 0.556, showing that decoding it as physics is the key (reference).

6. Real footage

FLOYD recorded raw 12-bit footage of a clear phone case held and moved by hand in front of the polarization camera (16 mm lens, about 1 m) and processed it four ways (real speed).

Real polarization camera footage (October 2026) — top left RGB, top right DoLP (degree of polarization, brighter = stronger), bottom left AoLP (direction as colour, stronger where DoLP is higher), bottom right I_min (polarized part removed)
PanelWhat you see
RGBThe background shows through the clear case and its outline is faint
DoLPReflections off the case's rim and curves are strongly polarized, so the outline shows as bright lines
AoLPColour changes with the direction the surface is tilted — the smooth colour change along a curve is the surface-direction information
I_minPolarized reflections removed — brightness with less glare
In dark areas (image edges, shadows) DoLP and AoLP look sandy because there is too little light for a stable polarization estimate; processing uses only areas that are bright enough. Light sources that are already polarized, such as LCD monitors, also show high DoLP, so they are avoided or filtered out.

7. ROVI-Polar processing

Raw polarization 12-bit · 5 MP one exposure 4 angles 0 · 45 · 90 · 135° Stokes RGB · DoLP · AoLP 612 × 512 pixel-aligned Normal · depth index · angle decoding transparent mask 6D pose ROVI-3D + ICP CAD based Polarization is decoded as physics, not used as "just another image channel" — the recovered depth feeds the existing 6D pose engine
ROVI-Polar processing — raw polarization image → four angles and Stokes → RGB, DoLP, AoLP → normal and depth recovery → 6D pose
  1. Capture: one polarization camera takes a raw 12-bit image in a single shot.
  2. Polarization: split into four directions, compute Stokes values, and produce pixel-aligned RGB, DoLP, AoLP and I_min.
  3. Normal and depth recovery: decode polarization into refractive index, zenith and azimuth to get normals, and combine with RGB-based depth to recover depth in transparent regions, along with a transparent-region mask.
  4. 6D pose: the recovered depth, mask and RGB plus the part's CAD go to AI ToolBox's existing 6D pose engine (ROVI-3D + ICP) for position and orientation — data conventions and the engine are reused unchanged.

Targets

ItemValueType
6D pose error≤ 5 mm / ≤ 3° @ 1 mTarget
Transparent-region depth δ1≥ 0.75Target
Processing speed≥ 5 fps (depth alone ≥ 20 fps)Target
Transparent-region depth δ1 (paper)0.859 — RGB-only depth model 0.547Reference
Depth inference time (paper)25 msReference

8. Strengths and limits

Strengths

StrengthWhy
Strong on clear and glossy surfacesReflections that disturb other 3D sensors are signal for a polarization camera
One camera, one shotRGB and polarization aligned pixel for pixel — no calibration between sensors
Passive sensorProjects no pattern or laser — no interference with other sensors or people
Reflection-reduced imageRemoving the polarized part helps see through glass and film
Same form as industrial camerasSame size, lens (C-mount) and interface (GigE)

Limits

LimitDetailsWhat to do
Resolution and light4 × 4 pixels per unit; polarizers cut incoming light, so exposures get longerChoose lens magnification, diffuse lighting
Material assumptionBased on clear materials with index about 1.3–1.6 — metals polarize differentlySeparate model for metal reflectors (future)
Weakly polarizing materials and anglesSurfaces seen face-on and rough surfaces polarize weaklyLimit the effect with the transparent-region mask, lighting layout
Stress birefringenceInternal stress in injection-moulded PC or PET changes the polarization of transmitted lightMeasure its extent, decode mainly the reflected component
Polarized lightingLCD monitors, polarized lamps and strong direct sunlight blur the resultUnpolarized diffuse lighting, capture guide
Development stageDepth recovery and pose integration are in developmentRoadmap (chapter 10)

9. Where to use it

Selection guide

◎ best · ○ possible · △ limited · ✕ unsuitable

TaskPolarizationToF / structured lightRGBWhy
Pose and picking of clear plastic or glass parts◎△△Tilt read from polarized reflections off the clear surface (ROVI-Polar)
Reading through glass or film○✕△Reflection-reduced image
Scratches and dents on glossy surfaces○△○Changes in surface direction show up in AoLP
Checking stress in clear parts○✕✕Stress birefringence changes polarization
3D of opaque, matte objects△◎○ToF and structured light are more accurate here
Colour and text inspection△✕◎Colour is RGB territory

FLOYD's application

ROVI-Polar is being developed first for picking injection-moulded clear plastic parts (PC, PMMA, PET), with inspection of clear and reflective objects to follow.

10. Camera and ROVI-Polar

LUCID Triton 5.0 MP Polarization camera

ItemDetails (spec)
ModelTriton TRI050S1-QC (colour polarization)
SensorSony IMX264MYR (Polarsens) · 2/3" · global shutter
Resolution and pixel2448 × 2048 (5.0 MP) · 3.45 µm
Polarizers0° · 45° · 90° · 135° (per 2 × 2 pixels), colour per 2 × 2 group
Speed and bit depthUp to 24 fps · 12-bit
InterfaceGigE Vision · PoE or 12–24 V
Lens and sizeC-mount · 29 × 29 × 45 mm

ROVI-Polar

ItemDetails
CameraLUCID Triton TRI050S1-QC (colour polarization · 5 MP)
Lens and distance16 mm C-mount · 1 m working distance (528 × 442 mm field of view)
ProcessingNVIDIA GPU PC (RTX 4080 class)
OutputRGB, DoLP, AoLP, reflection-reduced image → depth and transparent-region mask → 6D pose
MaturityRoadmap — capture and processing tools done, real-camera recording started; polarization depth recovery and pose integration in development (2026)

Send application review requests through floyd.kr contact.

11. Glossary

TermMeaning
PolarizationThe direction light vibrates in. Unpolarized = every direction, linearly polarized = one direction
DoLPDegree of Linear Polarization. 0 (unpolarized) to 1 (fully polarized)
AoLPAngle of Linear Polarization. Direction of polarization, 0–180°
Stokes values (S0, S1, S2)A description of the polarization state computed from the four brightnesses
I_minBrightness with the polarized part removed — an image with less glare
Refractive index (η)How much light slows inside a material. About 1.5 for clear plastics
Brewster angleThe angle at which reflected light is fully polarized. About 56° for index 1.5
Zenith, azimuthHow far the surface normal tilts from the camera axis, and in which direction
NormalThe direction perpendicular to a surface
Stress birefringencePolarization of light changed by stress inside a material
δ1A depth accuracy measure: share of pixels within 25 % of the true depth
6D poseAn object's position (3) + orientation (3)