Polarization Camera White Paper
What a polarization camera sees, how its sensor is built, and how polarization reveals the surface tilt and depth of clear and shiny objects — explained with diagrams and real footage.
1. Summary
Besides brightness and colour, light carries another piece of information: the direction it vibrates in (polarization). Our eyes and ordinary cameras throw it away, but a polarization camera puts four polarizing filters over its pixels and records it in a single shot. When light reflects off a surface, its polarization changes with the tilt and material of that surface, so reading polarization reveals the shape of objects that other 3D sensors miss because they are clear or shiny.
| RGB camera | Polarization camera | |
|---|---|---|
| Records | Brightness and colour | Brightness and colour + strength (DoLP) and direction (AoLP) of polarization |
| Filters over pixels | Colour filters | Colour filters + 0° · 45° · 90° · 135° polarizers |
| Clear or glossy objects | Background shows through, or only glare | Polarization of reflected light reveals outline and tilt |
| Reflections | Captured as they are | Removing the polarized part also gives a reflection-reduced image |
| Resolution | Full pixels | 4 × 4 pixels per unit (5 MP sensor → 612 × 512 aligned image) |
In one line: RGB tells you *how it looks*; polarization tells you *which way the surface faces*. ROVI-Polar is being developed to recover the depth and 6D pose of clear plastic parts from it.
2. Why clear and shiny objects are hard
Most industrial 3D sensors measure distance from light they send out and get back. Clear or mirror-like surfaces break that assumption.
| Method | What happens on clear or glossy surfaces |
|---|---|
| ToF (time of flight) | Light passes through to the background behind, or bounces off in another direction — depth goes missing or wrong |
| Active stereo, structured light | The projected pattern does not land on the clear object or is distorted by reflections, so no matches are found |
| Depth from a single RGB image (AI) | Mistakes the background seen through the object for the object — δ1 0.547 in transparent regions (reference) |
| 6D pose models | Fail along with the depth — public models also state they do not work on transparent or glassy objects |
So parts whose depth breaks, such as clear plastic parts, are hard to pick with the existing 3D vision pipeline. Polarization is information that actually gets stronger on such surfaces.
3. What polarization is
Light is a wave that vibrates across its direction of travel. Sunlight and lamp light vibrate in every direction — unpolarized. After a polarizing filter, light vibrates in one direction only — linearly polarized.
Reflection off a smooth surface also polarizes light. The component vibrating parallel to the surface reflects more strongly, so reflected light is partly polarized. How strongly (degree) and in which direction (angle) are set by the viewing angle, the direction the surface is tilted, and the material's refractive index — this is the Fresnel equations.
- Polarized sunglasses for fishing or driving cut glare from water and glass on the same principle — the glare is polarized in one direction.
- A polarization camera measures this polarization at every pixel and turns it into an image.
4. How a polarization camera is built
A polarization camera (the Sony IMX264MYR Polarsens used in ROVI-Polar) has very fine metal wire-grid polarizers built directly over its pixels. Each 2 × 2 group of neighbouring pixels has one filter at each of 0°, 45°, 90° and 135°, and in the colour sensor each such group sits under one R, G, G or B colour filter, so 4 × 4 pixels make one computing unit.
From the brightness of the four directions, I0, I45, I90 and I135:
| Value | Computed as | Meaning |
|---|---|---|
| S0 | (I0 + I45 + I90 + I135) / 2 | Total brightness — the ordinary image |
| S1, S2 | I0 − I90, I45 − I135 | Horizontal and diagonal parts of the polarization |
| DoLP (degree) | √(S1² + S2²) / S0 | 0 = unpolarized, 1 = fully linearly polarized |
| AoLP (angle) | ½ · atan2(S2, S1) | Direction of polarization, 0–180° |
| I_min (reflection suppression) | (S0 − √(S1² + S2²)) / 2 | Brightness with the polarized part (mostly reflections) removed |
- One shot, pixel-aligned: all four directions are captured at the same instant through the same lens, so RGB, DoLP and AoLP line up pixel for pixel. No calibration between several cameras is needed.
- Resolution: using 4 × 4 pixels as one unit, the 5 MP (2448 × 2048) sensor gives a fully aligned 612 × 512 image; interpolation can give 1224 × 1024.
- Raw 12-bit: the camera can compute 8-bit DoLP and AoLP on board, but ROVI-Polar takes the raw 12-bit image and computes them on the PC for precision.
5. Reading surface tilt from polarization
For light reflected from a clear plastic (refractive index about 1.5), the degree of polarization (DoLP) follows a fixed curve against the viewing angle θ.
- DoLP → tilt angle (zenith): putting the measured DoLP into the curve gives how far the surface is tilted from the camera (θ). The specular curve has two solutions either side of the Brewster angle; neighbouring information picks one.
- AoLP → tilt direction (azimuth): the direction of polarization tells which way the surface is tilted (its 180° ambiguity is resolved from neighbouring information).
- Normal → depth: joining up (integrating) the per-pixel surface directions (normals) gives relative depth.
- Refractive index: the target material's index sets the curve — PMMA (acrylic) 1.49, PET 1.57, PC 1.58, PS 1.59. Most clear plastics fall in this range.
PolarDepth (ICML 2026), which ROVI-Polar builds on, combines overall depth from a single RGB image with normals and depth decoded physically from polarization (index, zenith, azimuth) in transparent regions. The paper reports raising δ1 in transparent regions from 0.547 for the RGB-only model to 0.859 (reference). Simply adding polarization as extra image channels reached only 0.556, showing that decoding it as physics is the key (reference).
6. Real footage
FLOYD recorded raw 12-bit footage of a clear phone case held and moved by hand in front of the polarization camera (16 mm lens, about 1 m) and processed it four ways (real speed).
| Panel | What you see |
|---|---|
| RGB | The background shows through the clear case and its outline is faint |
| DoLP | Reflections off the case's rim and curves are strongly polarized, so the outline shows as bright lines |
| AoLP | Colour changes with the direction the surface is tilted — the smooth colour change along a curve is the surface-direction information |
| I_min | Polarized reflections removed — brightness with less glare |
7. ROVI-Polar processing
- Capture: one polarization camera takes a raw 12-bit image in a single shot.
- Polarization: split into four directions, compute Stokes values, and produce pixel-aligned RGB, DoLP, AoLP and I_min.
- Normal and depth recovery: decode polarization into refractive index, zenith and azimuth to get normals, and combine with RGB-based depth to recover depth in transparent regions, along with a transparent-region mask.
- 6D pose: the recovered depth, mask and RGB plus the part's CAD go to AI ToolBox's existing 6D pose engine (ROVI-3D + ICP) for position and orientation — data conventions and the engine are reused unchanged.
Targets
| Item | Value | Type |
|---|---|---|
| 6D pose error | ≤ 5 mm / ≤ 3° @ 1 m | Target |
| Transparent-region depth δ1 | ≥ 0.75 | Target |
| Processing speed | ≥ 5 fps (depth alone ≥ 20 fps) | Target |
| Transparent-region depth δ1 (paper) | 0.859 — RGB-only depth model 0.547 | Reference |
| Depth inference time (paper) | 25 ms | Reference |
8. Strengths and limits
Strengths
| Strength | Why |
|---|---|
| Strong on clear and glossy surfaces | Reflections that disturb other 3D sensors are signal for a polarization camera |
| One camera, one shot | RGB and polarization aligned pixel for pixel — no calibration between sensors |
| Passive sensor | Projects no pattern or laser — no interference with other sensors or people |
| Reflection-reduced image | Removing the polarized part helps see through glass and film |
| Same form as industrial cameras | Same size, lens (C-mount) and interface (GigE) |
Limits
| Limit | Details | What to do |
|---|---|---|
| Resolution and light | 4 × 4 pixels per unit; polarizers cut incoming light, so exposures get longer | Choose lens magnification, diffuse lighting |
| Material assumption | Based on clear materials with index about 1.3–1.6 — metals polarize differently | Separate model for metal reflectors (future) |
| Weakly polarizing materials and angles | Surfaces seen face-on and rough surfaces polarize weakly | Limit the effect with the transparent-region mask, lighting layout |
| Stress birefringence | Internal stress in injection-moulded PC or PET changes the polarization of transmitted light | Measure its extent, decode mainly the reflected component |
| Polarized lighting | LCD monitors, polarized lamps and strong direct sunlight blur the result | Unpolarized diffuse lighting, capture guide |
| Development stage | Depth recovery and pose integration are in development | Roadmap (chapter 10) |
9. Where to use it
Selection guide
◎ best · ○ possible · △ limited · ✕ unsuitable
| Task | Polarization | ToF / structured light | RGB | Why |
|---|---|---|---|---|
| Pose and picking of clear plastic or glass parts | ◎ | △ | △ | Tilt read from polarized reflections off the clear surface (ROVI-Polar) |
| Reading through glass or film | ○ | ✕ | △ | Reflection-reduced image |
| Scratches and dents on glossy surfaces | ○ | △ | ○ | Changes in surface direction show up in AoLP |
| Checking stress in clear parts | ○ | ✕ | ✕ | Stress birefringence changes polarization |
| 3D of opaque, matte objects | △ | ◎ | ○ | ToF and structured light are more accurate here |
| Colour and text inspection | △ | ✕ | ◎ | Colour is RGB territory |
FLOYD's application
ROVI-Polar is being developed first for picking injection-moulded clear plastic parts (PC, PMMA, PET), with inspection of clear and reflective objects to follow.
10. Camera and ROVI-Polar
LUCID Triton 5.0 MP Polarization camera
| Item | Details (spec) |
|---|---|
| Model | Triton TRI050S1-QC (colour polarization) |
| Sensor | Sony IMX264MYR (Polarsens) · 2/3" · global shutter |
| Resolution and pixel | 2448 × 2048 (5.0 MP) · 3.45 µm |
| Polarizers | 0° · 45° · 90° · 135° (per 2 × 2 pixels), colour per 2 × 2 group |
| Speed and bit depth | Up to 24 fps · 12-bit |
| Interface | GigE Vision · PoE or 12–24 V |
| Lens and size | C-mount · 29 × 29 × 45 mm |
ROVI-Polar
| Item | Details |
|---|---|
| Camera | LUCID Triton TRI050S1-QC (colour polarization · 5 MP) |
| Lens and distance | 16 mm C-mount · 1 m working distance (528 × 442 mm field of view) |
| Processing | NVIDIA GPU PC (RTX 4080 class) |
| Output | RGB, DoLP, AoLP, reflection-reduced image → depth and transparent-region mask → 6D pose |
| Maturity | Roadmap — capture and processing tools done, real-camera recording started; polarization depth recovery and pose integration in development (2026) |
Send application review requests through floyd.kr contact.
11. Glossary
| Term | Meaning |
|---|---|
| Polarization | The direction light vibrates in. Unpolarized = every direction, linearly polarized = one direction |
| DoLP | Degree of Linear Polarization. 0 (unpolarized) to 1 (fully polarized) |
| AoLP | Angle of Linear Polarization. Direction of polarization, 0–180° |
| Stokes values (S0, S1, S2) | A description of the polarization state computed from the four brightnesses |
| I_min | Brightness with the polarized part removed — an image with less glare |
| Refractive index (η) | How much light slows inside a material. About 1.5 for clear plastics |
| Brewster angle | The angle at which reflected light is fully polarized. About 56° for index 1.5 |
| Zenith, azimuth | How far the surface normal tilts from the camera axis, and in which direction |
| Normal | The direction perpendicular to a surface |
| Stress birefringence | Polarization of light changed by stress inside a material |
| δ1 | A depth accuracy measure: share of pixels within 25 % of the true depth |
| 6D pose | An object's position (3) + orientation (3) |