Article
https://doi.org/10.1038/s41467-023-38258-2
3D-patterned inverse-designed mid-infrared
metaoptics
Gregory Roberts
1
, Conner Ballew
1,3
, Tianzhe Zheng
1
,JuanC.Garcia
2
,
Sarah Camayd-Muñoz
1,4
,PhilipW.C.Hon
2
&AndreiFaraon
1
Modern imaging systems can be enhanced in ef
fi
ciency, compactness, and
application through the introduction o
f multilayer nanopa
tterned structures
formanipulationoflightbasedonits
fundamental properties. High trans-
mission multispectral imaging is e
lusive due to the commonplace use of
fi
lter
arrays which discard most of the incident l
ight. Further, given the challenges of
miniaturizing optical systems, most
cameras do not leverage the wealth of
information in polarization and spatial degrees of freedom. Optical metama-
terials can respond to these electr
omagnetic properties but have been
explored primarily in single-layer geometries, limiting their performance and
multifunctional capacity. Here we use
advanced two-photon lithography to
realize multilayer scattering structur
es that achieve highl
y nontrivial optical
transformations intended to process lig
ht just before it re
aches a focal plane
array. Computationally optimized mul
tispectral and polarimetric sorting
devices are fabricated with submicron f
eature sizes and experimentally vali-
dated in the mid-infrared. A
fi
nal structure shown in simulation redirects light
based on its angular momentum. These d
evices demonstrate that with precise
3-dimensional nanopatterning, one can
directly modify the scattering prop-
erties of a sensor array to create advanced imaging systems.
Nanophotonics synthesizes the stu
dy of light-matter interaction with
the precise, repeatable techniques of nanofabrication. For example,
dielectric metasurfaces are arrays of subwavelength scatterers that
apply a spatially varying phase, polarization or amplitude response to
an incoming wavefront
1
. The local control is related to the speci
fi
c
shape of each scatterer which can be chosen to compactly replicate
and combine functionalities of common optical components like len-
ses, beamsplitters, polarizers, and waveplates or realize more novel
devices such as those used for visible color routing at the pixel level
2
.
For metasurfaces, the absence of substantial inter-element electro-
magnetic coupling is often leveraged for ease of design, but this sim-
pli
fi
cation also limits the available degrees of freedom. Ultimately, we
would like to tailor unique scattering behaviors for wavefronts with
different spectral, spatial, and polarization properties. To do this, we
can expand the design space to volumetric devices where a material is
patterned at subwavelength resolution in three dimensions.
Three-dimensional (3D) devices take advantage of a larger set of
optical modes to achieve unprecedented performance in a variety of
tasks, but require an ef
fi
cient gradient-based optimization algorithm
based on full-wave electromagnetic simulation. Searching the high-
dimensional space of permittivity pro
fi
les, typically for a local opti-
mum to an electromagnetic merit function, is referred to as inverse
design
3
–
6
. In this area, quasi-2D on-chip photonic devices have been
explored extensively where patterning in the direction of light pro-
pagation is achieved in a single fabrication layer
7
–
9
. The fully 3D design
paradigm for free space applications is yet to emerge in the infrared
and visible spectra, mostly due to the increased fabrication complexity
of volumetric devices. However, early works in this area utilizing one-
Received: 25 November 2022
Accepted: 14 April 2023
Check for updates
1
Kavli Nanoscience Institute and Thomas J. Watson Sr. Laboratory of Applied Physics, California Institute of Technology, 1200 E California Blvd, Pas
adena 91125
CA, USA.
2
NG Next, Northrop Grumman Corporation, 1 Space Park Drive, Redondo Beach 90278 CA, USA.
3
Present address: Jet Propulsion Laboratory,
California Institute of Technology, 4800 Oak Grove Dr, Pasadena 91109 CA, USA.
4
Present address: Applied Physics Laboratory, The Johns Hopkins University,
11100 Johns Hopkins Road, Laurel 20723 MD, USA.
e-mail:
faraon@caltech.edu
Nature Communications
| (2023) 14:2768
1
1234567890():,;
1234567890():,;
and two-layer processes for optical applications or many-layer micro-
wave prototypes have shown the utility of moving to thicker
devices
10
–
12
. In this work, we optimized a two-photon polymerization
(TPP) lithography process to create multilayer structures at optical
wavelengths. This technique has been employed in the past for fabri-
cating refractive, diffractive, gradient index, and extruded 2D inverse-
designed optical components
13
–
16
. By exploiting TPP
fl
exibility for 3D
patterning at subwavelength resolution, we experimentally demon-
strated multiple inverse-designed, multilayer photonic devices with
applications to advanced imaging in the mid-infrared band (3
–
6
μ
m).
Compact imaging systems utilize wavelength- and polarization-
selective elements to characterize fundamental properties of wave-
fronts. Color imaging in consumer c
ameras follows this approach where
absorptive
fi
lters are placed on top of collections of pixels to sense
three or four spectral overlaps. The classic arrangement, referred to as a
Bayer pattern, consists o
f a red, blue, and two green
fi
lters arranged
2×2 in a square
17
. Filtering schemes like this come at a cost of trans-
mission ef
fi
ciency because they absorb all light outside of their pass-
band leading to average transmission values of ~33% under uniform
spectral illumination. Solutions to this problem have converged on the
concept of color routing where scattering structures accept light inci-
dent on a group of pixels and redirect each wavelength band to a dif-
ferent pixel
2
,
12
,
18
–
20
. In this manuscript, we demonstrate an ef
fi
cient,
multilayer inverse-designed device in the mid-infrared for accomplish-
ing this task and further augment it to sense linear polarization. Beyond
multispectral imaging, the geometry of splitting light at the focal plane,
depicted in Fig.
1
a, can be tailored to ef
fi
ciently decode other electro-
magnetic properties. Designing at the pixel level modularizes the
optical system, allowing focal plane
arrays equipped with arrangements
of scattering structures to control the imaging properties of the camera.
Figure
1
b
–
d indicates the breadth of devices in this manuscript.
Results
Multispectral and polarization sorting
The
fi
rst application we explored is combined multispectral and
polarization imaging. Absorption spectra in the mid-infrared, part of
the molecular
fi
ngerprinting region
21
, correlate strongly to distinct
chemical species. Among many areas of interest, this can be used for
environmental monitoring
22
,
23
and biomedical imaging
24
,
25
. Solutions
such as multiplexed
fi
lters in the mid-infrared suffer from low overall
transmission ef
fi
ciency
26
. They also lack a straightforward path
towards multifunctionality that may be critical for a given application.
In remote thermal monitoring, for example, multispectral and polar-
ization
fi
ltering can be used in tandem to distinguish radiated and
re
fl
ected light reducing instances of thermal blindness
27
. To address
these challenges, we designed and fabricated a multilayer color-
routing device with additional linear polarization discrimination.
The optimization goal, stated in Eq. (
1
), is constructed to sort
three spectral bands from 3.7
–
5
μ
m and distinguish between linear
polarization for the middle band. The device dimensions are
30.15
μ
m×30.15
μ
m×18
μ
m (6.6 × 6.6 × 4.0
λ
3
mid
), broken into six 3
μ
m
thick layers, compact enough to be tiled on a high-resolution focal
plane array.
max
ε
2f
ε
min
,
ε
max
g
N
g
ð
E
Þ
=
P
λ
S
P
p
P
q
κ
ð
q
,
p
,
λ
Þ
I
p
ð
r
q
,
λ
Þ
I
max
ð
λ
Þ
!
;
k
!
I
p
ð
r
q
,
λ
Þ
=
∣∣
E
p
ð
r
q
,
λ
Þ
∣∣
2
ð
1
Þ
Electromagnetic inverse design that utilizes the mathematical
adjoint method for calculating gradients with respect to material
permittivity, aims to ef
fi
ciently optimize merit functions like this,
where device performance is phrased in terms of electric and magnetic
fi
elds in an observation region
3
–
6
. Here, the electric
fi
eld intensity at the
center of each quadrant is maximized for correct wavelengths and
polarizations, and minimized for incorrect ones through choice of sign
in the
κ
(
q
,
p
,
λ
) weighting function where
p
indexes the linear polar-
ization and
q
indexes the quadrant with center
r
q
.The
fi
rst summation
targets broadband performance by including closely spaced wave-
lengths in each band to effectively optimize the device across a con-
tinuum. The purpose of the softplus function,
S
, is described in the
methods alongside other optimization
fi
gures of merit for this work
(Materials, methods, and additional text are available in the Supple-
mentary section). This optimization function is nonlinear over the
high-dimensional (~10
4
-dimensional for devices in this work) permit-
tivity tensor, composed of deeply subwavelength volumetric units
(voxels). It is optimized via gradient descent enabled by the well-
known adjoint method
3
,
7
,
8
. Combining the electric
fi
elds in the device
from adjoint simulations with those from the expected illumination, in
this case broadband linearly polarized plane waves, the gradient is
computed in a
fi
xed number of simulations independent of the num-
ber of design voxels. Fabrication constraints were incorporated for
layering, feature size control, and binarization using averaging, lateral
maximum blurring, and sigmoid projection
fi
lters, respectively
28
.
The optimization results are shown in Fig.
2
a, b, where three
sorting bands are present with the middle band focal spot conditioned
on linear polarization. Following this result, the device was fabricated
using the Nanoscribe Photonic Prof
essional GT, where subwavelength
features in the mid-infrared are readily created in a proprietary IP-Dip
polymer with low loss from roughly 3.5
–
5.5
μ
m
29
. The real and imagin-
ary refractive indices of this polymer averaged over the design wave-
length range are accounted for in the device optimization and
presented simulation results. Using
a photolithography-based liftoff
procedure, a series of 30
μ
m diameter circular aluminum apertures
were fabricated on a sapphire substrate. Apertures, also included in the
optimization, restrict the illumination to single devices for imaging and
experimental power calibration. The Nanoscribe was aligned to write
devices directly on top of the apertures. Figure
2
e shows scanning
electron microscope (SEM) images of fabricated devices. Each design
was illuminated by a quantum cascade laser (QCL) with a beam waist on
theorderofthedevicesizedefocusedsuchthattheapertureswere
over
fi
lled and sampled a roughly
fl
at amplitude and phase section of the
diverging beam. This is intended to mimic the plane wave input used for
device optimization. The QCL can be tuned spectrally to probe the
device at different wavelengths and the addition of linear polarizers and
waveplates were used to control the input polarization. Various focal
planes of the device were imaged by a zinc selenide (ZnSe) aspheric lens
onto a focal plane array (see Fig. S1). The QCL used in the experiment
had a limited wavelength tuning range from 3.95 to 5.05
μ
m, which is
why the plots in Fig.
2
c, d do not cover the full simulated spectra.
Figure
2
c, d contains the experimental spectral and polarization
sorting ef
fi
ciency, overall focal transmission, and focal spot intensities
to compare to simulation. Sorting ef
fi
ciency measures the ratio of total
focal plane signal incident on a given quadrant. In the simulated
sorting ef
fi
ciency, the middle band under horizontal (vertical) polar-
ization has a width of 420 nm (430 nm) with a center wavelength of
4.34
μ
m(4.35
μ
m) with respect to its crossover points with the upper
and lower bands. By comparison, in the experimental sorting ef
fi
-
ciency, the middle band under horizontal (vertical) polarization has a
width of 390 nm (420 nm) with a center wavelength of 4.42
μ
m
(4.43
μ
m) with respect to its crossover points with the upper and lower
bands, exhibiting a 20 nm (4.71%) smaller average width and an 80 nm
(1.84%) average redshift. Taking into account this redshift and con-
sidering data over an equivalent total bandwidth, the peak sorting
ef
fi
ciencies for the three bands under horizontally (vertically) polar-
ized illumination were 0.78, 0.63, 0.73 (0.78, 0.63, 0.74) in simulation
and 0.47, 0.38, 0.46 (0.45, 0.39, 0.44) experimentally for lower, mid-
dle, and upper bands, respectively.
We suspect the reduced contrast in experiment is due to imaging
aberrations, experimental beam non-idealities and fabrication errors,
Article
https://doi.org/10.1038/s41467-023-38258-2
Nature Communications
| (2023) 14:2768
2
which include device shrinkage and feature size mismatch from
proximity effects and resolution limits
30
. Transmission is measured as
power through the device printed on top of a 30
μ
m aperture that
reaches the focal plane versus power through an empty 30
μ
maper-
ture. We speculate the
fl
uctuations in the experimental transmission
around 4.25
μ
mand4.4
μ
m could be due to minor laser power
fl
uc-
tuations around its transition between two QCL modules, small beam
shifts between the device and pinhole normalization measurements,
or differing amounts of ambient carbon dioxide (CO
2
) absorption
between measurements given the long path optical path length and
strong CO
2
absorption near 4.25
μ
m
31
. In the focal plane images in
Fig.
2
c, d, one can see the focused spot move between the quadrants as
the wavelength changes demonstrating the splitting functionality with
the middle band sorted to opposite corners depending on its linear
polarization.
Full stokes polarimetry
For the second application, we investigated full Stokes imaging
polarimetry, where one characterizes not only the linear polarization
amplitudes, but also the phase relationship between them and the
degree of polarization. This rich information is widely applicable,
including in areas of biomedical imaging and diagnosis
32
,depth-based
imaging and facial recognition
33
,
34
,atmosphericmonitoring
35
,andbio-
inspired polarization based navigation
36
. In polarimetric imaging, the
C
C
camera
lens
inverse designed
structures
focal plane
array
a
c
b
30
μ
m
(~7
λ)
30
μ
m
(~7
λ)
25
μ
m
(~5
λ)
18
μ
m
(~4
λ)
30.15
μ
m
(~7
λ)
30.15
μ
m
(~7
λ)
25
μ
m
(~5
λ)
19.2
μ
m
(~4
λ)
30.15
μ
m
(~7
λ)
30.15
μ
m
(~7
λ)
25
μ
m
(~5
λ)
18
μ
m
(~4
λ)
b
c
d
Fig. 1 | Conceptual depiction of devices in this work. a
2D cross section sche-
matic of camera with inverse designed scattering elements placed on top of
photosensitive elements at the focal plane of the imaging lens. Green elements
sort by color and blue elements sort by polarization, shown in more detail in
(
b
,
c
).
b
Rendering of multispectral and linear polarization device that sorts
three bands of wavelengths with the middle band further split on polarization.
c
Rendering of full Stokes polarimetry device that sorts four analyzer Jones vec-
tors to different quadrants.
d
Rendering of angular momentum splitting device
that sorts combinations of orbital angular momentum (
l
) and spin (
s
) degrees of
freedom.
Article
https://doi.org/10.1038/s41467-023-38258-2
Nature Communications
| (2023) 14:2768
3
input state is cast in terms of a four-dimensional vector containing its
Stokes parameters, which together specify the orientation, handed-
ness, and degree of polarization. Complete reconstruction of this state
is done through at least four independent measurements. Measure-
ments can be multiplexed in time using a rotating waveplate
37
or in
space by dividing up the area on one or more focal plane arrays
38
.The
analogous geometry to using absorptive
fi
lters for color imaging is the
division of focal plane (DoFP) technique where pixels are grouped
together with each responsible for analyzing a speci
fi
c polarization
component. Many implementations use micropolarizer elements as
fi
lters
39
, thus limiting the transmission ef
fi
ciency of the camera to 50%
by rejecting orthogonally polarized light to each
fi
lter. Lost transmis-
sion can be recovered using pixel-sized metasurface lenses that apply
different phase masks to two orthogonal polarizations. For example,
six projections done pairwise onto orthogonal polarization basis states
directly measure the four Stokes parameters
40
. However, these six
measurements contain redundant information which reduces camera
resolution or degrades signal-to-noise ratio compared to a four-
measurement device with the same overall size. Recently, it was shown
that a metasurface grating could project incident light onto four
equally spaced analyzer states with each projection belonging to a
different order
41
. This approach requires propagation to spatially
separate each order and is inherently chromatic due to grating dis-
persion. We adopted bene
fi
ts and addressed shortcomings of both
approaches by employing the modularity of a pixel-level design for
adaptation of any camera sensor to full polarimetric imaging and uti-
lizing a minimal four-state projection for maximal compactness. Using
only four measurements is a 33% improvement in required chip area
or, alternatively, a commensurate resolution or signal-to-noise ratio
enhancement. As an added bene
fi
t, inverse design provides a path
towards broadband polarimetry, which is dif
fi
cult to achieve with
metasurface and waveplate based systems due to their inherent
chromatic dispersion.
We optimized a device of size 30
μ
m×30
μ
m×18
μ
minsix3
μ
m
layers for this purpose, with the optimization
fi
gure of merit adapted
to focus four analyzer polarization states to different quadrants and
reject their orthogonal states to those same quadrants. Further, we
augmented the experimental system to probe arbitrary polarization
e
a
c
Simulation
Experiment
Legend (Left Axis):
Sorting Efficiency (S)
Legend (Right Axis):
Transmission (T)
b
d
I
II
III
IV
V
I
II III
IV
V
I
II
III
IV
V
I
II
III
IV
V
III
III
IV
V
I
II
III
IV
V
I
II
III
IV
V
I
II
III
IV
V
Fig. 2 | Fabrication and measurement results of multispectral and linear
polarization sorting device. a
(top) Simulated device sorting spectrum showing
both relative sorting ef
fi
ciency (
S
) and net transmission (
T
) to focal plane normal-
ized to pinhole transmission. For quadrant
k
,
S
Q
k
=
T
Q
k
P
3
i
=0
T
Q
i
.(bottom)Intensity
images accounting for the expected imaging lens numerical aperture (NA = 0.67)
and showing the focal spot moving as a function of wavelength. Each numbered
plot corresponds to the labeled dashed vertical line in the spectrum above it.
Intensity units are arbitrary, but comparable between all plots in (
a
). Different
maximum values on the colorbars here and in other
fi
gures are labeled and
utilized for better visibility of the plotted intensity features.
b
Same plots as
in (
a
) for vertical polarization input.
c
,
d
Experimental comparison plots to
(
a
,
b
) respectively with standard deviati
on (SD) error bars. Wavelength cov-
erage differences between simulation
and experiment are due to the limited
tuning range of the QCL used experimentally.
e
Schematic and associated
SEM images of fabricated devices. The rightmost device was printed with one
quarter missing to show internal structure. Scale bars: 5
μ
m, 5
μ
m(inset
2
μ
m), 5
μ
m from top to bottom.
Article
https://doi.org/10.1038/s41467-023-38258-2
Nature Communications
| (2023) 14:2768
4
states for different wavelengths depicted in Figs. S1, S2. The simulation
and experimental results are presented in Fig.
3
a
–
dandfabricated
devicesareshowninFig.
3
e. Performance is quanti
fi
ed with two
metrics. First, for each quadrant, the contrast,
C
∈
[
−
1, 1], is the
transmission for an analyzer state versus its orthogonal state:
C
=
T
analyzer
T
orthogonal
T
analyzer
+
T
orthogonal
. The optimization solution performs better for the
three elliptical polarizations compared to the circular polarization
state in this case, likely due to a lack of degrees of freedom. In Sup-
plementary Fig. S3, we show a thicker 12-layer device in simulation with
improved contrast of the circular polarization state. Similar to the
multispectral device, there is a reduced contrast experimentally which
we attribute again to fabrication and experimental imperfections.
Second, transmission is quanti
fi
edforeachanalyzerstate,which,as
shown in the Polarimetry Contrast Bounds Supplementary section, is
limited to 50% in a perfect device due to required vector overlaps
between analyzer states
42
. We note that this is not a limit on total
device transmission, but simply a requirement of linearity. Observing
the focal plane images in Fig.
3
b, d demonstrates the polarization
sorting capability of the device. The most telling indication of desired
behavior is seen by observing the orthogonal state inputs where the
device can theoretically fully extinguish transmission to a quadrant
(Materials, methods, and additional text are available in the Supple-
mentary section). By comparison to the analyzer state, the same
quadrant under each orthogonal state is dark, which is supported
quantitatively with speci
fi
c transmission and contrast values in Fig.
3
a,
c. Due to experimental and fabrication non-idealities, the measured
device exhibits lower splitting contrast compared to simulation. For
example, at
λ
=4.5
μ
m, the simulated device achieved contrasts of
0.46, 0.82, 0.84, and 0.83 for analyzer states
S
0
,
S
1
,
S
2
,and
S
3
,respec-
tively. In comparison, for these same four states at
λ
=4.5
μ
m, the
measured device achieved contrasts of 0.25, 0.39, 0.45, and 0.37.
Practically, reconstruction of an arbitrary input polarization state can
be done via a calibration procedure to account for imperfect contrast.
An example calibration considering the simulated device behavior is
demonstrated in the supplementary and analysis of reconstruction
a
c
d
A
nal
yz
er
State
s:
O
rt
h
o
g
onal
State
s:
orthogonal
states
analyzer
states
analyzer
states
orthogonal
states
b
wavelength for (d)
wavelength for (b)
experimental plot region
e
Fig. 3 | Fabrication and measurement results of Stokes polarimetry device.
a
(top) Polarization contrast (
C
) in simulation quantifying the transmission (
T
)into
the desired quadrant for a given analyzer state versus transmission into the same
quadrant for the orthogonal state. For input
k
,
C
k
=
T
∣
S
k
i
!
Q
k
T
∣
^
S
k
i
!
Q
k
T
∣
S
k
i
!
Q
k
+
T
∣
^
S
k
i
!
Q
k
. (bottom)
Transmission into the desired quadrants for the analyzer states (solid) and their
orthogonal complements (dashed).
b
Simulated focal intensity images (
λ
=4.5
μ
m)
accounting for imaging lens numerical aperture (NA =0.67) for the various input
states where the top row contains analyzer states and the bottom row contains
orthogonal states. Intensity units are arbitrary, but comparable between all plots in
(
b
).
c
Comparison plots of contrast and transmission for the experimental results
with analyzer states shown with open circles and orthogonal states shown with
stars in the transmission plot (SD error bars). The experimental transmission plot
region is different than the simulation one in both the
x
-and
y
-axes. This region is
marked with a dashed box on the simulation plot in (
a
).
d
Experimental focal
intensity images (
λ
=4.5
μ
m) showing a bright quadrant for each analyzer state and
the same quadrant dark for the complementary orthogonal state. Intensity units are
arbitrary, but comparable between all plots in (
d
).
e
Schematic and associated SEM
images of fabricated devices. The rightmost device was printed with one quarter
missing to show internal structure. Scale bars: 5
μ
m(inset2
μ
m), 5
μ
m, 5
μ
m from
top to bottom.
Article
https://doi.org/10.1038/s41467-023-38258-2
Nature Communications
| (2023) 14:2768
5