Article
https://doi.org/10.1038/s41467-023-39018-y
Multi-Modal Mobility Morphobot (M4) with
appendage repurposing for locomotion
plasticity enhancement
Eric Sihite
1
, Arash Kalantari
2
, Reza Nemovi
1
,AlirezaRamezani
3
&
Morteza Gharib
1
Robot designs can take many inspiratio
ns from nature, where there are many
examples of highly resilient and fault-to
lerant locomotion strategies to navi-
gate complex terrains by recruiting multi-functional appendages. For example,
birds such as Chukars and Hoatzins can repurpose wings for quadrupedal
walking and wing-assisted incline runni
ng. These animals showcase impressive
dexterity in employing the same appendages in different ways and generating
multiple modes of locomotion, resulting in highly plastic locomotion traits
which enable them to interact and navig
ate various environments and expand
their habitat range. The robotic biomimicry of animals
’
appendage repurpos-
ing can yield mobile robots with unparalleled capabilities. Taking inspiration
from animals, we have designed a robot
capable of negotiating unstructured,
multi-substrate environments, including land and air, by employing its com-
ponents in different ways as wheels, thrusters, and legs. This robot is called the
Multi-Modal Mobility Morphobot, or
M4 in short. M4 can employ its multi-
functional components composed
of several actuator types to (1)
fl
y, (2) roll,
(3)crawl,(4)crouch,(5)balance,
(6) tumble, (7) scout, and (8) loco-
manipulate. M4 can traverse steep slop
es of up to 45 deg. and rough terrains
with large obstacles when in balancing mode. M4 possesses onboard com-
puters and sensors and can autonomous
ly employ its modes to negotiate an
unstructured environment. We present the design of M4 and several experi-
ments showcasing its multi-modal capabilities.
This work aims to design a robot capable of negotiating unstruc-
tured, multi-substrate environments with extensive locomotion
plasticity by transforming its multi-purpose appendages to achieve
different functions, including wheel, leg, and thruster. We call this
robot M4, which stands for Multi-Modal Mobility Morphobot (Fig.
1
).
This morphobot could be used in a broad number of applications,
including search and rescue operations, space exploration, auto-
mated package handling in residential spaces, and digital agriculture,
to name a few.
Envision search and rescue after natural disasters such as earth-
quakes,
fl
ooding, or windstorm (Fig.
2
). In the aftermath of unique
incidents such as
fl
ooding, one event may accompany another that
destroys the landscape differently. A hurricane may produce
fl
ooding
and wind damage to roads and buildings. Or, a landslide may cause the
movement of a large rock mass down a slope, dam a river, and create a
fl
ood. In these scenarios, M4 can leverage its versatility to achieve
mobility that
fi
ts diverse mission requirements in search and rescue.
For instance, when ground locomotion is not feasible, M4 delivers
Received: 4 January 2023
Accepted: 22 May 2023
Check for updates
1
Aerospace Engineering Department, California Institute of Technology, 1200 E California Blvd, Pasadena, CA, USA.
2
Jet Propulsion Laboratory (JPL), 4800 Oak
Grove Drive, M/S 82-105 Pasadena, CA, USA.
3
Electrical and Computer Engineering Department, Northeastern University, 360 Huntington Ave, Boston, MA,
USA.
e-mail:
a.ramezani@northeastern.edu
Nature Communications
| (2023) 14:3323
1
1234567890():,;
1234567890():,;
critical strategic situational awareness by employing aerial surveying
and reconnaissance through multi-purpose scans of the area with a
suite of sensors integrated into its design. Aerial mobility inside con-
fi
ned and collapsed buildings is not practical. Imagine mobility inside
tight, collapsed stairways and corridors on top
fl
oors needed. In that
case, M4 utilizes diverse forms of ground locomotion, including four-
wheel rolling and crouching, two-wheel rolling and standing (with or
without thrusters), quadrupedal walking, or tumbling to negotiate
inside collapsed
fl
oors. For instance, wheeled and legged mobilities
have limitations as they cannot handle rough terrains when obstacles
are larger than the wheels
’
and legs
’
size. Instead, M4 tumbles over
them, i.e., it leverages the ability to upright using its thrusters to
achieve the height advantage needed to fall over large obstacles.
This work presents the design and control of a versatile multi-
modal robot called M4 shown in Fig.
1
. The contributions of this work
are multi-fold. First, we show a signi
fi
cant modal diversity not reported
in the literature. Inspired by animals with considerable locomotion
plasticity, such as birds, the M4 robot can perform various modes of
locomotion by redundancy manipulation through appendage repur-
posing. M4 repurposes its appendages with its transforming body and
UGV
UAS
Crouching
Tumbling
MIP
Walking
tr
ansformation
transformation
leg swing motion
leg frontal motion
leg frontal motion
wheel rolling
thrust force
a
b
Fig. 1 | Multi-Modal Mobility Morphobot (M4). a
Shows M4 in wheeled mode.
b
Illustrates cartoon depictions of M4
’
s transformation to other modes.
a
b
c
Rescue Team
Multi-Modal
Mobility Morphobot (M4)
Critically injured victim
Robot snapshots during SAR operation
Closeup view
of landing
Landing location
autonomously selected
Goal
Start
Unintrusive wheeled
mobility inside building
Fig. 2 | Envisioned search and rescue example. a
An illustration showing the
deployment of M4 outside a collapsed multi-story building in the aftermath of an
earthquake.
b
M4 employs its aerial mobility to reach quickly and land on
inaccessible locations.
c
Other modes, such as wheeled mobility, are employed
when a
fl
ight is impossible.
Article
https://doi.org/10.1038/s41467-023-39018-y
Nature Communications
| (2023) 14:3323
2
switchable shrouded propellers to switch to an unmanned ground
vehicle (UGV), mobile inverted pendulum (MIP), unmanned aerial
system (UAS), thruster-assisted MIP, legged locomotion, and loco-
manipulation in MIP mode. Second, by repurposing the mobility
components in M4, we achieve a scalable design that supports fully
autonomous and self-contained operations. We show the robot pos-
sesses the payload capacity to carry computers and exteroceptive
sensors for fully autonomous multi-modal operations. Third, we
combine locomotion diversity and autonomy in M4 to perform novel
maneuvers such as tumbling over large obstacles and traveling over
steep ramps. This paper presents the mechanical design and the
algorithms that enable M4 to perform these modes. These algorithms
are explained in the Method Section and entail an optimization-based
control (collocation method) and path planning algorithm (multi-
modal probabilistic road map [MM-PRM] and A
*
algorithms). We report
the experimental results that substantiate the claimed capabilities.
The overarching objective of the M4 design is to achieve a scalable
solution with extensive locomotion plasticity to substantiate the sce-
narios explained above. We call a mobile robot design scalable if its
payload capacity can be increased such that its mobility is not severely
affected. While there are various ways to measure scalability, one
fundamental approach is to evaluate it based on the maximum
allowable payload that the system can carry before it becomes com-
pletely immobilized in any mode. Obviously, scalability depends on
several factors, including actuators and mechanisms
’
performance,
locomotion modes, and substrate characteristics. Since multi-modal
locomotion involves different actuators, mechanisms, modes, and
substrates, the scalability problem can be very confounding. For
instance, it is generally tough to accommodate the con
fl
icting
requirements dictated by ground and aerial locomotion in a single
platform. On the one hand, powerful actuators and rugged structures
are needed to generate and maintain traction forces or joint torques to
successfully realize wheeled or legged locomotion. The plurality of
actuators in these systems is very high to substantiate posture control.
On the other hand, these actuators and structures are often very bulky,
negatively affecting aerial mobility, which depends on light structures.
Here, the question to ask is: Which design views yield scalable
robots with large locomotion plasticity? We list three views, including
two mainstream views (1
–
2) that cover the multi-modal designs
introduced in literature and one view (3) that has been explored to a
very limited extent:
•
View 1: Morpho-Functionality-
In this view, multi-modal locomo-
tion is achieved through body a
nd appendage morphing. These
designs comprise manifold rigid (or soft) links and actuated
joints that form articulated bodies and appendages. Morphing
or shape-shifting is considered the primary mechanism for
changing appendage function. The appendages can be, e.g.,
legs, wings,
fl
ippers, wheels, slithering structures, etc., simulta-
neously by changing their shapes and motions. The transform-
ing body recruits these multi-functional appendages and shares
them among different mobility modes.
Many morpho-functional machines with promising morphing
designs based on rigid
1
–
16
and soft bodies
17
–
23
have been
introduced so far. A large number of these designs are legged
7
–
16
,
slithering
24
, and amphibious
6
,
25
,
26
robot. Other unconventional
designs such as quadruped with recon
fi
gurable joints
1
,trans-
forming robot that can use its wings as legs
27
, multi-rotor with
morphing body
2
, shape-shifting wheeled robot
3
,
5
,
9
, and adaptive
wheel-and-track
4
have been introduced as well. However, these
multi-modal robots showcase limited locomotion plasticity
(two-three modes)
2
–
4
. Soft morpho-functional options have
been extensively studied too.
However, they can accommodate
a limited number of modes and have faced scalability challenges.
For instance, while soft structures share strong similarities with
shape-shifting biological mechanisms in vertebrates and
invertebrates, these engineered elements cannot match their
biological counterparts in terms of generated force-motion
pro
fi
le per unit mass
18
,
20
,
22
. State-of-the-art soft robots cannot
scale up to large, self-contained systems with notable locomo-
tion plasticity since they depend on large accessories such as
pneumatic systems or high-voltage power supplies.
•
View 2: Redundancy-
In this view, multi-functionality is achieved
by brute-force approaches based on the plurality of appendages
that can deliver one function only. Hence, the appendages are
not shared among different modes and are
fi
xated on non-
morphing bodies. Note that by redundancy we refer to the
number of appendages involved in a locomotion mode. We label
it redundant if more appendages are required than the minimum
number needed for that mode. Therefore, redundancy in
actuated joints does not render a system redundant. Consider
human bipedal locomotion that consists of two legs each
comprising a plural of muscles (analog to robot actuators) that
would allow the leg to deliver different functions. In our view,
this example is not redundant.
There is a plethora of celebrated works
28
–
35
that successfully
have utilized redundancy in their designs to achieve multi-modal
locomotion. These redundant designs present less complexity,
which is a bene
fi
t, by carrying additional actuators and robotic
mechanisms to substantiate legged-aerial
32
–
34
,wheeled-
aerial
29
,
30
,
35
, and amphibious locomotion
28
. For example, the
robot designed by ref.
29
is a quad-rotor with wheels and motors
af
fi
xed at the base of the robot to enable ground mobility for the
initially aerial-only robot. Another notable example is
HyTAQ
by
ref.
36
which comprises a multi-rotor aerial system encapsulated
by a barrel-shaped guard that allows safe wheeled mobility.
However, in these designs, there is a strict limit on the number of
modes that can be integrated and these robots quickly face
added mass issues.
•
View 3: Manipulation of Redundancy by Morphing-
So far, both
Views 1
–
2 with various levels of complexity have been adopted in
robotics. In some concepts, morpho-functionality is the main
design theme and, in many examples, redundancy. However, in
nature, animals showcase a behavior that combines both views;
animals utilize their morpho-functional structures to repurpose
the appendages to create (or to eliminate) redundancy when
needed and gain mobility advantage. For instance, aquatic
animals such as turtles and sea lions use their front
fl
ippers for
swimming. They repurpose the same
fl
ippers (Fig.
3
a) to support
their heavy body weight and to walk on the ground like a
quadruped
37
. Or, Meerkats, as shown in Fig.
3
b, can eliminate the
redundancy in their locomotion apparatuses by standing on their
hindlimbs to scout their surroundings. These animals cannot
walk well on two legs, but they can use them to elevate their
fi
eld
of view to monitor their surroundings to avoid predators
38
.
Birds such as Hoatzins and Chukars manipulate redundancy in
their locomotion apparatuses as well. Juvenile Hoatzins show-
case wing-assisted walking
39
to move up vertical or steep slopes
to refuge and dodge danger (Fig.
3
c). They repurpose the wings
and shape-shift the articulated body to extract leg functions
from their wings and achieve quadrupedal locomotion. Young
Hoatzin nestlings retain functional claws in their wings which
helps them to manifest quadrupedal locomotion and even climb
in the vegetation.
On a similar note, Chukar birds adopt a similar wing repurposing
to increase redundancy to support legged locomotion over
steep terrain through a phenomenon known as wing-assisted
incline running (WAIR)
40
(Fig.
3
d). To walk over steep surfaces,
they leverage their wings
’
contributions differently to walk on
steep inclinations. Chukar chicks walk and run up steep slopes
by beating their developing wings and generating aerodynamic
Article
https://doi.org/10.1038/s41467-023-39018-y
Nature Communications
| (2023) 14:3323
3
lift force which increases the ground contact force at their legs
41
.
With the WAIR strategy, mature Chukar birds can negotiate
nearly vertical and overhanging slopes as if walking on
fl
at
ground.
The robotic biomimicry of these redundancy manipulations
through appendage morphing has remained unexplored. The
celebrated multi-modal robots presented by refs.
42
,
43
possess
interesting designs that permit
fl
ipper-leg and wheel-leg
repurposing to achieve aquatic-legged and wheeled-legged
modalities. However, M4 differs from refs.
42
,
43
work because
M4 exhaust appendage redundancy manipulation through
morphing to maximize locomotion plasticity. For instance
42
,
repurposes four
fl
ippers into four legs for walking. Instead, M4
repurposes four legs into:
•
Four legs for quadrupedal locomotion (Supplementary
Video 1),
•
Four thrusters for
fl
ight (Supplementary Video 2),
•
Two thrusters + two wheels for WAIR over 45-deg slopes
(Supplementary Video 3),
•
Two thrusters + two wheels for tumble over large obstacles
(Supplementary Video 4),
•
Two wheels + two hands for loco-manipulation (Supplemen-
tary Video 1),
•
Two wheels for MIP (Supplementary Video 5),
•
Four wheels for UGV (Supplementary Video 1),
•
Four wheels for crouching (Supplementary Video 1).
It can be seen that the redundancy manipulation through
appendage morphing in M4 is not matched by refs.
42
,
43
.The
extent by which these repurposings are strategized to diversify
locomotion modes is very limited in these examples. In addition,
in these works, appendage repurposing is not considered as a
tool to achieve scalability and combat the con
fl
icting require-
ments posed by a plurality of locomotion modes. For instance,
the MIP maneuver showcased in ref.
43
only works on
fl
at
ground and cannot be scaled to steep slopes like M4.
Results
Design rationale
By inspecting the state-of-the-art multi-modal robots, we notice that,
besides many redundant designs, a large number of soft- and rigid-
bodied morphing systems have been introduced so far. By using
redundancy and novel adaptive structures, the robotic community has
tirelessly worked on democratizing multi-modal robots that can
showcase animals
’
locomotion resiliency and fault tolerance. However,
the total number of modes achieved in these examples has remained
limited to small numbers. In addition, today
’
s multi-modal robots that
face con
fl
icting design requirements are not scalable, i.e., they do not
have the payload capacity needed to carry large items to render their
multi-modality useful. In these designs, in addition to the added mass
from each mode, there is another form of added mass that must be
considered to avoid the risk of immobilization. As the mass from other
modes adds up, some modes (e.g., UAS and legged modes) require the
addition of large actuators, power electronics, and batteries to prevent
the risk of immobilization. In other words, in these modes, component
size rapidly grows as the total mass increases. Other modes may be less
sensitive to mass increase. For instance, the manipulation mode can-
not be affected by an increase in the total mass since it depends solely
on the object
’
s mass, not the robot
’
s mass. On the contrary, the legged
mode is very sensitive to mass increase since joint actuators have to
carry the robot
’
sweight.
The main objective of M4 design is to achieve a scalable solution
with many mobility modes. Note that, in the design of M4, we are
focused on copying animals
’
strategies to enhance locomotion plasti-
city rather than mimicking the shape of animal appendages (
fl
apping
versus rotary wings). For this objective, we adopt the design approach
based on manipulating appendage redundancy through component
repurposing for the following reason. This view multi-folds the force-
to-weight ratio required for large payloads and demanding locomotion
modes through three mechanisms. First, added mass from compo-
nents is shared by all modes, a key mechanism that motivates appen-
dage repurposing. Second, force ampli
fi
cation becomes possible in a
mode through heterogeneous mobility component recruitment. For
Fig. 3 | Cartoon depiction of appendage repurposing in different species. a
Sea
lions
fl
ipper-assisted walking
37
.
b
Meerkats' hindlimb-assisted scouting
38
.
c
Hoatzin
nestlings wing-assisted quadrupedal locomotion (Image inspired and modi
fi
ed
with permission from
39
authors).
d
Chukar birds' wing-assisted incline walking
40
.
Article
https://doi.org/10.1038/s41467-023-39018-y
Nature Communications
| (2023) 14:3323
4
instance, aerodynamic lift forces can manipulate contact friction and
traction forces in wheeled mobility and allow steep slope locomotion,
a behavior inspired by birds
’
WAIR maneuvers. Third, force ampli
fi
-
cation becomes possible in a mode through homogeneous mobility
component recruitment. For instance, for a
fi
xed mass, the thrust-to-
weight ratio doubles and quadruples when switching from UGV to MIP
and UAS. To see other bene
fi
ts of appendage redundancy manipula-
tion that are not explored in M4
’
s design refer to a conceptual design
depicted in Supplementary Fig. 1.
System Overview
The M4 robot, shown in Fig.
4
, can switch its modes of mobility
between UGV, UAS, MIP, quadrupedal, thruster-assisted MIP, legged
locomotion, and manipulation. M4 possesses an articulated body with
four legs where each leg has two actuated hip joints for frontal and
sagittal leg movements and a shrouded propeller that acts as a wheel
and thruster simultaneously. The frontal joints permit the legs to move
in the sideway direction. On the other hand, the sagittal joints
accommodate forward and backward swing movements in each leg.
This body articulation allows various transformations. For instance, as
shown in Fig.
4
a, to achieve a UAS con
fi
guration with a four-fold thrust
force,
fi
rst, the legs swing forward and backward. Then, they turn
sideways with the frontal actuators. In M4, the propeller
’
sshroudacts
as a wheel which is actuated by a motor that drives through the gears
attached to the shroud
’
s rim, as illustrated in Fig.
4
b. The propulsion is
generated by the propeller and motor inside the shroud aligned with
the wheel axis. If the motion of the propellers and shrouds is con-
sidered, the robot possesses a total of 16 actuators and body degrees
of freedom (DOF). As a result, the total number of DOFs in M4,
including actuated coordinates, body positions, and orientations, is 22.
The mechanical design and components overview of M4 can be
seen in Fig.
4
b. The robot weighs approximately 6.0 kg with all com-
ponents, including the onboard comp
uters for low-level control and
data collection, sensors (encoders, inertial measurement unit, stereo
cameras), communication devices for
teleoperation, joint actuators,
propulsion motors, power electron
ics, and battery. M4 measures 0.7 m
in length, and 0.35 m in both width and height when in UGV mode.
When in MIP mode and dynamically balancing on its two wheels, it is 1.0
m tall, which permits reaching a better vantage point for data collection
using its exteroceptive sensors. When in UAS con
fi
guration, M4 is 0.3 m
tall, and propellers
’
center points can reach a maximum distance of 0.45
m far apart from each other. Each propeller-motor combination can
generate a maximum thrust force of ~2.2 kg-force, therefore reaching
roughly 9 kg thrust force in total. Its legs including the wheels are 0.3 m
long, and its wheels are 0.25 m in diameter, which allows for traversing
bumpy terrain. Table
1
lists the component weight distribution of the
most recent M4 design without a stereo camera attached.
The chassis structures and shrouded propeller components in M4
were primarily made of carbon
fi
ber and 3D-printed parts. The 3D-
printed parts are fabricated using a
fi
ber-inlay process based on Onyx
thermoplastic materials and carbon
fi
ber. These materials were con-
sidered due to their great strength-to-weight ratios. M4
’
ssystem
a
a1
a2
a3
a4
a5
a6
a7
a8
b
sideway motion
swing motion
rolling
Gear mesh
Lithium polymer battery
6S 40C 4000 mAh
Main controller
SAM3X8E
ARM Cortex M3
Hip Servos (x8)
55 kg.cm torque
Wheel motors
25D, 9.7:1 gearbox
Propeller motors
2514 1500KV
Lithium polymer battery
6S 40C 4000 mAh
Current and
voltage sensor
Propeller motors
2514 1500KV
Propeller ESCs (x4)
APD 80F3[X]
Wheel motors
25D, 9.7:1 gearbox
Wheel motor
drivers (x4)
TB67H420FTG
7.4V step-down
voltage regulator
Hip Servos (x8)
55 kg.cm torque
Telemetry module
915 MHz TTL UART
Wi-Fi module
ESP-WROOM-32
Ground station
computer
Flight controller
STM32H743AI
ARM Cortex M7
24V
7.4V
5V
12V step-down
voltage regulator
12V
Main controller
SAM3X8E
ARM Cortex M3
Power Electronics
Controller
Communication
Serial
Serial
RC receiver
R7008SB
RC transmitter
Motion capture
computer
Data signal
Control signal
Power line
Serial
Serial
PWM
PWM
PWM
PWM
Wireless
Wireless
Wi-Fi
Wi-Fi
c
Fig. 4 | An overview of M4
’
s hardware design. a
Illustrates an example appendage
repurposing process in M4 to increase thrust-to-weight ratio fourfold.
b
Shows a
closeup view of M4
’
s mechanical design, degrees of freedom, and components.
c
Shows M4
’
s electronics architecture, including the communication, controller,
and power electronics components.
Article
https://doi.org/10.1038/s41467-023-39018-y
Nature Communications
| (2023) 14:3323
5
architecture is outlined in Fig.
4
c showing the controller system, power
electronics, and communication protocol used in the robot. The robot
utilizes two microcontrollers for low-level locomotion control; one is
used for posture and wheel motion control, while the other is used to
regulate thrusters. In addition to the low-level locomotion controllers,
there is a high-level decision-making computer for autonomous multi-
modal path planning. The details of M4
’
s dynamic modeling, low-level
locomotion controller design, and high-level, multi-modal path plan-
ning can be seen in the Methods Section.
Experimental results
To substantiate the claimed locomotion plasticity in M4, we per-
formed several experiments, including, wheeled locomotion,
fl
ight,
MIP, crouching, object manipulation, quadrupedal-legged locomo-
tion, thruster-assisted MIP over steep slopes, and tumbling over large
obstacles. In addition, to show M4
’
sdesignisscalableandcanachieve
payload capacities that support self-contained operations, we tested
fully autonomous multi-modal path-planning using onboard sensors
and computers in M4. A summary of these experiments is shown in
Figs.
5
–
8
.
Figure
5
a shows snapshots of M4 navigating around and over a
pond from Supplementary Video 2. M4 is teleoperated (not autono-
mous) in this test. M4 employs its wheeled mobility to reach the pond
’
s
edge, then it transforms into a UAS and
fl
ies over the pond to the other
side of it. The UGV-UAS transformations follow the steps shown
in Fig.
4
.
Figure
5
b shows the snapshots of the MIP maneuver from Sup-
plementary Video 5. The MIP maneuver was performed in a closed-
loop fashion based on the collocation method (see Methods Section).
In this experiment, we performed controlled transitions from UGV to
MIP and MIP to UGV. In the MIP maneuver,
fi
rst, the front appendages
are repurposed from wheel to thruster by employing the sagittal and
frontal joints. Second, the thrusters
’
force and wheels
’
tractions are
regulated using an optimization-based, nonlinear closed-loop feed-
back controller in real-time. The body orientation and angular velocity
are sensed in real-time, then the control actions are generated to track
desired angular rates to achieve a stable MIP con
fi
guration. The
Table 1 | Detailed weight bre
akdown, total weight = 5.6kg
Name
Weight
Name
Weight
Battery (6S 4Ah)
590 g
Leg assembly (×4)
400 g
Chassis assembly
940 g
Hip servos (×8)
560 g
Microcontrollers
115 g
Propeller motors (×4)
270 g
Communication
120 g
Wheel motors (×4)
380 g
Power electronics
80 g
Tire assembly (×4)
1600 g
Cables, fasteners, etc.
440 g
Motor drivers (×4)
107 g
b10
b9
b8
b7
b6
b4
b5
b3
b2
b1
d
e
b
c
crouching
c1
c2
c3
d1
d2
d3
e1
e2
e3
e4
e5
a4
a3
a2
a1
a
hands open
object
low-ceiling pathway
pond
Fig. 5 | M4
’
s various locomotion modes. a
Ground-aerial locomotion near a pond.
M4 rolls to the edge of the pond (a1), transforms into UAS mode and takes off (a2),
fl
ies over the pond to the opposite side (a3), and
fi
nally lands before transforming
back into UGV mode (a4).
b
Illustrates the UGV to MIP and MIP to UGV maneuvers.
M4 repurposes its front appendages to thruster mode (b1-b2), performs MIP
maneuver to self-upright (b2-b4), dynamically balances in MIP mode (b5-b6),
descends in MIP maneuver (b7-b9), and
fi
nally transforms back into the UGV mode
(b9-b10).
c
M4
’
s crouching maneuver to pass under a low-clearance opening.
d
Shows M4
’
s manipulation ability in MIP mode based on repurposing its free
appendages.
e
M4 performs quadrupedal-legged locomotion on rough terrain by
locking the wheels and translating the legs.
Article
https://doi.org/10.1038/s41467-023-39018-y
Nature Communications
| (2023) 14:3323
6
desired angular rates of 10°/s and 5°/s were considered at the begin-
ning (sit-down con
fi
guration) and near the end (stand-up con
fi
gura-
tion) during the uprighting phase (UGV to MIP transition), respectively.
Similarly, the descending maneuver (MIP to UGV transition) uses the
same controller.
M4 can perform crouching, object manipulation, and
quadrupedal-legged locomotion as well (see Supplementary Video 1).
As shown in Fig.
5
c, the sagittal joints are employed to lower M4
’
smain
body to pass through low-ceiling pathways. Figure
5
d shows that the
free appendages (upper wheels) in MIP mode can be utilized for object
manipulation purposes; however, the dexterity of object manipulation
remains limited to grasping. Figure
5
-e shows the M4
’
squadrupedal
walking capability using quadrupedal-legged locomotion on rocky
terrain. To perform quadrupedal locomotion, the wheels are locked.
Then, the opposing legs are assigned swing and stance legs inter-
changeably. However, the absence of the knee joints in M4 prevents
more natural gaits seen in other quadrupedal robots with more
degrees of freedom in the legs.
We took two steps to show that M4 can operate in a fully self-
sustained fashion. First, we designed a multi-modal path-planning
algorithm and tested it using off-board sensing and computing (Fig.
6
based on Supplementary Video 6). Second, we translated this multi-
modal path planner to onboard computers and sensors in M4 (see
Fig.
7
based on Supplementary Video 7). Unlike the teleoperated pond
tests, our experiments in the lab environment entailed autonomous
multi-modal path planning and execution. The lab environment has an
OptiTrack motion capture (mocap) system. Several re
fl
ective markers
were attached to the robot and environment. The mocap system
’
srigid
body position and orientation measurements were transmitted to M4
’
s
computer through wireless communication. Then, a path-planning
algorithm based on MM-PRM and A
*
algorithm steered the system
towards the goal. The details and derivation of these algorithms can be
seen in the Methods Section. Figure
6
shows one of the tests where M4
follows the calculated trajectory to land on top of a 1.4-m tall platform
and transform back into UGV con
fi
guration. Then, we implemented
this MM-PRM algorithm on the Jetson Nano computer on M4 to
achieve fully autonomous and self-contained operation of M4 as
shown in Fig.
7
.
In the MIP maneuver (Fig.
5
b), we demonstrated that M4 could
repurpose its front and rear appendages to generate the external
forces required to stand up and sit down entirely independently
without external support. The maneuver provides two immediate
mobility advantages: increased reach (or higher vantage point) and
enhanced traction forces. The
fi
rst advantage can be leveraged to
tumble over large obstacles that cannot be handled with legged and
wheeled mobilities. The second advantage can be employed to travel
on steep slopes, similar to how birds use their wings and legs colla-
boratively to travel over inclined surfaces (i.e., WAIR maneuver). On
these steep slopes, large traction forces are required. These forces
cannot be substantiated by wheeled mobility.
The cartoon depictions of these maneuvers are shown in Fig.
8
a
and b. To perform the maneuver shown in Fig.
8
a, the robot transforms
1
2
3
4
5
6
tower
X [m]
Y [m]
Z [m]
-1
-2
-3
-4
1
0.5
0
-0.5
-1
-1.5
-2
0 sec
0.5
0
-0.5
-1
-1.5
-2
-2.5
0 sec
0 sec
30 sec
30 sec
30 sec
Estimation
Waypoint
UGV
UGV to UAS
UAS
UGV
UAS to UGV
Estimation
Waypoint
Estimation
Waypoint
MoCap
1.4 m
a
b
Fig. 6 | Multi-modal probabilistic road map (MM-PRM) path planning with
motion capture system. a
Shows the multi-modal path planning experiment
performed inside a
fl
ight arena with motion capture cameras. In this experiment,
the robot followed the path planning trajectory generated by the MM-PRM and A
*
algorithm to land on a 1.4 m tall platform. The robot started on the ground, drove to
the UAS morphing location (1), transformed into the UAS mode (2),
fl
ew on top of
the box (3 to 5), and transformed back into the UGV mode (6).
b
Illustrates the
desired and actual trajectories.
Article
https://doi.org/10.1038/s41467-023-39018-y
Nature Communications
| (2023) 14:3323
7
into the MIP con
fi
guration. The upper thruster pushes the robot up the
inclination while maintaining a certain pitch angle for stability. Then
the robot changes back into UGV mode once the inclination has been
cleared. Figure
8
b illustrates the tumbling maneuver, where the robot
uses its front or rear thrusters to lift one side of its body upwards and
gain height advantage to clear a tall obstacle or vault over a large gap.
First, M4 positions the front thrusters pointing upwards to lift the front
side. Then, the rear wheels drive forward so the front side vault over
the obstacle. M4 performs the same sequence with the rear thrusters
and front wheels to fully clear the large obstacle.
As shown in Fig.
8
canddbasedonSupplementaryVideos3and4,
we performed the WAIR and tumbling maneuvers in experiments. The
WAIR, shown in Fig.
8
c, was performed on a 45° upwards slope, which
the robot cannot climb with the UGV or legged modes. The robot was
initialized in the MIP con
fi
guration, then the wheel motors and thrus-
ters worked together to propel the robot up the incline. The upper
thrusters stabilized the robot
’
s upper body tilt angle, and the wheel
motors set the robot
’
s forward speed on the slope. The tumbling
maneuver, shown in Fig.
8
d, utilized the same MIP uprighting man-
euver shown in Fig.
5
btolifttherobot
’
s front side, drive forward, and
vault over a large obstacle that the robot is unable to roll or walk over it
(Fig.
8
d1 to Fig.
8
d4). Then, the same maneuver was performed to lift
the robot
’
s backside, then,
fi
nally, the robot transformed back into
UGV mode (Fig.
8
d5 to Fig.
8
d8).
Discussions
We have presented M4 and showcased the advantages of considering
morpho-functional appendages that can be repurposed to manipulate
redundancy to enhance locomotion plasticity and achieve payload
scalability. A few works that previously applied appendage repurposing
in their designs achieved limited locomotion plasticity. Instead, in this
paper, we demonstrated that our robot can (1)
fl
y, (2) roll, (3) walk, (4)
crouch, (5) balance, (6) tumble, (7
) scout, and (8) loco-manipulate
objects by switching the functionality of appendages between wheels,
legs, hands, or thrusters. In addition, we demonstrated M4 can drive on
steep slopes and vault over large
obstacles if other modes were not
applicable. We showed M4
’
s design is scalable and can substantiate fully
autonomous, self-contained, multi-modal operations. This modal
diversity and level of autonomy have not been reported in multi-modal
locomotion before and differentiate
s our robot from existing platforms.
The access to our wide array of actuators and locomotion modes
allows the robot to choose the most ef
fi
cient mode of locomotion
a
b
Ground path tracking
t = 155s
t = 190s
robot
obstacles
goal
target
path
path
taken
unreachable
goal by UGV
t = 205s
t = 196s
c
d
take-off
t = 245s
landing
e
Start
Goal
Transform to
Aerial Mode
Transform to
Ground Mode
Nvidia Jetson
Nano
Intel RealSense
D455
f
Fig. 7 | Autonomous and self-contained MM-PRM path planning. a
Shows the
online ground waypoint generation using MM-PRM and A
*
algorithm and the
robot
’
s ground trajectory tracking. The point cloud data captured by the Intel
RealSense camera
fi
xated in the front of M4 is processed in real-time by the Jetson
Nano computer for autonomous path planning and navigation.
b
Illustrates a
waypoint that is unreachable by the UGV mode.
c
Shows M4
’
s autonomous
transformation into UAS mode and
fl
ight over the obstacle to reach a desired
waypoint.
d
Once the robot lands on the other side of the obstacles, it transforms
into UGV mode and navigates towards the
fi
nal waypoint.
e
Depicts the whole multi-
modal path taken by M4 from the start to goal point.
f
A composite image showing
the path taken by the robot as it autonomously navigates the cluttered environ-
ment and switches from UGV to UAS modes.
Article
https://doi.org/10.1038/s41467-023-39018-y
Nature Communications
| (2023) 14:3323
8