DeliGrasp (2024, Paper, Website)

Large language models (LLMs) can provide rich physical descriptions of most worldly objects, allowing robots to achieve more informed and capable grasping. We leverage LLMs' common sense physical reasoning and code-writing abilities to infer an object's physical characteristics—mass m, friction coefficient µ, and spring constant k—from a semantic description, and then translate those characteristics into an executable adaptive grasp policy. Using a current-controllable, two-finger gripper with a built-in depth camera, we demonstrate that LLM-generated, physically-grounded grasp policies outperform traditional grasp policies on a custom benchmark of 12 delicate and deformable items including food, produce, toys, and other everyday items, spanning two orders of magnitude in mass and required pick-up force. We also demonstrate how compliance feedback from DeliGrasp policies can aid in downstream tasks such as measuring produce ripeness.

Modified Code as Policies (2022, repo, report)

• Imbued capability awareness and uncertainty into Code as Policies (Liang et al. 2022) system of human interaction for generated robot code-policy.
• Deployed with GPT-3 Codex model and UR5 robot arm in PyBullet simulation.
• For COMS 6998: Topics in Robot Learning, taught by Shuran Song
• Team: David Palumbo, Joanne Wang, William Xie

RollControl (2022, website, report)

• Developed, in team of four students, novel assistive walking device mechanisms: modular handles, eddy current brake powered resistive wheels, and variable, controllable weight support
• Utilized FEA and dynamics analysis in SolidWorks, Inventor CAM, waterjetting, CNC milling, laser cutting, 3DP, and 20+ prototype iterations over 2 semesters to create product
• Leverages COTS push down rollator
• For 2021-2022 Columbia Mechanical Engineering Senior Design
• Team: Daniel Kwon, Delphine Lepeintre, William Xie, Eric Xue

VentCU (2020, website, repo)

• Developed, in team of five students, open-source, fully COTS solution; purchasable with under $900 and able to be assembled within 2 hours
• Awarded $8000 to engineer rapid response, emergency ventilator for COVID-19 patients out of 80 proposals
• Implemented real-time performant control system and user interface in Python, PyQt with a Raspberry Pi 3
• Team: Noah Silverstein, Delphine Lepeintre, William Xie, David Kao, Eric Xue, Neil Nie, Jonathan Sanabria

Collaborations with Mimi Park

• Collaborated with sculptor Mimi Park for gallery show to create bespoke kinetic sculptures
• Co-ideated, designed, and crafted autonomous cars with Arduinos, DC motors, feathers, pipe cleaner, chimes, and steel wireframe
• Shown at Lubov in Spring 2022, as reviewed in Hyperallergic

• Developed walking, organically shaped quadrupedal robot resembling the Venezuelan Poodle Moth
• CAD Model

Full portfolio via PDF: