University of Hamburg

Will multimodal large language models ever achieve deep understanding of the world?
Despite impressive performance in various tasks, large language models (LLMs) are subject to the symbol grounding problem, so from the …
Cite DOI Article
Pointing-Guided Target Estimation via Transformer-Based Attention
Deictic gestures, like pointing, are a fundamental form of non-verbal communication, enabling humans to direct attention to specific …
Cite DOI code
LLM-based Interactive Imitation Learning for Robotic Manipulation
Recent advancements in machine learning provide methods to train autonomous agents capable of handling the increasing complexity of …
Cite DOI code
Comparing Apples to Oranges: LLM-Powered Multimodal Intention Prediction in an Object Categorization Task
Human intention-based systems enable robots to perceive and interpret user actions to interact with humans and adapt to their behavior …
Cite DOI
Open-Vocabulary Robotic Object Manipulation using Foundation Models
Classical vision-language-action models are limited by unidirectional communication, hindering natural human-robot interaction. The …
PDF Cite DOI
Diffusing in Someone Else’s Shoes: Robotic Perspective-Taking with Diffusion
Humanoid robots can benefit from their similarity to the human shape by learning from humans. When humans teach other humans how to …
Cite DOI
Robots Can Multitask Too: Integrating a Memory Architecture and LLMs for Enhanced Cross-Task Robot Action Generation
Large Language Models (LLMs) have been recently used in robot applications for grounding LLM commonsense reasoning with the robot’s …
Cite DOI
When Robots Get Chatty: Grounding Multimodal Human-Robot Conversation and Collaboration
We investigate the use of Large Language Models (LLMs) to equip neural robotic agents with human-like social and cognitive …
Cite DOI
Robotic Imitation of Human Actions
Imitation can allow us to quickly gain an understanding of a new task. Through a demonstration, we can gain direct knowledge about …
Cite DOI
NICOL: A Neuro-inspired Collaborative Semi-humanoid Robot that Bridges Social Interaction and Reliable Manipulation
Robotic platforms that can efficiently collaborate with humans in physical tasks constitute a major goal in robotics. However, many …
PDF Cite DOI IEEE Xplore