Autonomous Agents from Automatic Reward Modeling and Planning

Zhenfang Chen†, Delin Chen†, Rui Sun†, Wenjun Liu†, Chuang Gan

• A new framework for LLM-based agents using automatic reward models.
• We build effective, scalable, and controllable reward models to integrate Reflexion and MCTS.

FlexAttention for Efficient High-Resolution Vision-Language Models

Junyan Li, Delin Chen, Tianle Cai, Peihao Chen, Yining Hong, Zhenfang Chen, Yikang Shen, Chuang Gan

Paper | GitHub | Project Page |

• FlexAttention is a plug-and-play attention module that can enhance VLMs’ ability to perceive details in high resolution image in an efficient way.

CoVLM: Composing Visual Entities and Relationships in Large Language Models Via Communicative Decoding

Junyan Li, Delin Chen, Yining Hong, Zhenfang Chen, Peihao Chen, Yikang Shen, Chuang Gan

Paper | GitHub | Project Page|

• CoVLM is specifically designed to guide the VLM to explicitly compose visual entities and relationships among the text and dynamically communicate with the vision encoder and detection network to achieve vision-language communicative decoding. It boosts the compositional reasoning ability of VLMs and achieve SoTA performance on various tasks involving compositional analysis.

Scratch Each Other’s Back: Incomplete Multi-Modal Brain Tumor Segmentation via Category Aware Group Self-Support Learning

Yansheng Qiu†, Delin Chen†, Hongdou Yao, Yongchao Xu, Zheng Wang

Paper

• We proposed Group Self-Support Learning framework to utilize the dominating characteristics of modalities to direct the distillation of mutual knowledge between modalities without expanding the complexity of the baseline network. The result obtained SOTA on BraTs 2015, 2018 and 2020 datasets.

TransRef: Multi-Scale Reference Embedding Transformer for Reference-Guided Image Inpainting

Liang Liao†, Taorong Liu†, Delin Chen, Jing Xiao, Zheng Wang, Chia-Wen Lin, and Shin’ichi Satoh

Paper | GitHub

• Transref utilizes a reference image that depicts a scene similar to the corrupted image to facilitate image inpainting.