Human Aligned Reward Modeling for Automated Transfer Function Generation of 3D Rendering of Medical Image Data

NLDL 2025 Conference Submission15 Authors

Keywords: Direct Volume Rendering, Transfer Function, Reinforcement Learning from Human Feedback

Abstract: In recent years, the quality of medical image data, such as computed tomography or magnetic resonance tomography, has continued to improve and the resolution and detection of the smallest structures has become increasingly accurate. Along with these developments, new techniques for three-dimensional visualization using volume rendering techniques are emerging, enabling extremely realistic visualization of medical images. This helps to improve patient communication, diagnosis, and treatment planning. An extremely critical step in the development of a realistic rendering is the design of a suitable transfer function. However, this requires a high level of experience and manual fine-tuning to the given image data. To automatize this process, we propose to train a reinforcement learning agent that extracts a two-dimensional transfer function from the given joint histograms of the image data. The focus of this study is primarily on the development of a suitable reward model, which is critical for the reinforcement learning framework, incorporating human feedback.

Submission Number: 15