Seminar

A reinforcement learning agent produces more than sequences of rewards; it gives rise to many notions of behavior, including values, safety properties, bisimulation relations, and behavioral metrics. To define and understand these notions in a unified way, this talk begins by asking how reward sequences become policy evaluation objectives. Discounted sum, max, mean, variance, and the Sharpe ratio can all be described through a common recursive aggregation pattern.

This perspective leads to a broader coalgebraic framework in which predicates, quantities, relations, and metrics are treated uniformly as behavioral structures satisfying fixed-point or post-fixed-point conditions with respect to the system dynamics. The talk will also discuss how coalgebras represent one-step system behavior, and how coalgebra homomorphisms, pullbacks, and pushforwards relate behavioral structures across systems. It will conclude with future directions involving behavior-preserving objectives, approximate abstractions, and stateful agents.

Bio. Yivan Zhang is an Assistant Professor in the Department of Complexity Science and Engineering, Graduate School of Frontier Sciences, The University of Tokyo, and a Visiting Scientist in the Imperfect Information Learning Team at the RIKEN Center for Advanced Intelligence Project. Yivan received a Ph.D. from The University of Tokyo under the supervision of Prof. Masashi Sugiyama. Yivan’s research spans the theory and application of machine learning, including representation learning and reinforcement learning, with a recent focus on algebra and applied category theory in machine learning.