Zhiying (Gin) Jiang

@Quinn drew this Don't Starve character of me

Hi, I’m Gin, a researcher and builder interested in how representations shape both machine learning and human learning.

I currently work on decentralized AI at Bagel Labs, where I lead Project Paris and the development of a decentralized diffusion framework. Our goal is to train generative models through independently trained experts on distributed, heterogeneous, and consumer-grade hardware. We began with text-to-image generation, expanded to text-to-video, and are now exploring action-conditioned world models for robotics and games.

Previously, I co-founded AFAIK.io, a personalized learning platform from the NextAI 2024 cohort. We built knowledge graphs and adaptive learning tools to make systematic, reliable learning more accessible.

I earned my PhD in Computer Science from the University of Waterloo. My thesis, Less is More: Restricted Representations for Better Interpretability and Generalizability, explored how constrained representations can improve generalization and interpretability.

Across my work, I keep returning to one idea: good representations remove accidental complexity while preserving the structure needed to reason, generalize, and create.

Outside machine learning, I’m interested in neuroscience, physics, architecture, film, violin, and food science.

Selected Publications

CVPR2026

Heterogeneous Decentralized Diffusion Models

Jiang, Zhiying, Seraj, Raihan, Villagra, Marcos, and Roy, Bidhan

In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2026

Abs arXiv HTML PDF Supp

Training frontier-scale diffusion models often requires substantial computational resources concentrated in tightly-coupled clusters, limiting participation to well-resourced institutions. While Decentralized Diffusion Models (DDM) enable training multiple experts in isolation, existing approaches require 1176 GPU-days and homogeneous training objectives across all experts. We present an efficient framework that dramatically reduces resource requirements while supporting heterogeneous training objectives. Our approach combines three key contributions: (1) a heterogeneous decentralized training paradigm that allows experts to use different objectives (DDPM and Flow Matching), unified at inference time without any retraining; (2) pretrained checkpoint conversion from ImageNet-DDPM to Flow Matching objectives, accelerating convergence and enabling initialization without objective-specific pretraining; and (3) PixArt-α’s efficient AdaLN-Single architecture, reducing parameters while maintaining quality. Experiments on LAION-Aesthetics show that, relative to the training scale reported for prior DDM work, our approach reduces the compute by 16\times and data by 14\times. Under aligned inference settings, our heterogeneous configuration achieves better FID and higher intra-prompt diversity than the homogeneous baseline. By eliminating synchronization requirements and enabling mixed DDPM/FM objectives, our framework makes decentralized generative model training accessible to contributors with single GPUs requiring only 24–48GB VRAM.
Tech Report

Paris: A Decentralized Trained Open-Weight Diffusion Model

Jiang, Zhiying, Seraj, Raihan, Villagra, Marcos, and Roy, Bidhan

arXiv preprint arXiv:2510.03434 2025

Abs arXiv

We present Paris, the first publicly released diffusion model pre-trained entirely through decentralized computation. Paris demonstrates that high-quality text-to-image generation can be achieved without centrally coordinated infrastructure. Paris is open for research and commercial use. Paris required implementing our Distributed Diffusion Training framework from scratch. The model consists of 8 expert diffusion models (129M-605M parameters each) trained in complete isolation with no gradient, parameter, or intermediate activation synchronization. Rather than requiring synchronized gradient updates across thousands of GPUs, we partition data into semantically coherent clusters where each expert independently optimizes its subset while collectively approximating the full distribution. A lightweight transformer router dynamically selects appropriate experts at inference, achieving generation quality comparable to centrally coordinated baselines. Eliminating synchronization enables training on heterogeneous hardware without specialized interconnects. Empirical validation confirms that Paris’s decentralized training maintains generation quality while removing the dedicated GPU cluster requirement for large-scale diffusion models. Paris achieves this using 14\times less training data and 16\times less compute than the prior decentralized baseline.
ACL2023

“Low-Resource” Text Classification: A Parameter-Free Classification Method with Compressors

Jiang, Zhiying, Yang, Matthew, Tsirlin, Mikhail, Tang, Raphael, Dai, Yiqin, and Lin, Jimmy

In Findings of the Association for Computational Linguistics (ACL) 2023

Abs PDF

Deep neural networks (DNNs) are often used for text classification due to their high accuracy. However, DNNs can be computationally intensive, requiring millions of parameters and large amounts of labeled data, which can make them expensive to use, to optimize, and to transfer to out-of-distribution (OOD) cases in practice. In this paper, we propose a non-parametric alternative to DNNs that’s easy, lightweight, and universal in text classification: a combination of a simple compressor like gzip with a k-nearest-neighbor classifier. Without any training parameters, our method achieves results that are competitive with non-pretrained deep learning methods on six in-distribution datasets.It even outperforms BERT on all five OOD datasets, including four low-resource languages. Our method also excels in the few-shot setting, where labeled data are too scarce to train DNNs effectively. Our code is available at https://github.com/bazingagin/npc_gzip.
ACL2023

What the DAAM: Interpreting Stable Diffusion Using Cross Attention

Tang, Raphael, Liu, Linqing, Pandey, Akshat, Jiang, Zhiying, Yang, Gefei, Kumar, Karun, Stenetorp, Pontus, Lin, Jimmy, and Ture, Ferhan

In Proceedings of Association for Computational Linguistics (ACL), Best Paper Award, 2023

Abs

Diffusion models are a milestone in text-to-image generation, but they remain poorly understood, lacking interpretability analyses. In this paper, we perform a text-image attribution analysis on Stable Diffusion, a recently open-sourced model. To produce attribution maps, we upscale and aggregate cross-attention maps in the denoising module, naming our method DAAM. We validate it by testing its segmentation ability on nouns, as well as its generalized attribution quality on all parts of speech, rated by humans. On two generated datasets, we attain a competitive 58.8-64.8 mIoU on noun segmentation and fair to good mean opinion scores (3.4-4.2) on generalized attribution. Then, we apply DAAM to study the role of syntax in the pixel space across head–dependent heat map interaction patterns for ten common dependency relations. We show that, for some relations, the head map consistently subsumes the dependent, while the opposite is true for others. Finally, we study several semantic phenomena, focusing on feature entanglement; we find that the presence of cohyponyms worsens generation quality by 9%, and descriptive adjectives attend too broadly. We are the first to interpret large diffusion models from a visuolinguistic perspective, which enables future research. Our code is at https://github.com/castorini/daam.
NeurIPS2022

Few-Shot Non-Parametric Learning with Deep Latent Variable Model

Jiang, Zhiying, Dai, Yiqin, Xin, Ji, Li, Ming, and Lin, Jimmy

In Proceedings of the 36th Conference on Neural Information Processing Systems (NeurIPS) Spotlight. 2022

Abs PDF

Most real-world problems that machine learning algorithms are expected to solve face the situation with 1) unknown data distribution; 2) little domain-specific knowledge; and 3) datasets with limited annotation. We propose Non-Parametric learning by Compression with Latent Variables (NPC-LV), a learning framework for any dataset with abundant unlabeled data but very few labeled ones. By only training a generative model in an unsupervised way, the framework utilizes the data distribution to build a compressor. Using a compressor-based distance metric derived from Kolmogorov complexity, together with few labeled data, NPC-LV classifies without further training. We show that NPC-LV outperforms supervised methods on all three datasets on image classification in low data regime and even outperform semi-supervised learning methods on CIFAR-10. We demonstrate how and when negative evidence lowerbound (nELBO) can be used as an approximate compressed length for classification. By revealing the correlation between compression rate and classification accuracy, we illustrate that under NPC-LV, the improvement of generative models can enhance downstream classification accuracy.
BlackBoxNLP

How Does BERT Rerank Passages? An Attribution Analysis with Information Bottlenecks

Jiang, Zhiying, Tang, Raphael, Xin, Ji, and Lin, Jimmy

In Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP 2021

Abs PDF

Fine-tuned pre-trained transformers achieve the state of the art in passage reranking. Unfortunately, how they make their predictions remains vastly unexplained, especially at the end-to-end, input-to-output level. Little known is how tokens, layers, and passages precisely contribute to the final prediction. In this paper, we address this gap by leveraging the recently developed information bottlenecks for attribution (IBA) framework. On BERT-based models for passage reranking, we quantitatively demonstrate the framework’s veracity in extracting attribution maps, from which we perform detailed, token-wise analysis about how predictions are made. Overall, we find that BERT still cares about exact token matching for reranking; the [CLS] token mainly gathers information for predictions at the last layer; top-ranked passages are robust to token removal; and BERT fine-tuned on MSMARCO has positional bias towards the start of the passage.
EMNLP2020

Inserting Information Bottleneck for Attribution in Transformers

Jiang, Zhiying, Tang, Raphael, Xin, Ji, and Lin, Jimmy

In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings 2020

Abs PDF

Pretrained transformers achieve the state of the art across tasks in natural language processing, motivating researchers to investigate their inner mechanisms. One common direction is to understand what features are important for prediction. In this paper, we apply information bottlenecks to analyze the attribution of each feature for prediction on a black-box model. We use BERT as the example and evaluate our approach both quantitatively and qualitatively. We show the effectiveness of our method in terms of attribution and the ability to provide insight into how information flows through layers. We demonstrate that our technique outperforms two competitive methods in degradation tests on four datasets. Code is available at https://github.com/bazingagin/IBA.
EMNLP2020

Document Ranking with a Pretrained Sequence-to-Sequence Model

Nogueira, Rodrigo*, Jiang, Zhiying*, Pradeep, Ronak, and Lin, Jimmy

In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings 2020

Abs PDF

This work proposes the use of a pretrained sequence-to-sequence model for document ranking. Our approach is fundamentally different from a commonly adopted classification-based formulation based on encoder-only pretrained transformer architectures such as BERT. We show how a sequence-to-sequence model can be trained to generate relevance labels as “target tokens”, and how the underlying logits of these target tokens can be interpreted as relevance probabilities for ranking. Experimental results on the MS MARCO passage ranking task show that our ranking approach is superior to strong encoder-only models. On three other document retrieval test collections, we demonstrate a zero-shot transfer-based approach that outperforms previous state-of-the-art models requiring in-domain cross-validation. Furthermore, we find that our approach significantly outperforms an encoder-only architecture in a data-poor setting. We investigate this observation in more detail by varying target tokens to probe the model’s use of latent knowledge. Surprisingly, we find that the choice of target tokens impacts effectiveness, even for words that are closely related semantically. This finding sheds some light on why our sequence-to-sequence formulation for document ranking is effective. Code and models are available at pygaggle.ai.