Zhiying (Gin) Jiang

@Quinn drew this Don't Starve character of me

Hi, I’m Gin, a researcher with a passion for understanding and improving both machine learning and human learning.

I’m the co-founder of AFAIK.io (NextAI 2024 cohort). AFAIK is a personalized learning platform that aims at addressing inequalities in access to higher education. Our platform enables anyone to systematically learn anything in-depth at their own level without concerns about hallucinations or misinformation.

Before founding AFAIK, I earned my PhD from the University of Waterloo, where I completed my degree in about 3.5 years under the mentorship of Professor Jimmy Lin and in collaboration with Professor Ming Li. Prior to that, I spent four years at Rensselaer Polytechnic Institute, conducting research at Blender under the guidance of Professor Heng Ji.

My research focuses on the interpretability and generalizability of machine learning models, with a strong interest in the intersection of information theory and learning. I’m deeply inspired by the idea that compression lies at the heart of both human and machine learning, guiding my exploration of fundamental, theory-driven approaches.

Beyond machine learning, I love neuroscience, physics, and food science.

Selected Publications

ACL2023

“Low-Resource” Text Classification: A Parameter-Free Classification Method with Compressors

Jiang, Zhiying, Yang, Matthew, Tsirlin, Mikhail, Tang, Raphael, Dai, Yiqin, and Lin, Jimmy

In Findings of the Association for Computational Linguistics (ACL) 2023

Abs PDF

Deep neural networks (DNNs) are often used for text classification due to their high accuracy. However, DNNs can be computationally intensive, requiring millions of parameters and large amounts of labeled data, which can make them expensive to use, to optimize, and to transfer to out-of-distribution (OOD) cases in practice. In this paper, we propose a non-parametric alternative to DNNs that’s easy, lightweight, and universal in text classification: a combination of a simple compressor like gzip with a k-nearest-neighbor classifier. Without any training parameters, our method achieves results that are competitive with non-pretrained deep learning methods on six in-distribution datasets.It even outperforms BERT on all five OOD datasets, including four low-resource languages. Our method also excels in the few-shot setting, where labeled data are too scarce to train DNNs effectively. Our code is available at https://github.com/bazingagin/npc_gzip.
ACL2023

What the DAAM: Interpreting Stable Diffusion Using Cross Attention

Tang, Raphael, Liu, Linqing, Pandey, Akshat, Jiang, Zhiying, Yang, Gefei, Kumar, Karun, Stenetorp, Pontus, Lin, Jimmy, and Ture, Ferhan

In Proceedings of Association for Computational Linguistics (ACL), Best Paper Award, 2023

Abs

Diffusion models are a milestone in text-to-image generation, but they remain poorly understood, lacking interpretability analyses. In this paper, we perform a text-image attribution analysis on Stable Diffusion, a recently open-sourced model. To produce attribution maps, we upscale and aggregate cross-attention maps in the denoising module, naming our method DAAM. We validate it by testing its segmentation ability on nouns, as well as its generalized attribution quality on all parts of speech, rated by humans. On two generated datasets, we attain a competitive 58.8-64.8 mIoU on noun segmentation and fair to good mean opinion scores (3.4-4.2) on generalized attribution. Then, we apply DAAM to study the role of syntax in the pixel space across head–dependent heat map interaction patterns for ten common dependency relations. We show that, for some relations, the head map consistently subsumes the dependent, while the opposite is true for others. Finally, we study several semantic phenomena, focusing on feature entanglement; we find that the presence of cohyponyms worsens generation quality by 9%, and descriptive adjectives attend too broadly. We are the first to interpret large diffusion models from a visuolinguistic perspective, which enables future research. Our code is at https://github.com/castorini/daam.
NeurIPS2022

Few-Shot Non-Parametric Learning with Deep Latent Variable Model

Jiang, Zhiying, Dai, Yiqin, Xin, Ji, Li, Ming, and Lin, Jimmy

In Proceedings of the 36th Conference on Neural Information Processing Systems (NeurIPS) Spotlight. 2022

Abs PDF

Most real-world problems that machine learning algorithms are expected to solve face the situation with 1) unknown data distribution; 2) little domain-specific knowledge; and 3) datasets with limited annotation. We propose Non-Parametric learning by Compression with Latent Variables (NPC-LV), a learning framework for any dataset with abundant unlabeled data but very few labeled ones. By only training a generative model in an unsupervised way, the framework utilizes the data distribution to build a compressor. Using a compressor-based distance metric derived from Kolmogorov complexity, together with few labeled data, NPC-LV classifies without further training. We show that NPC-LV outperforms supervised methods on all three datasets on image classification in low data regime and even outperform semi-supervised learning methods on CIFAR-10. We demonstrate how and when negative evidence lowerbound (nELBO) can be used as an approximate compressed length for classification. By revealing the correlation between compression rate and classification accuracy, we illustrate that under NPC-LV, the improvement of generative models can enhance downstream classification accuracy.
BlackBoxNLP

How Does BERT Rerank Passages? An Attribution Analysis with Information Bottlenecks

Jiang, Zhiying, Tang, Raphael, Xin, Ji, and Lin, Jimmy

In Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP 2021

Abs PDF

Fine-tuned pre-trained transformers achieve the state of the art in passage reranking. Unfortunately, how they make their predictions remains vastly unexplained, especially at the end-to-end, input-to-output level. Little known is how tokens, layers, and passages precisely contribute to the final prediction. In this paper, we address this gap by leveraging the recently developed information bottlenecks for attribution (IBA) framework. On BERT-based models for passage reranking, we quantitatively demonstrate the framework’s veracity in extracting attribution maps, from which we perform detailed, token-wise analysis about how predictions are made. Overall, we find that BERT still cares about exact token matching for reranking; the [CLS] token mainly gathers information for predictions at the last layer; top-ranked passages are robust to token removal; and BERT fine-tuned on MSMARCO has positional bias towards the start of the passage.
EMNLP2020

Inserting Information Bottleneck for Attribution in Transformers

Jiang, Zhiying, Tang, Raphael, Xin, Ji, and Lin, Jimmy

In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings 2020

Abs PDF

Pretrained transformers achieve the state of the art across tasks in natural language processing, motivating researchers to investigate their inner mechanisms. One common direction is to understand what features are important for prediction. In this paper, we apply information bottlenecks to analyze the attribution of each feature for prediction on a black-box model. We use BERT as the example and evaluate our approach both quantitatively and qualitatively. We show the effectiveness of our method in terms of attribution and the ability to provide insight into how information flows through layers. We demonstrate that our technique outperforms two competitive methods in degradation tests on four datasets. Code is available at https://github.com/bazingagin/IBA.
EMNLP2020

Document Ranking with a Pretrained Sequence-to-Sequence Model

Nogueira, Rodrigo*, Jiang, Zhiying*, Pradeep, Ronak, and Lin, Jimmy

In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings 2020

Abs PDF

This work proposes the use of a pretrained sequence-to-sequence model for document ranking. Our approach is fundamentally different from a commonly adopted classification-based formulation based on encoder-only pretrained transformer architectures such as BERT. We show how a sequence-to-sequence model can be trained to generate relevance labels as “target tokens”, and how the underlying logits of these target tokens can be interpreted as relevance probabilities for ranking. Experimental results on the MS MARCO passage ranking task show that our ranking approach is superior to strong encoder-only models. On three other document retrieval test collections, we demonstrate a zero-shot transfer-based approach that outperforms previous state-of-the-art models requiring in-domain cross-validation. Furthermore, we find that our approach significantly outperforms an encoder-only architecture in a data-poor setting. We investigate this observation in more detail by varying target tokens to probe the model’s use of latent knowledge. Surprisingly, we find that the choice of target tokens impacts effectiveness, even for words that are closely related semantically. This finding sheds some light on why our sequence-to-sequence formulation for document ranking is effective. Code and models are available at pygaggle.ai.