CHI 2022 Conference Highlights

9 min readJul 19, 2022

The CHI 2022 conference took place last month in the vibrant city of New Orleans, Louisiana. It was a hybrid event — almost 50% of the participants joined in person, and the rest joined virtually. With my research interests spanning visualization, NLP, and human-AI interaction, I was looking forward to my first in-person visit to CHI. And it didn’t disappoint. I came back from the conference energized after seeing presentations of high-quality research work, coming across thought-provoking ideas, and syncing up with outstanding researchers.

In this blog post, I summarize my experience attending CHI, provide an overview of the keynotes and awards, and summarize several interesting papers on human-AI interaction, mixed-initiative system design, and visualization. Human-centered AI is a key research area of Megagon Labs where we explore challenges related to scalability, usability, and explainability in diverse projects such as data integration, natural language generation, and knowledge graphs. Therefore any research work at the intersection of NLP, data management, and HCI is of significant interest to us.

My Experience at CHI 2022 @ NOLA

I once read somewhere that science is a social process. That’s an observation I agree with entirely. Conferences provide the opportunity to connect with peers, meet new folks, and discuss exciting ideas and projects. Therefore, I was looking forward to going to New Orleans after two years of attending conferences virtually. In my view, the face-to-face discussions in the in-person setting felt more spontaneous and inspiring than the virtual Gather discussions.

I presented a paper on understanding the challenges data science workers face in performing information extraction tasks in the “Reasoning and Sensemaking” session on the final day of the conference. More details can be found in my previous blog post. Surprisingly, our session chair was remote while all the presenters were present in the conference room. The hybrid setup and minor technical issues aside, my overall experience at CHI was fantastic. The organizers did a phenomenal job in organizing and managing a hybrid event.

Overview of Papers, Keynotes, and Awards

First, I will highlight a few exciting papers on human-AI interaction, mixed-initiative systems, and data visualization. Then, I will briefly discuss the keynotes and awards, which often reflect what the community is thinking about and deems to be important topics and issues.

Papers

CHI welcomes diverse research topics with papers presented in several parallel sessions. So I had to prioritize which sessions to attend. Here, I will highlight papers/sessions that I found to be most interesting and relevant to our work at Megagon Labs. I have listed the papers with their ACM digital library link, which contains video presentations of the work.

Human-AI Interaction

Storytelling. Two papers related to collaborative writing/storytelling explored GPT-3’s capabilities as a writing collaborator. One of the papers, TaleBrush, introduced a generative story ideation tool that uses line sketching techniques to control the highs and lows of a character’s fortune in a story co-created with GPT-3. This work aimed to evaluate a pipeline for controlling the story generation process within the human-AI collaboration setting. Another work in a similar vein, CoAuthor, introduces a dataset of 1445 human-AI (GPT-3) collaborative writing sessions. The goal of this paper was to initiate discussions regarding the pros and cons of utilizing generative language models such as GPT-3. However, one common theme for both of these papers was a lack of an in-depth discussion related to potential guardrails that we may need to put into practice to counter racial or gender bias and insensitive content in the generated text. During their presentation, the CoAuthor paper authors briefly mentioned using stopwords as guardrails in their work. However, I would have liked to see more discussions around this topic, especially at a conference like CHI, where algorithm auditing, equality, and inclusivity are highly important. Here are the paper links:

Prompting Language Models. The recent research trend around prompting language models was explored in several papers at this year’s CHI conference. One of the projects studied how users interacted with a tool called GenLine, which leveraged language model prompting to generate code. However, the study found that practitioners struggled with prompting for code generation and felt the need to augment their interaction to obtain the desired outcomes. During the Q&A session, there was a very lively discussion on the degree to which user behavior should change to adapt to the system vs. how much a system should change for the user. The paper can be found here:

Discovering the Syntax and Strategies of Natural Language Programming with Generative Language Models

Besides the aforementioned work, researchers at Google presented the idea of “chain of thought” prompting to elicit desired content from language models. In a chain, a problem can be broken down into a number of smaller sub-tasks. Each sub-task can be mapped to a distinct step with a corresponding natural language prompt. Then, results of one or more previous steps can be fed into the next step in aggregated form as an input prompt. The rationale behind chaining is that the process enables users to run the same model on multiple sub-tasks, thereby ensuring each sub-task has a higher likelihood of achieving the desired outcome. Here is a blog post that explains the idea in detail. In this work, the researchers explored whether humans can successfully operate within this chain-of-thought prompt setting and showed such a strategy might help calibrate model outcomes and debug unexpected model outputs. The paper can be found here:

AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Prompts

Another work explored the text-to-image generation problem. Researchers studied how to effectively prompt generative models with style and subject keywords to generate artistic images and proposed several design guidelines:

Design Guidelines for Prompt Engineering Text-to-Image Generative Models

Demystifying Models. The theme of model explainability and transparency was omnipresent in all the sessions involving AI agents. To this end, Shared Interest proposed a framework to explore/explain model behaviors in various tasks such as image classification and sentiment analysis. The key idea is to leverage a set of metrics to align human preference with a model’s preference for the tasks on computer vision and NLP mentioned earlier. Model preference is captured via saliency, i.e., which features in the input are weighted more, and human preference is collected via annotations. The work is also very relevant to the question of validating the outcomes of AI agents. The paper can be found here:

Shared Interest: Measuring Human-AI Alignment to Identify Recurring Patterns in Model Behavior

The following are some additional pointers related to the work: project page, NLP demo, and vision demo. As I conclude the human-AI interaction discussion, I also wanted to highlight the “Human-centered Explainable AI” workshop at CHI which tackled important questions related to the study of explanation.

Photo courtesy: Vera Liao (source: twitter)

Mixed-initiative Systems

The paper I found most interesting at this year’s CHI was Symphony by researchers at Apple. The project is not open source, though. Symphony is a framework for composing interactive ML interfaces with task-specific, data-driven components that can be used across platforms such as computational notebooks and web dashboards. The extensible and adaptive component-driven design of the tool makes it very appealing. The extensibility is achieved through wrappers. These wrappers have two primary functions — first, passing data from a platform to Symphony in the correct format, and second, rendering Symphony components in the platform’s UI. Other exciting features include a persistent toolbar to structure users’ workflow and automate their unwanted workload. While Symphony seems to draw from existing formalization in visualization literature, it is not backed by any formal grammar. Our work on LEAM, presented more than a year ago at CIDR’21 (and surprisingly missing from the citations), explored similar ideas in the context of text analysis workflows. We proposed a formal grammar for visual interactive text analysis. We also presented ideas such as metadata and interaction state management which Symphony explored. Nevertheless, Symphony is an important work that opens the door to ongoing and exciting research. Here is the link to the paper:

Symphony: Composing Interactive Interfaces for Machine Learning

Regarding mixed-initiative systems, one work explores the merits of the automated agents in the context of 3D touch interactive techniques and highlights challenges related to agency and expressiveness. The tension between human agency and automation is long-discussed in HCI research and poses vital challenges for designing and engineering mixed-initiative systems. How can we effectively integrate automated reasoning into interactive systems without impeding human agency? Then there is the question of trustworthiness. Another work explored how people trust and rely on an AI assistant in an object identification task. The work demonstrates that people can detect and rely on an AI assistant that performs both expertly and non-expertly in different parts of the same task. The papers can be found here:

Last but not least… visualizations!

There were several very interesting data visualization and analytics papers that I missed due to conflicts/overlaps with other sessions. However, I would love to explore the following papers in detail sometime soon and wanted to highlight them here:

Now let’s briefly discuss the keynote talks and awards next.

Keynotes

The opening keynote at CHI 2022 was delivered by Kishonna Gray who talked about hybrid communities that online gamers form on various platforms such as Discord, Twitter, and Twine. More specifically, Kishonna talked about how gamers communicate, innovate, and create on these platforms and how the dynamics within such communities may help reimagine the future in an increasingly digital world.

“Gaming could be the model for what tech should be.” — Kishonna Gray. Photo courtesy: Andrés Lucero

I have personal experience engaging with such communities by playing online games like FIFA. However, those experiences revolved more around toxicity and how in-game settings almost felt like gambling. Here is a link to the talk.

The closing keynote resonated with me as I am originally from the global south (Bangladesh). Payal Arora is doing fantastic work on empowering women in the South Asian community. Her talk called attention to the fact that we need to recognize the varied dynamics of systemic inequality, shaped by race, caste, gender, and religion, as we develop technological solutions. For example, her talk pointed to US-centric industries and academia despite the US having only a small percentage of internet users. Often, papers on the global south are questioned on the ground of generalizability to, say, the US.

Awards

In CHI 2022, there were several (25) best paper award winners. A more detailed list can be found here. There were three dissertation award winners also. I would specifically like to highlight one of the winners, Fred Hohman (Georgia Tech.) Fred’s research focused on machine learning interpretability related to scaling for deep learning, communicating explanations, and investigating emerging practices. His dissertation spans both visualization and ML and proposes a number of open-source interactive interfaces, scalable algorithms, and novel communication paradigms. In my view, this recognition highlights that human-centered AI is of significant interest to the HCI community. As a researcher working on this topic at Megagon Labs, I am very excited about the prospect of seeing a lot more amazing work in the coming years! Please, check out Fred’s award talk here.

And with that, it’s a wrap. I hope the readers will find some of the highlighted papers interesting.