This project modularized the classical sensemaking loop by Pirolli and Card, by clarifying the inputs and outputs in each of the five components in sensemaking, so that it can be customized to cater different investigation goals and strategies, design and assign specific tasks to crowd workers. The pipeline allows flexible and dynamic implementation of workflows, expert guidance, monitor and feedback. This provide a broader possibility of collaboration among experts, crowds and algorithms.

This project explores how crowdsourcing can be used to help an intelligence analyst find connections within a large body of text-based evidence. For example, an analyst may have access to dozens of evidence documents, and needs to identify a hidden terrorist plot that links the evidence together. We have developed the concept of “context slices,” in which we intelligently divide up large amounts of text so that transient, novice crowd workers can contribute to solving the bigger mystery. From these ideas, we have developed Connect the Dots, a system that uses crowd workers and natural language processing techniques to build an interactive visualization from textual evidence.

While training a machine learning model, data scientists often need to determine some hyperparameters to set up the model. The values of hyperparameters configure the structure and other characteris- tics of the model and can significantly influence the training result. However, given the complexity of the model algorithms and the training processes, identifying a sweet spot in the hyperparameter space for a specific problem can be challenging. This work characterizes user requirements for hyperparameter tuning and proposes a prototype system to provide model-agnostic support. We conducted interviews with data science practitioners in industry to collect user requirements and identify opportunities for leveraging interactive visual support. HyperTuner is a prototype system that supports hyperparameter search and analysis via interactive visual analytics.

The growing number of attacks on sensitive information stored in company databases has required the need for more advanced data protection systems. A new class of data-centric systems is allowing data security teams in organizations to better detect security risks across distributed data stores. However, data security analysts still face two key challenges when making data protection decisions: information overload from too many protection targets and having to build optimal protection plans given current goals and available resources. In this work, we characterize user classes and requirements in data security teams, and propose a system design that separates the decision-making process into two sub-tasks: identifying what data is worth protecting and building an impactful plan to protect it. We implemented a system prototype through four iterative design and evaluation cycles, by applying user-centered design to this new domain of data security applications.