What do AI Models Know? A Case Study on Visual Question Answering

A major challenge in recent AI literature is understanding why state-of-the-art deep learning models show great success on a range of datasets while they severely degrade in performance when presented with examples which slightly vary from their training distribution. In this proposal, we will examine this question in the context of visual question-answering, a challenging task which requires models to jointly reason over images and text. We will start our exploration with the GQA dataset, which, along with images and text, also includes a rich semantic scene graph, representing the spatial relations between objects in the image, and thus lends itself to probing through high-quality automatic manipulation. In prelminary work we have augmented GQA with examples that vary slightly from the original questions, and shown that here too high-performing models perform much worse on the augmented questions compared to the original ones. Our proposal will analyze our results, exploring the reasons for the drop in performance, and what makes our new questions more challenging. We also plan to to generalize the reasons we find to other datasets of visual question answering, and more broadly to other AI datasets. Given those insights, we will augment the training set with instances that capture model “blind spots”, in an attempt to improve the model’s generalization ability. Our results will improve our understanding of what state-of-the-art AI models know, what they are still missing, and how can we improve them based on this new understanding.

Goal

To achieve a better understanding of the limitations of current state-of-the-art models and datasets, as well as ways to improve them.

Researchers

What we did

In progress

Results

Forthcoming

Formally Verifying Deep Learning Policies for Computer Systems

Deep machine learning is revolutionizing computer science. Instead of manually creating complex software, engineers now use automatically-generated deep neural networks (DNNs). DNNs are being used in critical financial, medical and autonomous driving systems, obtaining previously unimaginable results. Recently, they are being adopted also in the field of computer systems, replacing hand-crafted algorithms that control key infrastructure, such as adaptive bit rate selection and congestion control algorithms. But despite their remarkable achievements, DNN opacity is a tremendous challenge: DNNs work, but we do not fully understand how or why, and cannot manually prove their correctness. Consequently, there is a crucial need to ensure that DNNs operate correctly. This issue is urgent, as errors have already been observed in modern DNNs: for example, slight perturbations to inputs (“adversarial examples”) that can cause modern DNNs to perform severe misclassification errors. This lack of formal guarantees about DNN behavior is preventing their safe deployment in critical systems.
Recent and exciting developments in the field of formal methods allow us to automatically reason about DNNs. However, this is a nascent technology, and we are just now beginning to tap its full potential. Here, we propose to bring together Prof. Schapira’s expertise in computer networks and Dr. Katz’s expertise in neural network verification, in order to create techniques and tools suitable for the formal verification of learning-based computer networks policies. We will develop novel certification techniques, capable of leveraging common traits of computer network algorithms in order to achieve better flexibility and scalability than general, off-the-shelf verification tools. We will then apply these techniques to state-of-the-art controllers in computer networks. Our research will thus help ensure the safety and correctness of key computer network systems before they are deployed, greatly benefiting programmers, engineers, and users of these systems.

Goal

To combine the recent advances in learning-based controllers for computer systems with those in DNN verification, in order to ensure the robustness and reliability of these controllers.

Researchers

What we did

In progress

Results

Forthcoming

The complexity of social complexity

Developing quantitative multidimensional approaches for studies on social complexity

The evolution of complex behaviours is a major challenge in evolutionary and behavioural biology. The current “Omics” revolution in biology opens new opportunities to study this problem in molecular terms, but this requires the development of new statistical and data analyses tools. We propose to start addressing this challenge by focusing of the evolution of social complexity in insects. Research of social insects such as ants, termites, bees and wasps has provided excellent model systems for developing hypotheses and theories on the evolution of social complexity. These include seminal contributions such as the development of kin selection theory, multi-level selection theory, and the influential idea that the evolution of complexity has progressed through a series of major transitions. Currently, the evolution of social complexity in insects relies on qualitative classifications. We argue that this approach suffers from several significant limitations. These include, lumping together species showing a broad range of social complexity, and falsely implying that social evolution always progresses along a single linear stepwise trajectory that can be deduced from comparing extant species (“rungs on a ladder”). We recently showed that a single species can have both higher and lower levels of social complexity compared to other taxa, depending on the social trait measured. This study proposes a new approach which is based on measuring the complexity of individual key social traits.

Goal

Developing statistical and data analyses tools for studying social complexity within an appropriate phylogenetic framework.

Researchers

What we did

In progress

Results

Forthcoming

Integrating large-scale datasets into a metamodel of multiscale communication networks underlying Alzhimer’s disease progression

Alzheimer’s disease (AD) is a progressive neurodegenerative disease of old age and the most common cause of dementia. Despite great advances made in understanding the pathogenic features leading to AD, we still only partially understand its cause, and currently have no effective treatments and prevention strategies. While AD research has been largely focused on the damage to neuronal cells, accumulating evidence suggests that multiple non-neuronal cell types in the brain are directly involved in the degeneration process. Cellular communication between different types of brain cells are predicted to have a major contribution to AD progression, yet much remains unknown regarding cellular communication networks spanning multiple cell types.
We are interested in expanding the research focus from single cell types to profiling entire cellular environments, aiming to build a model of the cellular cascade leading to AD, i.e. the crosstalk between cell types and the consequent changes in their internal cell states that drives the disease. We are relying on our combined expertise in: applying cutting edge genomics and imaging technologies in the brain to generate large scale datasets with single cell resolution; in machine learning, image analysis, and graph theory approaches to tackle the data analysis challenges; and in integrative modeling of dynamic biological systems across scales using Bayesian modeling approaches.

Goal

To create a comprehensive multi-resolution model that describes the cascade of cellular communications during Alzheimer’s disease (AD) progression

Researchers

What we did

In progress

Results

Forthcoming

Brain wide-cognitive networks of individual mice

There are no two similar brains on earth. But the brain-wide mechanisms of individuality are largely unknown, especially in animal models. In a collaboration between Ariel Gilad’s and Oren Forkosh’s labs, we aim to study brain-wide networks of the mouse, during cognitive and social functions, on the basis of individuality. To do this, we combine state-of-the-art experimental methods along with BIG data computational analysis to compute cognitive and social networks in behaving mice. First, mice will be
monitored in a ‘social box’ and we will implement a computational framework to map the personality of each mouse. Next, using multi-fiber photometry we will simultaneously record from dozens of cortical and subcortical areas as mice interact within the social box and perform complex cognitive tasks. Using BIG data network analysis on dynamic spatiotemporal patterns we will extract network motifs related to the social and cognitive function of each individual mouse.

Goal

To make a substantial scientific impact by collaborating between experimental and computational disciplines

Researchers

What we did

In progress

Results

Forthcoming