UMN VIP

Database

• GQA-REX database. A database with over 1M multi-modal explanations of decisions with reasoning processes and grounded keywords in images. Chen et al. CVPR [pdf] [bib]
• AiR-D database. An reasoning-aware eye-tracking database under visual question answering. Chen et al. CVPR [pdf] [bib]
• IQVA database. A large-scale attention database on 360° videos under question-answering. Jiang et al. CVPR [pdf] [bib]
• SALICON database. Saliency in Context - a large-scale attention database on MS COCO images. Jiang et al. CVPR [pdf] [bib]
• Video Story database. A dataset for generating text story/summarization for videos containing social events. Li et al. TMM [pdf] [bib]
• EMOd database. A database with rich sentiment and semantic attributes. Fan et al. CVPR [pdf] [bib]
• HII database. A dataset of images with four types of interactions: hand shake, high five, hug, kiss. Li et al. ACMMM [pdf] [bib]
• PISC database. An image dataset consisting of 22,670 images of 9 types of social relationships. Li et al. ICCV [pdf] [bib]
• OSIE database. Object and Semantic Images and Eye-tracking database - a database for object and semantic saliency (700 images, 5551 objects with fine contour and semantic attribute labeling). Xu et al. JoV [pdf] [bib]
• EyeCrowd database. Eye Fixations in Crowd database - a database for saliency in crowd. Jiang et al. ECCV [pdf] [bib]
• FiWI database. Fixations in Webpage Images database - a database for webpage saliency. Shen and Zhao ECCV [pdf] [bib]
• EyeScene database. A database for fixations in various scene types (500 natural images, representing a variety of common outdoor environment from SUN2012 database).
• ET-PASCAL-MSRC dababase. Eye Tracking in PASCAL VOC07 and MSRC-21 database - a database for joint attention, segmentation, and recognition. Jiang et al. BMVC [pdf]

Code

• REX: Reasoning-Aware and Grounded Explanation. Code for explanation generation that encodes grounding and reasoning.
• Query and Attention Augmentation for Knowledge-Based Explainable Reasoning. Code for explainable visual reasoning with query and attention augmentation to incorporate knowledge.
• Explicit Knowledge Incorporation for Visual Reasoning. Code for explainable visual reasoning with neural modules and explict knowledge incorporation.
• Learning to Predict Trustworthiness with Steep Slope Loss. Code and pre-trained trustworthiness predictors with steep slope loss.
• Predicting Human Scanpaths in Visual Question Answering. Code for predicting human visual scanpaths in visual question answering, free-viewing, and visual search tasks.
• AiR: Attention with Reasoning Capability. Code for implementing attention and reasoning supervision on the AiR-D dataset.
• Immersive Question-directed Visual Attention. Code for implementing correctness-aware attention prediction on the IQVA dataset.
• Direction Concentration Learning. Code for enhancing congruency (i.e., agreement between the learned knowledge and the new information in a learning process) in classification and continual learning.
• Learning to Learn from Noisy Labeled Data. Code for performing noise-tolerant learning.
• Attention-based Autism Spectrum Disorder Screening with Privileged Modality. Code for performing ASD screening on the publicly available Saliency4ASD dataset.
• Context-Adaptive Saliency Network. Code for network implementation of context-adaptive saliency prediction, and evaluation metrics.
• Saliency with Objects and Semantics. Code for object and semantic feature computation, model training with SVM, saliency prediction, and evaluation measures.
• Saliency in Crowd. Code for crowd feature computation, crowd stats calculation, model training with MKL, saliency prediction, and evaluations.
• Multi-Layer Sparse Network. Code for multi-layer sparse network, model training, and saliency prediction.
• Subtle Cueing. Experiment stimuli and system source code.
• Saliency Prediction with Scene Structural Guidance. Code for low and structural level feature computation, model training with MKL and saliency prediction.