Luis Herranz | Researcher in machine learning, computer vision and multimedia

Welcome to my web page. Here you will find information about my research activities (for recent works please check publications and blog).

My research interests include machine learning (mostly deep learning) applied to computer vision and multimedia, and artificial intelligence in general. I have worked on topics such as generative models (GANs in particular), transfer learning, continual learning, multimodal representations and neural image compression.

In a broader sense, I am increasingly interested in investigating artificial intelligence within its social and environmental context, and their effects and interplays.

Latest news

Oct. 2024. The paper “Enhanced Neural Video Compression for Cloud Gaming Videos with Aligned Frame Generation by Expert Systems With Applications.
Sept. 2024. The paper “A slimmable framework for practical neural video compression” has been accepted by Neurocomputing.
July 2024. The paper “Learned Image Enhancement via Color Naming” has been accepted at ECCV 2024.
May 2024. The paper “Lightweight Deep Exemplar Colorization via Semantic Attention-Guided Laplacian Pyramid” is accepted by IEEE Trans. on Visualization and Computer Graphics.
May 2024. The paper “Palette-based Color Harmonization via Color Naming” is accepted by IEEE Signal Processing Letters.
Jan. 2024. The paper ” Improving the perception of low-light enhanced images” is accepted by Optics Express
Jan. 2024. I joined the Video Processing and Understanding (VPU) group of the Universidad Autónoma de Madrid (UAM).
Dec. 2023. The paper “Task-Switchable Pre-Processor for Image Compression for Multiple Machine Vision Tasks” is accepted by IEEE Trans. CSVT.
Sep. 2023. The paper “MineGAN++: Mining Generative Models for Efficient Knowledge Transfer to Limited Data Domains” is accepted by IJCV
Sep. 2023. The paper “A Palette-based Image Recoloring Framework for Multiple-image Color Consistency” is accepted by Computer Graphics Forum.
Aug. 2023. The paper “Trust your Good Friends: Source-free Domain Adaptation by Reciprocal Neighborhood Clustering” is accepted by IEEE Trans. PAMI.
June 2023. The paper “Casting a BAIT for Offline and Online Source-free Domain Adaptation” is accepted by Computer Vision and Image Understanding.
May 2023. The paper “Chroma Intra Prediction with Lightweight Attention-Based Neural Networks” has been accepted by IEEE Trans. CSVT.
Feb. 2023. Three papers accepted at ICASSP 2023.
Aug. 2022. The paper “A Novel Framework for Image-to-image Translation and Image Compression” has been accepted by Neurocomputing.
July 2022. The paper “Main Product Detection with Graph Networks for Fashion” has been accepted by Multimedia Tools and Applications.
June 2022. The paper “SlimSeg: Slimmable Semantic Segmentation with Boundary Supervision” was accepted at ACM Multimedia 2022, and “Towards Lightweight Neural Network-based Chroma Intra Prediction for Video Coding” at ICIP 2022.
April 2022. Two papers were accepted at CVPR workshops: “Slimmable Video Codec” at Challenge on Learned Image Compression (CLIC 2022), and “Incremental Meta-Learning via Episodic Replay Distillation for Few-Shot Image Recognition” at Continual Learning in Computer Vision (CLVISION 2022)
Jan. 2022. The paper “DCNGAN: A deformable convolution-based GAN with QP adaptation for perceptual quality enhancement of compressed video” has been accepted at ICASSP 2022.
October 2021. The paper “Exploiting the Intrinsic Neighborhood Structure for Source-free Domain Adaptation” has been accepted at NeurIPS 2021.
August 2021. The paper “DVC-P: Deep Video Compression with Perceptual Optimizations” has been accepted at VCIP 2021, and the paper “Spatial Information Refinement for Chroma Intra Prediction in Video Coding” at APSIPA 2021.
July 2021. The paper “Generalized Source-free Domain Adaptation” has been accepted at ICCV 2021.

More news...

July 2021. The paper “ACAE-REMIND for Online Continual Learning with Compressed Feature Replay” has been accepted by Pattern Recognition Letters.
April 2021. Two papers were accepted at CVPR workshops: “DANICE: Domain adaptation without forgetting in neural image compression” at Challenge on Learned Image Compression (CLIC 2021), and “Continual learning in cross-modal retrieval” at Continual Learning in Computer Vision (CLVISION 2021)
March 2021. The paper “Slimmable compressive autoencoders for practical neural image compression” has been accepted at CVPR 2021.
Feb. 2021. The paper “On Implicit Attribute Localization for Zero-Shot Learning” has been accepted by IEEE Signal Processing Letters.
Jan. 2021. The paper “Distributed Learning and Inference with Compressed Images” has been accepted by IEEE Transactions on Image Processing.
Nov. 2020. We were awarded the 2nd prize of the CSIG Natural Science award. 恭喜蒋老师，宋老师，黄老师，叶老师！
Aug. 2020. The paper “Controlling biases and diversity in diverse image-to-image translation” was accepted by Computer Vision and Image Understanding
July 2020. I have been selected for the Ramón y Cajal grant
July 2020. The paper “Generalized Zero-shot Learning with Multi-source Semantic Embeddings for Scene Recognition” was accepted at ACM Multimedia
May 2020. The paper “Mix and match networks: multi-domain alignment for unpaired image-to-image translation” was accepted by the International Journal of Computer Vision (more info).
April 2020. The paper “Generative Feature Replay For Class-Incremental Learning” was accepted at CVPR Workshop on Continual Learning on Computer Vision (CLVISION 2020)
Feb 2020. Two papers (“MineGAN: effective knowledge transfer from GANs to target domains with few images” and “Semantic Drift Compensation for Class-Incremental Learning of Embeddings” ) have been accepted at CVPR 2020
Jan 2020. The paper “Variable Rate Deep Image Compression with Modulated Autoencoders” has been accepted by IEEE Signal Processing Letters.
July 2019. The paper “Scalable and Diverse cross-domain Image Translation” has been accepted at ACM Multimedia 2019.
Nov 2018. The paper “Cross-Modulation Networks For Few-Shot Learning” has been accepted at the NeurIPS Workshop on Meta-Learning (MetaLearn 2018).
Sep 2018. The paper “Learning Effective RGB-D Representations for Scene Recognition” has been accepted by IEEE Transactions on Image Processing (more info).
Sep 2018. The paper “Memory replay GANs: learning to generate new categories without forgetting” has been accepted at NeurIPS 2018 (more info).
July 2018. I started a blog to introduce some of our works.
July 2018. The paper “Transferring GANs: generating images from limited data” has been accepted at ECCV 2018.
April 2018. The paper “Rotate your networks: better weight consolidation and less catastrophic forgetting” was accepted at ICPR 2018 (more info).
Feb 2018. The paper “Mix and match networks: encoder-decoder alignment for zero-pair image translation” was accepted at CVPR 2018 (more info).
July 2017. The paper “Domain-adaptive deep network compression” was accepted at ICCV 2017 (more info).
July 2017. LIUM-CVC submission for WMT17 Multimodal Machine Translation task ranked first in both En→De and En→Fr. Details about the system can be found here.
April 2017. The paper “Combining Models from Multiple Sources for RGB-D Scene Recognition” was accepted for publication at IJCAI 2017.
Mar 2017. The paper “Multi-scale multi-feature context modeling for scene recognition in the semantic manifold” was accepted for publication at IEEE Transactions on Image Processing.
Jan 2017. I joined the Learning and Perception (LAMP) group of the Computer Vision Center, at the campus of the Universitat Autònoma de Barcelona (UAB).
Dec 2016. The paper “Being a Super Cook: Joint Food Attributes and Multi-Modal Content Modeling for Recipe Retrieval and Exploration” was accepted for publication at IEEE Transactions on Multimedia.
Nov 2016. The paper “Depth CNNs for RGB-D scene recognition: learning from scratch better than transferring from RGB-CNNs” was accepted at AAAI 2017 (more info).
Aug 2016. The paper “Modeling Restaurant Context for Food Recognition” was accepted for publication at IEEE Transactions on Multimedia.
The paper “Image captioning with both object and scene information” was accepted at the ACM MM 2016 Grand Challenge. We obtained the best result at the Yahoo–Flickr Challenge on User Tag and Caption Prediction!
The overview “A survey on context-aware mobile visual recognition” was accepted for publication at the Multimedia Systems journal.
The paper “Scene Recognition With CNNs: Objects, Scales and Dataset Bias” was accepted at CVPR 2016.
The paper “Category co-occurrence modeling for large scale scene recognition” was accepted for publication at the Pattern Recognition journal.
The paper A probabilistic model for food image recognition in restaurants was accepted for oral presentation at ICME 2015.
The paper “Joint multi-feature spatial context for scene recognition in the semantic manifold” was accepted at CVPR 2015.

Latest posts

Source-free unsupervised domain adaptation

Can we perform unsupervised domain adaptation without accessing source data? Recent works show that it is not only possible but also very effective. In this post I review our recent works (ICCV 2021, NeurIPS 2021, CVIU 2023 and TPAMI 2023), ...

November 9, 2023 / continual learning, CVIU, deep learning, domain adaptation, ICCV, NeurIPS, TPAMI, transfer learning

Compression for training on-board machine vision: distributed data collection and dataset restoration for autonomous vehicles

Unmanned vehicles require large amounts of diverse data to train their machine vision modules. Importantly, data should include rare yet important events that the vehicle may face while in autonomous operation. In addition, modern vehicles capture data from multiple cameras ...

September 28, 2022 / deep learning, generative adversarial networks, image compression, TIP

MAE, SlimCAE and DANICE: towards practical neural image compression

Neural image and video codecs achieve competitive rate-distortion performance. However, they have a series of practical limitations, such as relying on heavy models, that hinder their adoption in practice. In this aspect, traditional codecs are usually designed with such practical ...

September 17, 2022 / continual learning, CVPR, deep learning, image compression, transfer learning

Neural image compression in a nutshell (part 2: architectures and comparison)

Neural image codecs typically use specific elements in their architectures, such as GDN layers, hyperpriors and autoregressive context models. These elements allow exploiting contextual redundancy while obtaining accurate estimations of the probability distribution of the bits in the bitstream. Thus, ...

August 31, 2022 / deep learning, image compression

Neural image compression in a nutshell (part 1: main idea)

Neural image compression (a.k.a. learned image compression) is a new paradigm where codecs are modeled as deep neural networks whose parameters are learned from data. There has been increasing interest in this paradigm as a possible competitor to traditional image ...

August 24, 2022 / deep learning, image compression

Mix and match networks (part 2)

This is a brief update on mix and match networks (M&MNets), describing the new ideas included in the extended version (IJCV 2020). An earlier post contains more details about the original CVPR 2018 version. Mix and match networks (summary) We ...

February 8, 2021 / deep learning, generative adversarial networks, transfer learning

MeRGANs: generating images without forgetting

The problem of catastrophic forgetting (a network forget previous tasks when learning a new one) and how to address it has been studied mostly in discriminative models such as image classification. In our recent NeurIPS 2018 paper (video), we study ...

October 29, 2018 / continual learning, deep learning, generative adversarial networks, NeurIPS, transfer learning

Learning RGB-D features for images and videos

Depth sensors capture information that complements conventional RGB data. How to combine them in an effective multimodal representation is still actively studied, and depends on different factors. Here I will focus on scenes and discuss several approaches to RGB-D scene ...

October 17, 2018 / deep learning, RGB-D, transfer learning

Mix and match networks

We recently explored how we can take multiple seen image-to-image translators and reuse them to infer other unseen translations, in an approach we call mix and match networks, presented at CVPR 2018 (also IJCV 2020). The key is enforcing alignment ...

August 31, 2018 / CVPR, deep learning, generative adversarial networks, IJCV, transfer learning

Rotating networks to prevent catastrophic forgetting

In contrast to humans, neural networks tend to quickly forget previous tasks when trained on a new one (without revisiting data from previous tasks). In our recent ICPR 2018 paper we propose the rotated elastic weight consolidation (REWC) method to ...

August 21, 2018 / continual learning, deep learning, ICPR, transfer learning

Topics: continual learning, deep learning, domain adaptation, generative adversarial networks, image compression, master degrowth, model compression, RGB-D, transfer learning, uncategorized

Short bio

I am a researcher (Ramón y Cajal fellow) with the Video Processing and Understanding Lab of the Universidad Autónoma de Madrid. From 2017 to 2023 I was a senior researcher with the Computer Vision Center. From 2012 to 2016 I was with the Institute of Computing Technology (ICT) of the Chinese Academy of Sciences (CAS) in Beijing (China). Previously, I worked with Mitsubishi Electric R&D Centre Europe in Guildford, United Kingdom, and with the Universidad Autónoma de Madrid (UAM), where I received my Ph.D.