Latest News
- September 2023: Fantastic news! Our bot TWIZ, was an the 1st prize winner of the second Alexa TaskBot Challenge! Read all the details in the final report.
- October 2022: ACM Multimedia 2022 was a massive success! Record number of submissions. :-)
- June 2022: Our bot TWIZ, was an award-winning solution in the Alexa TaskBot Challenge!
- July 2022: This was a great year to be in Lisbon: The European Summer School in Information Retrieval was at NOVA University :)
- October 2021: We are organizing ACM Multimedia 2022 in Lisbon! Check the web site for details.
Research Interests
My research interests are in the area of vision and language information understanding and applications. In particular:
- Multimodal information understanding.
- Multimodal conversational AI (Workshop @ ACM Multimedia)
- Multimedia search and summarization (read this post about our work
with the BBC R&D).
- Temporal and memory models.
- Social media information quality.
Projects@NOVASearch group
I coordinating the NOVASearch group at FCT/UNL. Our group's applied research targets the Web, social media and clinical domains, and a number of research and industry projects have funded the team's work:
- Amazon Alexa TaskBot Challenge 2021-2023 (Faculty Advisor)
- iFetch (Local-PI, 2020-2023, CMU|Portugal) - Multimodal conversational agents for online marketplaces.
- SmartyFlow (Local-PI, 2017-2020, ANI PT2020) - Face anti-spoofing biometrics for airport border control.
- COGNITUS (Local-PI, 2016-2019, H2020 ICT) - Convergence of UGC and professional media.
- GoLocal (PI, 2016-2020, CMU|Portugal) - Social media monitoring for context aware recommendations.
- QSearch (Local-PI, 2012-2015, ANI QREN) - Enterprise domain search.
- ImTV (PI, 2010-2013, UTAustin|Portugal) - On-Demand Immersive TV.
- CS4SE (PI, 2010-2013, FCT/MCTES) - Compressive sensing for multimedia search engines.
Selected Publications
- Diogo Glória-Silva, Rafael Ferreira, Diogo Tavares, David Semedo, João Magalhães. Plan-Grounded Large Language Models for Dual Goal Conversational Settings. EACL 2024.
- Rafael Ferreira, Diogo Tavares, Diogo Silva, Rodrigo Valerio, João Bordalo, Inês Simões, Vasco Ramos, David Semedo, João Magalhães. TWIZv2: The Wizard of Multimodal Conversational-Stimulus, Amazon Alexa TaskBot Challenge, 2023.
- Rafael Ferreira, Mariana Leite, David Semedo, João Magalhães. Open-domain conversational search assistants: the Transformer is all you need. ECIR 2021.
- G. Marcelino, D. Semedo, A. Mourão, S. G. Blasi, J. Magalhães, M.
Mrak:
Assisting News Media Editors with Cohesive Visual Storylines. ACM Multimedia 2021. - D. Semedo, J. Magalhaes, Diachronic cross-modal embeddings , ACM Multimedia 2019.
- D. Semedo, J. Magalhaes, Temporal cross-media retrieval with soft constraints , ACM Multimedia 2018.
- G. Marcelino, R. Pinto, J. Magalhaes, Ranking news-quality multimedia , ACM International Conference in Multimedia Retrieval 2018. (Best paper nomination).
My full list of publications is available on DBLP.
Graduate Students (co-)Advising
- Pedro Valente, co-supervised with Prof. Rui Nóbrega
- João Silva, co-supervised with Prof. Bruno Martins
- João Coelho, co-supervised with Prof. Bruno Martins
- Artur Guimarães, co-supervised with Prof. Bruno Martins
- Diogo Silva, co-supervised with Prof. David Semedo and Prof. Alex Haupmann
- Diogo Tavares , co-supervised with Prof. David Semedo and Prof. Alex Rudnicky
- Rafael Ferreira, co-supervised with Prof. David Semedo and Prof. Alex Haupmann
- Carla Viegas, co-supervised with Prof. Alex Haupmann
- David Semedo (2020)
- Flavio Martins (2018), co-supervised with Prof. Jamie Callan
- Andre Mourao (2017)
- Filipa Peleja (2015)
If you are motivated and willing to work-hard + play-hard in researching vision and language understanding:
- We have several research positions and projects. I'm happy to discuss ideas and opportunities that fit the scope of ongoing research.
- Please, read this first: Richard Hamming, "You and your research" (Jamie Callan says this is for senior researchers, but I still recommend this for young minds, like mine was at the time I read it :) ).
Teaching
Office hours: Thursdays 17h00 - 19h00
Courses:
- Information Retrieval (Computer Science BSc/MSc and Big Data Analysis and Engineering MSc)
- Web Data Mining and Search (Computer Science MSc)
Chairing and Service
- ACM Multimedia 2026 Program Committee Chair
- ACM Multimedia 2024 Authors' Advocate Chair
- ACM Multimedia 2022 General Chair
- ACM Multimedia 2020 Workshop on Multimodal Conversational AI, organizer
- ECIR2020 General Chair ( After the COVID lockdown, it became the first ever 100% online conference in Retrieval !! :) )
- ACM Multimedia PC Area Chair 2019, 2020
- ECIR2015 Demo Chair
- ACM ICMR 2014 Publicity Chair
- PC member: ACM Multimedia, ACM SIGIR, ACM CIKM, ACM ICMR, ECIR.
Biography
João Magalhães (PhD, Imperial College London, UK, 2008) is a Full Professor at the Department of Computer Science at the Universidade NOVA de Lisboa. His research interests cover the different problems of vision and language understanding, in particular: foundational vision and language models, multimedia search, multimodal conversational AI, and multimodal temporal models. He has coordinated and participated in several research projects, where he aims to generalize his vision and language research to solve real-world problems across different domains. The work of his group has been awarded or nominated for several awards and distinctions: in 2023 and 2022 his group was awarded the 1st place and 2nd place respectively in the Amazon Science Alexa TaskBot Challenge, a conversational AI challenge to incorporate multimodal (voice and vision) user data, in 2020 a best paper award at the Portuguese NLP conference (PROPOR), in 2018 two nominations for best paper at ACM Int’l Conference on the Theory of Information Retrieval and at ACM Int’l Conference in Multimedia Retrieval.