This year’s KDD was in Barcelona, Spain. It was the first time I visited the country and the city. Barcelona has a unique charm as a good combination of old Roman districts as well as 20 century art. The conference has more than 2300 participants, another year of high excitement despite economic challenges that made fewer corporations set up a booth to hire.

Here are some highlights from the conference.

Tutorials

• Graph Intelligence with Large Language Models and Prompt Learning
• RAG Meets LLMs: Towards Retrieval-Augmented Large Language Models
• Multimodal Pretraining, Adaptation, and Generation for Recommendation
• A Survey of Large Language Models for Graphs
• Causal Inference with Latent Variables: Recent Advances and Future Prospectives

Workshops

• 2nd Workshop on Causal Inference and Machine Learning in Practice
• 2024 International Workshop on Talent and Management Computing
• TSMO 2024: Two-sided Marketplace Optimization
• 3rd Workshop on End-End Customer Journey Optimization

Selected Papers

Ranking/Recommender System

• Enhancing Pre-Ranking Performance: Tackling Intermediary Challenges in Multi-Stage Cascading Recommendation Systems [Link]
  • From Ant Group
  • Large-scale recommender systems have 3 stages: Recall, Pre-Ranking and Ranking. Pre-Ranking is a selection stage that the team uses to sample down items from Recall to Ranking to reduce the size. The paper proposed a framework to tackle sample biases and model consistent performance issues for such 3 stage systems. The deployed system showed 7% improvement over the baseline.
  • [Note]: Worth reading to see how different  companies tackle multi-stage challenges.
• Achieving a Better Tradeoff in Multi-stage Recommender Systems through Personalization [Link]
  • From Meta
  • In this paper, the authors claimed that a key observation is that, all else equal, ranking more items indeed improves the overall objective but has diminishing returns. With this observation, the authors proposed a framework to exploit the trade-off of performance and computation cost. The real-world experiments showed the effectiveness of the proposed framework.
  • [Note]: Worth reading to see how different  companies tackle multi-stage challenges.
• On (Normalised) Discounted Cumulative Gain as an Off-Policy Evaluation Metric for Top-$n$ Recommendation [Link]
  • From ShareChat
  • The paper formally presented the assumptions that are necessary to consider DCG an unbiased estimator of online reward, providing a derivation for this metric from first principles whilst linking it to off-policy estimation. The authors then proved that the widespread practice of normalizing the DCG metric renders it inconsistent with respect to DCG, in that the ordering given by nDCG can differ from that given by DCG, and provide empirical evidence. Empirical results from off- and online experiments on a large scale recommendation platform show that the unbiased DCG metric strongly correlates with online metrics over time, whereas nDCG does not, whilst differences in online metrics directionally align with differences in both nDCG and DCG, the latter can enjoy improved sensitivity to detect statistically significant online improvements.
  • [Note]: Worth reading as the results are a bit surprising and might be worth examining whether it can be applied to a wide range of use cases.