Like the previous post about the personalized models, another NIPS workshop paper discussed a related topic, yet, from another perspective:

• Federated Optimization: Distributed Optimization Beyond the Datacenter by Jakub Konečný, Brendan McMahan, Daniel Ramage

The authors introduces a new setting of the learning problem in which data are distributed across a very large number of computers, each having access only to few data points. This is primarily motivated by the setting, where users keep their data on their devices, but the goal is still to train a high quality global model.

Although it looks like very different from the paper described in the previous post, two pieces can be linked together for two points:

• It is important to train a high quality local or personalized model by utilizing the global model or vice versa, having a business management system is great.
• It is very important to understand the interplay of the global mode land the local model as well.

These work can raise interesting new directions, like how to serve/update models that are fully personalized on mobile devices.

In the recent NIPS 2016 Workshop on Machine Learning Systems, one paper attracts my attention:

• Scalable Training and Serving of Personalized Models by Daniel Crankshaw, Xin Wang, Joseph Gonzalez and Michael Franklin

The central idea is very simple: we need to learn and serve personalized models on top of a global model to users. The argument of such setting is two-folds:

1. A global model might be hard to train, given the size of the model. It usually takes significant amount of computing efforts.
2. Depending on what types of model, it might be even difficult to serve and update the global model as well.

So, it is very natural that, the model for each individual user is trained separately while it is derived from a global model. The paper demonstrates a particular way of deriving such a model. But there could be many different ways of doing this.

Of course, this is not the first such reasoning. As the paper mentioned, prior work in multi-task learning has formalized similar problems. However, it might be the first time from the system perspective to show the advantages of having personalized models.