I have been interested in deep learning applications since I took a class on the subject during my semester abroad in Budapest, Hungary. Deep learning methods, a subset of machine learning methods, mimic human neural networks to make predictions and inferences from data. The applications of these methods range from machine translation of text to self-driving cars to content-recommendation systems. What fascinates me about deep learning methods is the seemingly endless potential to both gather insights from data and create utility by using them.
I utilized part of my AMS grant to attend the Deep Learning Summit, hosted by Re-Work in San Francisco, February 17–18. The summit featured talks by academics; researchers in the technology industry; and engineers involved in the development, production, and implementation of deep learning algorithms. I had the opportunity to attend several talks that exposed me to groundbreaking ideas and technological products involving deep learning, enterprise AI, and AI ethics.
One of the first talks I attended was a technical one by Sebastian Raschka, an academic at the University of Wisconsin–Madison. He spoke of the problem of dealing with ordinal variables in machine learning — and his solution.
The problem is that there is no obvious method for classifying categorical variables when they have an inherent order. Typical machine learning methods account for purely categorical and purely numerical variables. He solved this problem by proposing two frameworks that extended binary classification and compared the probabilities of each pair of labels, so that “distance” between labels, or rank inconsistency, is not a problem anymore.
Another talk I attended was by Matt Linder of Headspace Health. He spoke about the problems of transparency in the implementation of artificial intelligence and of generating personalized recommendations, e.g. if the user liked product A, they should try product B. In the space of mental health, it is important to look at recommendations being generated and make sure that they are appropriate and sensitive, which may mean avoiding recommendations related to pregnancy, depression, etc., and it is also important to explain the recommendation, e.g., “Since you liked morning meditation, you should try calming ritual.”
Omar Florez of Twitter spoke of using transformers and multimodal content in machine learning, which in this case means using both the image and text in a Tweet as inputs for a machine learning model. These models are used for several applications like predicting if a tweet is hateful. He showed that transformers, a new deep learning method used in natural language process applications, were very effective in information retrieval.
Recently, I have been especially interested in natural language processing (NLP), which involves the conversion of text to numbers that are readable by computers and then reconversion to words recognizable by humans. The methods are used in language translation and sentiment analysis, to name a few. Transformers, a deep learning architecture introduced only in 2017, have already taken the NLP industry by storm. I realized, from the prevalence of transformers in several of the product presentations at the summit, how important it is for data scientists to constantly educate themselves about the latest inventions in the field, as methods in this industry very quickly become obsolete when new research is introduced. The idea is both exciting and a little daunting.
Spotify’s director of algorithmic assessment spoke at the summit about bias in recommendation systems and LinkedIn’s director of engineering spoke about how it models job seekers, hiring managers, and their interactions, and how it makes decisions about these interactions — such as how the popularity of a job posting impacts its visibility. They presented the evolution of their algorithm through several models.
Overall, I learned not only from the content in the talks, but also through my conversations during lunch breaks with technology industry professionals who do exactly the kind of work I hope to be doing in the future. I am grateful to have had the opportunity to do so.