Minds Mastering Machines (m3) 2019 Recap - Day 1

Usually I don’t like that the time moves so fast as I get older but from time to time it is a good thing. Roughly one year ago I posted an article about the first installment of a data science conference called minds mastering machines or m3 in Cologne. I was very excited back then to finally meet a lot of data scientists and talk amongst peers. The talks were very good and I learned a lot. Naturally I was looking forward to m3 2019 in Mannheim, Germany.

This year I couldn’t make the workshops on the first day but I heard that they were all sold out. I got a feeling over the whole event that a lot of visitors were interested in hands-on experience. They wanted to see source code on the slides as I heard a lot in-between talks. I know that the organizers heard that as well. I for one don’t really need that much detail in my talks. I know that I can find a lot of useful information in thousands of python notebooks all over the internet. What really interested me this year were experiences from real projects. This drove my decisions on what talks to attend.

Just as last year there were three tracks so I got to attend only just above one third of all talks (since the two keynotes were in a single slot). In the following I will write about the talks I attended with a small note about the two talks I missed. My feeling is that most of the presenters would be happy to share their slides or talk to you about the topic if you ask them.

Take a look around

Before I start with the talks I want to show you some pictures from the venue. We met at the Congress Center Rosengarten in Mannheim.

Day 1 - Wednesday

So now that you have seen the venue I will start with the talks. First we were greeted by the organizers and then the first keynote kicked of the conference.

Keynote - Dark Data

The honor to start the conference was given to Mark Whitehorn and he talked about Dark Data. This term was coined by himself on a whiskey-fueled evening as we were told. Just like dark matter can’t be observed in the universe dark data stands for all the data that exists and influences the reality but somehow does not land our systems or simply does not get analyzed. He even defined three levels of dark data:

  • Dark Data 1.0

    • The data that is in the transactional systems but never makes it to the analytical - visible but unseen. It influences our analysis but we can’t see it it in the analytical system

  • Dark Data 2.0

    • Data that is published but which the publisher does not expect to be analyzed

  • Dark Data 3.0

    • Data that is not there (or vanishes) and the lack is highly significant

He gave two examples for DD1.0. The first was the Andrea Doria, an Italian ship that collided with the Swedish ship Stockholm. The tragic thing is that both ships had the equipment the easily see the other ship on the radar. However it seems that the data was misinterpreted or “not seen”. Another example were bats flying into nets after coming back from the hunt. It seems that they are more relaxed and therefor ignoring the strong echo signals of the nets and got caught. This is called the “Andrea Doria Effect”.

The part about DD2.0 was the most fun and exciting for me. He gave a lot of examples for companies leaking information that was never intended to be analyzed.

  • In the second world war the serial numbers on the tanks indicated how many tanks were produced by the germans.

  • He could correlate the position of this books on amazon with his selling data. This gave him the possibility to find out about how many books his competing authors sold. A piece of information that would been shared with him if he had asked.

  • Ever wondered how many complaints a company receives? They might just tell you. Just send some complaints and you will most likely get a nicely formatted incident number that you can analyze.

A lot of company might leak some interesting data without knowing it

A lot of company might leak some interesting data without knowing it

DD3.0 is data that is missing. He gave an example about a document that was given to the residents of a street that should be changed. He looked through it and found out that a tiny part of the street was not listed in the data. Low and behold this was the part of the street that was too small for people to walk on the sidewalk.

Towards the end of his presentation he introduced two papers of his colleagues about Data Augmentation and Autoencoders. I must say that this keynote was very entertaining and a great kickoff for the conference.

Processintegration of machine learning models

So after the keynote I had to choose between talks for the first time. Since I wanted to learn more about getting machine learning models into production and real projects I chose this talk over Machine Learning Overview and What’s new in tensorflow 2?. In this talk they showed the complete process of developing and integrating a model that is used to classify invoices of car repair shops sent to insurances.

fitzner-schieblon-processintegration.png

The goal was to decrease the manual checks of the invoices while making sure that only invoices that are good will be automatically processed. They worked in the Azure Cloud and showed us a little bit of that. However I was a little bit disappointed that things like model versioning weren’t talked about. I guess the Azure Cloud takes care of this.

All in all it was a nice overview of the process of getting a machine learning model into production. This talk was held by Antje Fitzner and Jeremy Schieblon.

Machine Learning Model Management - A framework comparison

Storing your machine learning models is just like storing recipes

Storing your machine learning models is just like storing recipes

A very active topic in the machine learning community is model management. Everything from versioning and reproducing the results. In this talk the lecturer used a very fitting analogy: recipes for creating great tasting food. You have basically the same requirements.

This topic seems to spawn a lot of interest right now as I can see on the visitor numbers on my article about Sacred. We got introduced to DVC, Sacred and MLFlow. It seems that MLFlow is a very cool and easy to use tool. I might play around with it and write about it.

To sum up the conclusions of the presenter: If you want to get started fast use MLFlow. If you need to set up a bigger system and need reproducible results and you maybe combine several difference ecosystems you might need DVC and/or Sacred. This talk was held by Nico Kreiling. And I think he also wouldn’t mind me mentioning his podcast here. I missed the talks Continuous Intelligence - From notebook to production and an introduction to the boosting technique.

Security Weaknesses of Machine Learning

This talk was held by Daniel Etzold (@etzoldio). It was a nice summary about the latest research in machine learning model security. I think most of you have heard about creating adversarial images by now. It is relatively easy to construct images that fool a neural network to think a given image is something different (for example that a lion is an ostrich). The most surprising thing about these images is that a human will not be able to see a difference between the images. Especially in high risk environments like autonomous driving these security issues will become a very hot topic in the next years and I seriously hope that the car manufacturers are already working towards solving these issues by combining their computer vision systems with other sensors and higher level rule-based logic (for example: I am driving towards a traffic sign and the system continuously detects a STOP sign with high confidence until the car gets close and then it suddenly changes something might be off). Adversarial images weren’t the only security risk that were mentioned but the one I think that has the most potential to be a danger in the future.

Great visual explanation of this attack on an object classifier

Great visual explanation of this attack on an object classifier

Recommender-Systems for invoicing

I was not really sure what to expect in this talk but in the end I got a really nice and well-presented introduction to different techniques to create recommender systems. The presenter downplayed the role of recommender systems because Amazon is using it for ages now but in the end a lot of recent machine learning tools are used here. Three approaches were nicely explained here all in the context of issuing invoices.

  • Collaborative filtering

  • Autoencoders

  • Neural networks

Collaborative Filtering

This is the classic approach to recommender systems. Let’s say you analyze a shopping cart in a webshop with three items. Then you look at all the shopping carts you sold in the past and try to find a fourth item that was sold a lot together with these three items. And then you recommend this fourth item: “80% of customers also bought this”. For the technical details just look around the internet. There are plenty of resources.

Autoencoders

Autoencoders were introduced in the keynote to be used in data augmentation. Here they should be used as a recommender? How does that work? Well the general idea of an autoencoder is to output the same data that was put into the neural net. The trick is that someone inside the hidden layers there is a layer of significantly smaller size. This means that in this layer the core information of our data is present. So what happens if we insert data of a shopping cart? The autoencoder will compress the information to the core and will then output a shopping cart based on that core information. We then pick the most promising items to recommend.

Neural Networks

You can also use standard neural networks to recommend items. For each item on each invoice you create a training record. Just use the item as the label and the remaining items as input data. A neural net can learn non-linear dependencies and superior combinations. However there has to be a lot of parameter tuning and the result is not as easy to intrepret. On top of the a neural net is essentially a black box and you can’t really know why a certain item was recommended.
In the pictures below you can see the input into the recommender system and the output of the three approaches. The brightness of a point (item) correlates with the probability to recommend the item.

This talk was held by Harald Bosch. I missed the talks Neural Embeddings in production - What could go wrong? and Interpretability of machine learning models.

AI in the context of IOT and predictive maintenance

Since I am working with IoT devices at the moment as well I was very interested in this talk about predictive maintenance. It was about a project in France where the lights inside a tunnel should be monitored. We got some insights about working with a customer and what problems appear in the real world. My takeaway is that for every data science project you have to work as close as possible with the customer and especially with the domain experts.

The first part of the talk was a broad introduction into IoT and its possibilities. It could have been shorter because I think that most of the people in the audience knew about this already. After that it got interesting. They introduced the project and talked about all the different systems that are involved.

Always make sure that you are aware that your machine learning model is just a small part of a working production system

Always make sure that you are aware that your machine learning model is just a small part of a working production system

They even got into detail (also due to questions after the talk) about the systems and application they used. They provided a nice overview that I added below. Since I am working as an software architect these insights are very valuable to me.

This talk was held by Olga Mordvinova and Cyrille Waguet. I missed the talks Introduction to Deep Learning and Transparent Creditscoring: How to better explain decisions with consequences.

IoT Infrastructure Overview

IoT Infrastructure Overview

AI-supported quality control

The last talk of the day was hit-or-miss for me. I was not sure what to expect beforehand and was positively surprised. It was about quality control in the aircraft production. Honestly the domain is far away from my usual projects. They talked about the production errors in aircraft bodies. You can see one of these pieces on the picture below. They are currently working on a system to support the human testers. These testers get to see ultrasound scans of the bodies and have to find errors. These are highly trained specialists and I was humbled by the images I saw in the presentation. Some errors were simply not recognizable for me.

They approached this problem at first using random forests and got to a classification accuracy of about 79%. After that they consulted a company that specializes in machine learning and they created a dense net approach which gave them 93% in the end. The slides were very detailed and I found it really refreshing how open they were about their project and also all problems that came with it.

This talk was held by Olaf Beesdo and Mathias Burger. I missed the talks Graph Search: The dark side of AI and the secret of Alpha Go and Satellite Computer Vision with Keras and tensorflow.

You can see the ultrasound on the left. Try to spot the error.

You can see the ultrasound on the left. Try to spot the error.

Themed Tables and Quiz

After the talks on the first day there were themed tables prepared for everyone to get in touch and talk about topics of interest. There also was a great quiz that started with data science topics but quickly turned to memes and other funny stuff. It was very enjoyable and a nice conclusion of the first day. So after a few more drinks I headed back to the hotel and the first day was already over. Make sure to read my article about the second day as well. I heard talks about ai ethics, law, text mining and the keynote about style transfer. Oh and I also won something in the raffle.