Harald: Don't Watch if You Don't Want Your Head to Explode

[NoCleverName]

So this is from a keynote video at the Microsoft "Build" (a software developers) conference. After about 3 or 4 minutes they show some AI work they are doing with PSIA:
https://channel9.msdn.com/Events/Build/2017/C9L30

Now, the beauty of this is that you'll see the PSIA Demo team perform some of the most precision and clean "pop-up" skiing you'll ever see. But the next part, where they "train" an AI bot in the Azure cloud to "learn" what so-called "good" skiing is and then compare some poor slob hooked up to some sensors is where you better be sitting down. (Actually, you'll be lucky to tell what's going on at that point).

But I'll give you this, if the system were to be trained with real good skiing, rather than PSIA, there might be something to it. Maybe. Although I think a video MA would probably work better.

And of course, what'll really boil your blood is where they claim to be the "educational arm of the snow sports industry."

[ChrisC]

The methods discussed in the video are about differences between any two skiers, so there's no concept of good versus bad. If it helps substitute Skier A and Skier B whenever they mention the IT guy and the instructor.

What's interesting is the easy way to capture the data

The analysis of the data is a different story because the methods used are very standard: logistic regressions. Take away all the nice user interfaces to the analysis software and the databases and there's nothing particularly interesting here. The data analysis shown in this video uses methods that have been available for many years.

[NoCleverName]

The methods discussed in the video are about differences between any two skiers, so there's no concept of good versus bad.

Well, you're being charitable. Observe that the demo team was rigged up to provide the base data (so the "difference" is between the demo team and the candidate skier). Well, at least that's what I gathered... I could be wrong. It was a bit murky there.

But then we have the main question: if it's only showing the "difference" between two arbitrary skiers (as you claim) ... what's the point? It appeared to me they were taking 70% of the data base (in this case the movements of the team) and seeing how the candidate differed from them. Of course, the demo team wouldn't be the "demo team" if they weren't "demoing" something, in this case, presumably, better than average skiing. On the other hand, we could see that the demo team was in fact showing less-than-efficient movements and that the candidate was the unfortunate recipient of lessons representing that style. Unsurprisingly, he was having a rough time.

In the end, this seems to be a case of taking sports metrics too far ... a trend we see in pro sports.

[jbotti]

Moved from Primary Movements section.

[ChrisC]

The methods discussed in the video are about differences between any two skiers, so there's no concept of good versus bad.

Well, you're being charitable. Observe that the demo team was rigged up to provide the base data (so the "difference" is between the demo team and the candidate skier). Well, at least that's what I gathered... I could be wrong. It was a bit murky there.

But then we have the main question: if it's only showing the "difference" between two arbitrary skiers (as you claim) ... what's the point? It appeared to me they were taking 70% of the data base (in this case the movements of the team) and seeing how the candidate differed from them. Of course, the demo team wouldn't be the "demo team" if they weren't "demoing" something, in this case, presumably, better than average skiing. On the other hand, we could see that the demo team was in fact showing less-than-efficient movements and that the candidate was the unfortunate recipient of lessons representing that style. Unsurprisingly, he was having a rough time.

In the end, this seems to be a case of taking sports metrics too far ... a trend we see in pro sports.

It works a bit differently to this. The model (logistic regressions) predicts between 2 classes based on the measurements from the devices strapped to the skiers. The 2 classes are the 2 skiers in this case or 2 groups of skiers in the general case. Let's call them Group A and Group B for the moment.

So each observation in the database consists of the positions of the devices at a particular time and which group of skiers is being measured.

70% of the data is used to train the model (also called fitting the model or estimating the model - there is a lot of IT jargon here). This contains data from both Groups A and B.

30% of the data is used to test the model for predictive accuracy. This involves taking the measurements from the devices and using the logistic regressions to predict whether it's from Group A or Group B. If the predictions are mostly correct that shows you can tell between the groups.

So there's nothing in the analysis that shows one group is better than the other. It just shows that they have different measurements and it gives some details about how they differ.

[ChrisC]

Another point is that the instructor Nick has some significant alignment problems (look at the A frame he shows in the video).

[noobSkier]

I haven't seen the video, but it looks like a classic PR move to promote Microsoft's new machine learning platform. Its not meant to be sensible or accurate, its just for engineers to go "ooooh" (the ones who know nothing about skiing anyway...which is most of us so well played Microsoft).

[Obrules15]

I feel like people have missed the salient point here, on the video PSIA ADMITS they don't know how to recognize the movements of good skiing--that's why they are interested in machine learning so they can communicate to their students whether they are engaging in appropriate movements. In other words, they need Microsoft, Logistic Regression, Machine Learning etc to do what every PMTS coach does on a daily basis. Stunning!!!

[DougD]

I feel like people have missed the salient point here, on the video PSIA ADMITS they don't know how to recognize the movements of good skiing--that's why they are interested in machine learning so they can communicate to their students whether they are engaging in appropriate movements. In other words, they need Microsoft, Logistic Regression, Machine Learning etc to do what every PMTS coach does on a daily basis. Stunning!!!

Agreed, except that they have all those tools and still DON'T do what every PMTS coach does. If you don't understand what effective skiing movements are, recording and comparing skier movements is an exercise in randomness, regardless of the tools you use to do it.

As Harald just pointed out on another thread...

People like ______, who can't teach a lesson, never raced, and is a medium to low level intermediate skier. He's a computer scientist, who made himself a PSIA guru because they had [and apparently still have] nothing else. His descriptions of skiing are nebulous and outcome based, without understanding of movements. If you don't understand skiing movements and how they are achieved, you can't teach or convey teaching information.

This reminds me of a project at a company I recently left, an IT team working on software to replace human experts at interpreting legal/commercial contract language. The language in these contracts is negotiated by lawyers and contract specialists. The final version can be forbiddingly complex and typically includes finely crafted compromise positions, some of which are susceptible of different interpretations. This may be by design.

I was asked to spend some time helping the IT head of this project. His level of understanding was exemplified by the following question, "So if we search a contract for the words "X" and "Y" and find them, that makes it a good contract in this area?" That's rather like asking, "If a skier's feet and hips move during a turn, that makes it a good turn?"

Breathtaking ignorance masquerading behind high tech tools does not impress.