AlphaGo

Apreche

This needs its own thread now.

Also, Round 2 starts in 5 minutes

open_sketchbook

Win number 2.

RIP humanity. Thanks Obama.

Luke Burrage

I like that not only is AlphaGo better than Lee Sedol, but also blindsides all the commentary teams with its center board play. Things that look like mistakes? Hiding moves which are waaay more important later on that humans just can't spot or realize.

Dromaro

Game 2 to AlphaGo.

It seems like the machine efficiency of not worrying about time is paying off for it. The team behind AlphaGo says it does not factor in the opponents time when deciding its move but I think it had a definite effect on Lee. I'm curious what an untimed game would look like.

The analyst was confident it was very close but then acknowledged that he (in hindsight) greatly undervalued a tactical choice made my AlphaGo about 30 stones prior. My understanding of how AlphaGo thinks (and I don't claim to be anywhere near an expert) is that since it doesn't brute force much in the same way DeepBlue does with Chess that if that move/sequence does turn out to be as pivotal as the analyst suspects, it may mean that AlphaGo understands some deeper tactical level of Go that humans haven't discovered yet.

Apreche

What's holding us back now is that systems like AlphaGo take very smart people to make, and lots of expensive computing resources to run. If we could get over those two things, such systems could replace most human intellectual labor. Robots can already replace the manual labor. Then we can go enjoy the beach. Please read Astro Boy and iRobot to prepare yourself for the new society.

zehaeva

The time and resources it's taken to make this AI are incredible, nearly as much as AlphaGo's results! I"m not sure how well a third party will be able to utilize similar techniques to create anything that really amazing. We have access to the algorithms and code, Google even released TensorFlow, their ML libraries (IIRC AlphaGo was made using Torch). The secret sauce is the data and computational time. and Google has that in spades.

In this case I think an independent party could have done it in the same way, the game records are public, but it would have taken a lot more time to train the neural nets. Not sure how much money the Google has spent just letting AlphaGo play itself though. That'd probably be the real limiting factor here.

Apreche

I just thought of something in the shower.

During a game, a human needs time to think. That's why Go, and other games, have clocks. AlphaGo also needs time to think, but it's much faster than a human. AlphaGo playing so fast must seriously cut into Sedol's thinking time. I do most of my thinking during my opponent's turn.

Here's what's even more interesting. Does AlphaGo "think" when it's not it's turn? Or does it just idle until a move is made by the opponent? If the human opponent takes longer on their turn, and AlphaGo is thinking during that time, that is really bad for the human. It can explore an extraordinary number of branches of play in the time it takes a human to explore one. If it thinks during it's opponent's turn, the way to beat it might be to play very fast and loose, and to play with a very short clock. 5 second turns or something.

zehaeva

I were wondering something similar last night while watching the game. The clock has a huge affect on a human's psychology. Is it fair to let the computer think on the human's turn? Is it fair to time the human in the same way as the computer?

Towards if AlphaGo thinks on his opponent's turn I would imagine you could find it in the nature paper that Google published. I'm not sure how much it'll help, I guess you can just run through the algo and memoize the results and then throw away the ones that don't match the move made and run through even more variations with your left over time.

Apreche

The two-time world Champion of Netrunner, plays fast on purpose. He does this for quite a few reasons. The effect it has on the opponent, however, is huge. Suddenly you can't think on his turn because you are trying to figure out what he did. All the thinking happens during your turn. That's not so great from a psychological standpoint.

If AlphaGo thinks during the human's turn, it will be able to take its turns much faster. All the thinking will be during the human turn, creating this same psychological pressure.

The other question is whether or not AlphaGo ever takes a shortcut. Does it always fully and completely calculate its best move, or does it ever estimate a best move from available choices due to time constraints? If it ever does the latter, then allowing it to process during the human's turn will allow it to make even better decisions.

okeefe

Of course AlphaGo is working during the human's turn. That's twice the effective compute resources. Likewise, Lee Sedol is working on the AI's turn.

Why would the AI playing faster help the AI in any way? Humans playing faster leads to weaker moves and, worse, opportunities for mistakes.

Apreche

okeefe said:

Of course AlphaGo is working during the human's turn. That's twice the effective compute resources. Likewise, Lee Sedol is working on the AI's turn.

Why would the AI playing faster help the AI in any way? Humans playing faster leads to weaker moves and, worse, opportunities for mistakes.

Playing faster in a timed game gives the opponent less time to think. If you can make decisions faster than your opponent without making mistakes, they are going to be at an extreme disadvantage if there is any time pressure whatsoever.

Gotta go fast!

zehaeva

I imagine that Lee Sedol is less affected by the pressures of his opponent's rhythm than most people but it would still unnerve him at some level.

When I play tutorial games in Go I have to purposefully measure out my play time so I don't overwhelm the less experienced opponent, playing at my natural pace would cause them to cease thinking and playing considered moves and go into just responding to my moves, which isn't really all that beneficial to them. In tournament games playing through a sequence (joseki) that you know really well that your opponent doesn't can unsettle them and allow you to eat through their clock. I've won many games doing just that. Even in informal games, not timed ones, managing pace and rhythm can really turn the game in your favor.

no fun girl

Computers can calculate so much faster than humans and are not vulnerable to psychology, that I imagine cutting turn time would have a greater impact on the human.

Apreche

no fun girl said:

Computers can calculate so much faster than humans and are not vulnerable to psychology, that I imagine cutting turn time would have a greater impact on the human.

You might be able to win by time out. AlphaGo often takes quite a bit of time on its turn. If you can force it into overtime, and force it into difficult situations, it might lose by time out.

Also, AlphaGo was trained using pro Go game recordings and by playing against itself. I highly doubt it, but I am very curious if it has a vulnerability to effective, but less unpredictable amateur play patterns it knows less about.

Rym

One thing I noticed (purely anecdotally) is that in Chess, I as a relatively inexperienced player could force masters I played into tight spots, and they would always claim it was due to my unpredictability in the early mid game. (I'd still lose, but they wouldn't win straight away, and would express frustration at not having won already).

But in Go, I never observed an amateur player giving any pause to an experienced player. If anything, the lower level players were guaranteed to lose at a time of the skilled player's choosing.

I suspect that Chess has more low level heuristics to guide "basically proficient play" even in novices, and the pattern play of experts is disrupted beyond that point, but ahead of near-mastery. But in Go, there are almost no low level heuristics, and thus MANY invisible traps that doom a novice.

The unseen traps may well preclude a novice from ever doing anything that is both "surprising" and "not a terrible mistake."

okeefe

Apreche said:

You might be able to win by time out. AlphaGo often takes quite a bit of time on its turn. If you can force it into overtime, and force it into difficult situations, it might lose by time out.

It's unlikely. I only know what I've seen from the conclusion of these two games, but after your primary clock runs out you have three one-minute rounds you can use, and a round doesn't go away unless you go past that minute. AlphaGo has one minute to assess a much smaller search space at the endgame (plus whatever time the opponent takes). It might make slightly less optimal moves, but I would guess that AlphaGo has a superior late game.

Apreche

okeefe said:

Apreche said:

You might be able to win by time out. AlphaGo often takes quite a bit of time on its turn. If you can force it into overtime, and force it into difficult situations, it might lose by time out.

It's unlikely. I only know what I've seen from the conclusion of these two games, but after your primary clock runs out you have three one-minute rounds you can use, and a round doesn't go away unless you go past that minute. AlphaGo has one minute to assess a much smaller search space at the endgame (plus whatever time the opponent takes). It might make slightly less optimal moves, but I would guess that AlphaGo has a superior late game.

Yeah, I noticed the same thing. Before overtime it often takes over a minute to decide. In overtime, it often decided within 30 seconds. It knows it's in overtime, and has to decide quickly. It does have a smaller search space in the late-game, but it also might be making suboptimal moves due to the time constraint. If you can keep the game close enough through the mid-game, you might be able to exploit a small mistake in overtime to get a few points ahead and win.

Really, what other hope is there for a person to outsmart a computer? It's like. Could you calculate the square root of 7343 faster or more accurately than a calculator? No, that's absurd. And now beating a computer at Go is equally absurd. Some big Chinese Go master said he could beat it. Bring it on!

zehaeva

It's an interesting thought but my gut would say that AlphaGo would trounce an amateur. Their play is really only unpredictable at lower skill levels, and at such levels even conservative play by a professional would crush them.

Rym

I suspect that AlphaGo's early game is the key. It's already doing things that appear to be more effective there than what humans have uncovered in over a thousand years.

Apreche

Rym said:

I suspect that AlphaGo's early game is the key. It's already doing things that appear to be more effective there than what humans have uncovered in over a thousand years.

Knowing how true this is makes it even more fascinating when AlphaGo makes the same move that the commentator predicts. Does that mean that humans found optimal plays for at least a few patterns on our own, or is that residual suboptimal play from AlphaGo being trained based on human play?

zehaeva

Early game is the key, if you don't set up good formations at the start you'll find yourself behind in ways that endgame simply can not make up for. This is worse at the Pro levels, in their end game the path is clear and fairly impossible to make mistakes. The only really open points in end game are during ko fights and that's more because the point estimation of ko can be a bit fuzzy.

Rym

Apreche said:

Does that mean that humans found optimal plays for at least a few patterns on our own, or is that residual suboptimal play from AlphaGo being trained based on human play?

The answer to that question will have profound implications on the future of humanity.

Starfox

I feel that a lot of machine learning is like this:



But how do neutral nets really work?
MATH

MATATAT

Speaking of Go...

Apreche

They're going to use DeepMind for health care.

Luke Burrage

Finish him!

Linkigi(Link-ee-jee)

FATALITY

gg humans

Rym

So... Apparently he still has to play two more games. Even though it's best of five, they agreed to play all five games.

Apreche

I want to see the two more games. Maybe he can pull something out? The pressure is totally off now that he had no expectation to win anything.

Churba

Apreche said:

I want to see the two more games. Maybe he can pull something out? The pressure is totally off now that he had no expectation to win anything.

Oddly precient - He just won his first game. It's hard to tell from the outside how the machine was doing during the match, but looking at it afterwards, it appears that he's found a weakness. He played a very similar strategy to game two, but this time he tweaked it a little.

Daikun

AlphaGo won three out of four games. One more to go, but AG is the champ already.