›
This forum is in permanent archive mode. Our new active community can be found here.

Augmented Reality Allows Hatsune Miku to Perform Live in Concert

Victor FrostVictor Frost
edited May 2008 in Technology
5spuwsAeoIo

Not arriving at a stadium anywhere soon, the video above shows a 3d model of the vocoloid singer Hatsune Miku performing with a pint-sized holographic jumbotron behind her. What makes it so interesting is that neither She nor the jumbotron is real. This video is taken from the perspective of the video output portion of an Augmented Reality system, ARToolkit, which uses single camera orientation, pattern recognition, and image tracking. AR isn't new, however advances in modern computing technology now allow for dynamic reorientation of the virtual objects in space and full inter activity.

oIqgrsYNKs8

This is one of many wonderful little AR demos available on youtube for us to see, but when can we expect to see things like this in our everyday lives? Will the personal assistant of the future be no further than the lens of our glasses? Could our children's anime be 3d and interactive? Will the next big MMORPG take place, not in front of a computer terminal, but in an open field? Could this be our "Chobits" in 30 years or so?

Comments

  • ......
    Oh, I saw this before... I didn't realize what it was then, since I couldn't read the Japanese in the description and wasn't actively searching for Augmented Reality videos. When watching the second vid I was surprised when suddenly Miku sat on the little carton pattern, really awesome stuff. The only thing that's lacking is reaction. When the guy was rocking the carton pattern Miku kept standing/stepping, whereas she should've lost her balance. But that's for later. Great stuff, thanks for the info.

    On a sidenote, it's kinda amazing how this Vocaloid/Hatsune Miku craze goes on for so long and keeps growing bigger and becoming more widespread.

    P.S. Why didn't he hold the carton pattern thing upside down?
  • Victor FrostVictor Frost
    Because if the camera loses track of the pattern Miku would have disappeared. The AR system has to see the whole pattern otherwise it can't calculate distance and orientation.
  • AprecheApreche
    I really want to get this Hatsune Miku software to give it a try. Is it in English? How hard is it to make it work?
  • Conan-SanConan-San
    edited May 2008
    Scott: AFIK, the vocaloid software isn't in English par-cé but you can get it to sing in the Japanese approximation of English, with...somewhat hilarious results.
    Post edited by Conan-San on
  • KieyKiey
    I spent a day a few weeks back just looking at this AR stuff. It's really amazing, I'd love to learn how to do that stuff.
  • GreatTeacherMacRossGreatTeacherMacRoss
    I don't quite understand what I'm looking at. I understand what a vocaloid is, but am I looking at an overlay using the computer's video out? This is essentially a composite image right?
  • KieyKiey
    It's a camera recording into a computer, then in the computer it is adding to the image the 3D model of the Vocaloid singer. Then by moving the camera the computer is able to adjust the way the model appears in the video to appear as is it is really there in real life. And it does this all in real time.
  • ......
    Because if the camera loses track of the pattern Miku would have disappeared. The AR system has to see the whole pattern otherwise it can't calculate distance and orientation.
    I know, I figured that from watching the video, every time his finger touched the corner of the black square Miku disappeared, but who says you have to hide the pattern when you hold it upside down eh?

    @ Mr. MacRoss, yeah, it is as far as I can tell. All the person who was holding the camera saw was the little carton thingy. The computer took the video it received and overlaid the 3D model of Hatsune Miku, but that doesn't really matter, could've been a 3D model of Bill Gates. And by using some algorithms and particular points on the carton (the outside corners of the black square, the small black rectangle inside the black square indicating the front) it showed the 3D model properly. Neat stuff.

    @ Scott, the software is called Vocaloid, there's a version 2 out (Vocaloid2), you'd have to check the second video to find the romanized Japanese spelling though. You should be able to find it, as for getting it to work, yeah, Emily will probably be able to translate it, and Rym will be able to actually make some decent music come out.
  • Victor FrostVictor Frost
    edited May 2008
    I really want to get this Hatsune Miku software to give it a try. Is it in English? How hard is it to make it work?
    If you mean the 3d AR Hatsune, I haven't the slightest idea. I think this guy made the 3d model himself and wrote a program in Visual C to interface with the ARToolbox SDK. At least, thats what I gathered from watching his other videos. He's running windows so I wonder if I contact him if He'll email me binaries of his program.
    Post edited by Victor Frost on
  • Victor FrostVictor Frost
    edited July 2008
    Oh, Japan, why am I not surprised?
    Post edited by Victor Frost on
  • Victor FrostVictor Frost
    FOUND IT! The programs required for the first video! I haven't tested it yet, but oh manz, I have hope.
  • VhdbloodVhdblood
    GE or some other company did this to promote clean energy. They just had you print out a square like the one in the top video, and then took your webcam feed and showed a windmill on top of it.
  • jtvhjtvh
    Anybody planning to attend Hatsune Miku Live Party on November 10? I'll be in Seattle, and am thinking of attending there. I kicked myself for missing her concert at Anime Boston.
  • Victor FrostVictor Frost
    Anybody planning to attend Hatsune Miku Live Party on November 10? I'll be in Seattle, and am thinking of attending there. I kicked myself for missing her concert at Anime Boston.
    Very interesting.....
Sign In or Register to comment.