SIGGRAPH Asia 2019 Real-Time Live! Full scenario and feedback
We have demonstrated “Real-Time Live” in SIGGRAPH Asia 2019 Brisbane. The title is “Global Bidirectional Remote Haptic Live Entertainment by Virtual Beings” (Official Info).
The presentation was not best in show. Anyway, here is a full scenario of this session.
Start in real
Hello everyone! My name is Akihiko Shirai, Director of GREE VR Studio Lab. We will be presenting: “Global Bidirectional Remote Haptic Live Entertainment by Virtual Beings” This is my first SIGGRAPH in Australia. and I want to show our real time live demo beyond the pacific ocean. First of all I would like to transform into my avatar!
Switch to Virtual
Lea: Hello Everyone! Yay! I would like to introduce my friends! MilkyQueen.
MIlkyQueen(MQ): Hello, rice to meet you! My name is MilkyQueen. Member of VR “Rice” Idrols “Mai-Bei-Bei”. Today I will be performing in English from Tokyo.
Lea: Thank you MilkyQueen, and Kilin!
Kilin: (Japanese)こんったったーKilinです、日本のAIアイドルやってます よろしく～
Lea: Thanks, she is from Japan.and Xi!
<OBS: Name | Tokyo- Brisbane>
Xi: (Chinese)你好，我是AI偶像 犀。 Nihao、私はAIアイドルのXiです
Lea: Thanks, she is from China.
Kilin: ぼくたちはキャロットークという翻訳アイテムを使って日本語と中国語を自由に切り替えできるよ～
MQ: So Kirin and Xi can use this CarrotTalk system to translate and freely switch between speaking Japanese and Chinese! Amazing!!!
<OBS: 5G Logo>
MQ: 5G mobile network will be the next generation of streaming entertainment. Sitting in your living room experiencing this virtual world with people all over the globe will soon become a reality.
Lea: This session uses “VirtualCast” system, we are currently performing on a same stage virtually, but physically they are 7 thousand kilometers away in Tokyo.
MQ: I’m currently performing using HTC Vive Pro, a consumer HMD. Our motion data is being synchronized between Tokyo and Australia. Lea: It’s ideal for next gen live entertainment instead of video streaming. Now we don’t feel network delay between each virtual beings.
＜OBS: Name | Tokyo- Brisbane＞
Lea: And we have a simple haptic device “Hapbeat” to enhance interactive emotions for real time live entertainment.
MQ: Most Vtubers can only see the audience that they are performing to, but we now can feel the emotions of the audience.
Lea: So let’s test it out! Please shake my hand. Nihao Xi!
(Shake hand with Xi, Kilin stop it) LeaはXiと握手して、Kilinが止めに入る）
Lea: Can you see my Hapbeat? Is it lighting up? So, give me a hug, Kilin?
<OBS: Hapbeat>
(During the hug) MQ: So usually if we try and hug in the virtual world we bump into each other and can’t feel the hug, but with this technology it feels like we’re hugging for real!
Lea: Thanks! MilkeyQueen, what do Virtual Beings usually get up to?
MQ: Every Virtual Being has a different reason for existing. I am working hard to promote Japanese rice culture all over the world! Japanese rice production is falling, so I am trying my best everyday as a virtual rice idol to make sure everyone sees how great rice is!
Lea: How interesting! SIGGRAPH is an academic conference for computer graphics and interactive techniques, now a new era has arrived thanks to hyper virtuality. What about you? Kilin and Xi?
<OBS: Introduction movie>
Kilin&Xi: わたしたちは日本と中国でかけはしとして、生アニメシリーズ「直感アルゴリズム」をやったよ～ このまえは六本木の東京国際映画祭で5Gやすごい技術で、楽しいイベントを作ったよ
MQ: They are AI idols that are building bridges between Japan and China through real-time live animation “heart algorithm”. They also performed at Tokyo International Film Festival showcasing the latest 5G technologies, combined with their funny sense of humor.
Kilin&Xi: たのしかったよ～！
<Return to VC>
Lea: Thanks. So Let’s explore what Virtual Beings can do together! Now we’re connected to bidirectional remote haptic devices which can analyze sound from the audiences, your clapping and excitements.
Lea: So, where is everybody from?
<OBS: Excitement and Clap＞
Clap your hands if you’re from China!
(画面に “Clap your hands if you’re from China / Confetti comes”)
Xi: 今天有从中国来的朋友吗？有的话请拍手！鼓掌!! (しばらく待つ)
Lea: 謝謝！
MQ: Now your applause has made confetti appear!
Lea: So, who comes from Japan?
Kilin: 日本からの人～拍手～！もっといいんだよ～ (しばらく待つ)
MQ: Who comes from United States? (interact)
MQ: Thanks Who comes from Europe? (interact)
Lea: Okay, finally, How about from Australia?
MQ: Did you forget about New Zealand? (expect laughing)
Lea: As you can see we analysed your laughing and loudness too, now it raining Baoz!
Kilin & Xi: バオズ降ってきた～おいしそー
MQ: Those Chinese buns look tasty!
Lea: Thanks for your laughing and applause. So, let’s play game?
Lea: Thanks for your laughing and applause. So, let’s play game?
MQ: Let’s play a game as virtual beings over 7000km.
Lea: Okay I give you a virtual bow, Milky. Then shoot the apple on my head!
MQ: Okay, it’s virtual William Tell Game over the Pacific ocean.
Kilin: それじゃあ二人とも、位置についてねー
Xi: 加油～。
MQ: Let’s begin the countdown together! Lea: In your language!
MQ: Ten, nine, eight, seven, six, five, four Kilin: はち、なな、ろく・・（シンクロ） Xi: （中国語で） Lea: （フランス語で）
MQ: three, two, one…. Shoot!!
Lea&MQ: Yeah-! / Kilin:Yatta! / Xi: Lihaiya!
(うまくいっても行かなくても連射OK)
Lea: Thaanks! We have discovered a new way to play together over such a long distance! So, MilkyQueen, do you have any ideas on how to conclude this amazing show?
MQ: Well, what is more fitting to end with than dancing together?
（ダンス以外にあってるものはないとおもうよ！）
＜OBS: 520動画フルスクリーン＞
Kilin: 「ごーにーまる」はどうかな？
MQ: What does “Gonimaru” mean?
Xi: うんうん “Wo Er Ling”
MQ: Ah, do you mean “Wo Ai ni” “I Love You” in Chinese?
Lea: Let’s dance together!!
<Return to VC>
https://youtu.be/A2zkKvuFdd4
<CREDIT>
Producer/Contributor:
GREE VR Studio Lab — Akihiko Shirai, Yusuke Yamazaki
EXR, Inc. — Kensuke Koike
NTT DOCOMO, Inc. — Yoshitaka Soejima
Cast:
Kilin : Akina Homoto
Xi:Emiri Iwai
MilkyQueen — Mai Princess：Inori Rose Oguruma
Lea “Realité” Imai: Akihiko Shirai
Music: “520”/ Tacitly — ©NTTDOCOMO,INC. ©MIGU.,Ltd
Special Thanks:
Just Pro — Ayumi Mino, Xiangyuan “Shin” Qin, Shunya Igarashi, Naoto Sato
Virtual Cast, Inc. — Shinnosuke “MIRO” Iwaki, Takeshi Sasama, Naoto Nakajima (Okada Video)
Hapbeat LLC.
WFLE, Inc. — Eiji ARAKI, and all “REALITY” team
GREE VR Studio Lab Interns: Ryota Kayanoma, Mizuki Aoki, Takanori Horibe, Rex Hsieh
SIGGRAPH Real-Time Live!: Prof. Mike Seymour (and all reviewers)
That’s end of the show but…
Here are some feedback.
Very rare shot on the stage:
Give you feedback via #SA19RTL!
Actually, our perfomance were not completely executed, we have still reflections which contains audio related technical issues, the winner of session was also great. Other performances were also respectable. So, we have experienced the world wide level during this oversea presense. We may challenge for next opportunities like SIGGRAPH2020 and/or other real-time “live entertainment” scenes.
Keep in touch, and tell us @VRStudioLab if you have interested to be collaborate or invite us!
