[Interview] “AR Emoji Can Be a Visual AI Assistant” – Developers on the Galaxy S10 AR Emoji

The ‘emoji’ originated in 1997 as a simple pictogram used to express emotion. But after years of evolution, the technology has progressed to the point that the Galaxy S10 AR (augmented reality) Emoji can transform your face into an animated version of itself before your eyes. Once you have your avatar, your AR Emoji can mirror a wide range of emotions and body movements with advanced facial and body tracking. You can even impose your AR Emoji on top of a photo of yourself and watch it move just like you!

Samsung Newsroom met with the developers of the Galaxy S10 AR Emoji, Yong-Il Kim, HanJun Ryu, MinKyung Lee, Jinho Lim, and MinSheok Choi, to discuss how they are leading the future of AR Emoji, and what’s next for the technology.


Q. What is the current trajectory of AR Emoji development?



The AR Emoji feature is a way to express and represent yourself. As we switched over from the Galaxy S9 to the Galaxy S10, we changed our focus from the representative ‘Avatar’ to the ‘Stylized Avatar’, which portrays users’ individual traits so attractively that they can’t help but show it off! To achieve this, we have created avatars that users feel affectionate toward because they represent their individual traits, but which also maintain a universal aesthetic appeal.



People use AR Emoji to reproduce themselves in the online sphere. Therefore, the feature must be able to portray not only their physical appearance, but also their online personality. We are continually coming up with ways for users to more freely and uniquely personalize their AR Emojis.




Q. What makes AR Emoji special?



The AR Emoji feature’s biggest differentiator is that it starts off by making a complete, detailed figure that best reflects the user’s actual traits. AR Emoji has avatar shapes to suit everyone and features a huge number of ways for users to represent their unique characteristics.



The features available in AR Emoji on the Galaxy S10 line include ‘Mask’ mode, ‘Mini motion’ and ‘Live figure’, all of which can be used in a range of situations. Because the AR Emoji feature is a specialized camera function, it can be used within the default camera app. It has proven to be a popular feature among users.

With ‘Mini motion’, body movements are captured so that the AR Emoji moves just like the user.

Q. What was your process for developing AR Emoji?



During planning it is important to identify the users’ needs. To figure out what these were, we monitored our acquaintances, families and a range of online communities to make sure that we made an AR Emoji that everyone would be satisfied with.



In the design stage, we considered how we could best express users’ emotions through facial expressions and stickers. The most common human emotions were categorized, then we had to establish the various degrees of each emotion. For example, there are many different kinds of sadness. Some make you want to scream, some make you want burst into tears, and some only cause a downturn of the mouth. When we struggled to find specific references for certain emotions, our team stepped in as actors to record their emotions. During this process, having emotive team members was a big help! Our recording sessions were always full of laughter.

Also, as an AR Emoji is a full body avatar, we needed to include a wide range of fashionable clothing items, shoes and accessories to make sure users had enough options to choose from. Research was very important at this stage, and some of our team members scoured shopping malls to ensure that we chose the right clothing options.



The version of AR Emoji that’s available on the Galaxy S10 line has updated editing tools. In developing these, we had to make internal decisions about how we were going to structuralize and define every component of the 3D data. For instance, we had to make decisions like ‘let’s divide the bone structure of the arm into five parts, and cover two of them with the sleeve while the rest are just bare arm.’ This was a useful process though, because as we were defining the details, a number of other potential features occurred to us. For example, we added the shadows on the avatar’s backs during this stage.

For more customization and creativity, users can utilize their own AR Emoji to create stickers.

Q. Do you have any tips for how best to use the AR Emoji feature?



You can use your avatar to make stickers that can be sent via messenger. This feature allows you to send customized emoji stickers with any expression you like. You can also create memes or stickers that include text to give the image more context.

A mask mode that applies your avatar’s face over your own is another useful function. When you need to take a photo but you haven’t had time to get yourself camera-ready, you can cover your face with an emoji. AR Emojis can also be very helpful for the many online creators who like to dress up or disguise themselves while recording their videos.

‘Mini motion’ is a function that both adults and children can enjoy. If you enjoy dancing, it can be fun to have a virtual figure to dance along with, instead of dancing alone.

Users can have more fun with the ‘Mask mode’, which adds the AR Emoji’s face onto users’ faces.

Q. Can you tell us what’s currently being worked on with regards to AR Emoji?



We are thinking about how AR Emojis can be used in new ways, such as users being able to play as their avatars in games. We are also working on incorporating popular characters into AR Emoji, as well as allowing users to custom-add things like make-up and tattoos so they can better express themselves with their emojis.



The global trends are centered around incorporating more and more decorative tools. However, we have also tried to be focus on being inclusive, with features like artificial hands currently available, and artificial legs to become available soon. By continuing to add more customization tools to AR Emoji, we hope to help people better express themselves. But we also want to see AR Emojis become a place where users’ imaginations can be brought to life.



We’re looking for ways to allow users to represent not just their faces, but their bodies too. Another area of focus is finding a way to utilize AR Emojis on other Samsung apps. Users can currently set their phones to display contacts’ emojis when they call, but we are exploring other uses that include a mask mode for video calls too.



As developers, our priority is to improve usability. Currently, in ‘Live figure’ mode, users can tap the figure on the screen to trigger a range of gestures and dances. We want to make these motions more fluid, while also expanding the feature by incorporating further actions and promoting deeper interaction between users and their emojis.


Q. What does the future hold for AR Emoji?



I hope AR Emoji will offer users a chance to familiarize themselves with AR technology and become more comfortable using it. As virtual reality-based services are commercialized and become more common, we expect AR emojis to be the best method for people to represent themselves in the virtual sphere.



Ultimately, our goal is to make AR Emoji a personified visual assistant in the coming age of AI (Artificial Intelligence). For example, currently Bixby can give us updates on the weather, but I believe that information could be delivered more effectively by an AR Emoji holding an umbrella.

(Clockwise from the top left) Samsung developers Yong-Il Kim and HanJun Ryu from Unique Sales Point Strategy Group, MinKyung Lee from Media UX Group, Jinho Lim from Graphic R&D Group and Minsheok Choi from Visual S/W R&D Group

* This article was originally published here

Seoul After Sunset: The Galaxy S10+’s Night Mode, In Pictures

There’s something special about the way cities look in the nighttime. Once the day draws to an end, our favorite cityscapes come alive and we often find ourselves itching to capture the scenes in front of us with our smartphones. But without a camera able to take on the nuances and challenges of nighttime photography, this undertaking can be a fruitless one.

The Galaxy S10+’s Night Mode is the perfect tool for capturing photos after the sun has gone down. Accessible from the Camera app, Night Mode can be manually toggled on and off whenever you wish to use it in the Camera app’s sliding menu. Compared with photos taken without it switched on, Night Mode minimizes the noise from a dark environment to enhance a photo’s details and corrects colors, along with other image qualities, to make the resulting low-light shots more vivid.



Samsung Newsroom traveled to some of the top landmarks around Seoul after dark to see how the Galaxy S10+’s Night Mode can bring out every detail in a nighttime scene, even in low light conditions.

A Palace for All Hours of the Day

Deoksu Palace is one of Seoul’s four ancient palaces, drawing visitors keen to take in its striking architecture day and night.

Junghwajeon, one of the royal courts of Deoksu Palace, was built in 1902 and has a remarkable golden exterior featuring intricate patterns, colors and textures that are not easy to capture at night. However, by simply turning on Night Mode on the Galaxy S10+, all the courts’ details come to life, thanks to Night Mode’s intelligent gradation adjustment that brings vibrancy to pictures taken after dark. Try using a tripod to level the top and bottom of the frame for an ever better shot of the ancient palace that captures every striking detail.

While the Galaxy range’s Night Mode function previously only automatically activated when conditions were very dark (below 1 lux), the Galaxy S10+’s Night Mode now allows users to make the most out of it as they can manually switch it on or off and use it for lighter nighttime scenes.

Stand-Out Statues Amid City Life

Seoul’s Gwanghwamun Plaza, often bustling with pedestrians, is a place where the traditional and the modern come together. Neon lights from the buildings that flag it, along with the bright lights of passing cars, make up a vibrant patchwork of color during the nighttime, but can easily drown out the statues at the Plaza’s heart in pictures.

Thanks to the Galaxy S10+’s Night Mode, the two statues that sit right at the heart of downtown Seoul (Admiral Yi Sun-sin, first picture, Sejong the Great, second picture) can be captured without any compromise to the eclectic lighting of their surroundings. Night Mode lowers camera sensitivity and decreases shutter speed, capturing bright lighting well despite the high contrast. Paired with the Ultra Wide lens of the Galaxy S10 line, Night Mode provides after-dark shots that can capture everything in the camera’s field of vision.

When Urban Meets Rural

Naksan Park in northern Seoul features vistas that include a striking contrast between the area’s historic fortress wall, idyllic mountain views and the nighttime bustle of downtown Seoul.

But Galaxy S10+ users need not worry about losing any detail when capturing close (first picture) or wide (second picture) shots of scenes with such subject contrast. Without Night Mode switched on, distinct features of the landscape would be lost, with city lights obscuring lower contrast areas such as the striking mountainous terrains in the far distance. But just by switching Night Mode on, you can maximize the visibility of your shot thanks to its intelligent noise reduction that balances different areas of contrast and light to cleanly bring out a shot’s every aspect – even in low-light conditions.

Sunset By the River

The Hangang Riverside Park lights up as the sun goes down, and the variety of different lighting attractions on display reflect in the Han River to often dazzling effect. Yet capturing this display of light from a variety of different light sources, from the natural light of the fading sun to the colorful façade of the Hangang Floating Islands, can present a challenge when looking to achieve balance in twilight pictures.

With the Galaxy S10+’s Night Mode, pictures emerge more vibrantly thanks to its auto color-correcting software, resulting in bright and clean shots that carefully negotiate all the different sources of light on offer. For an even more vivid take, users can alter Night Mode’s settings for lower exposure correction to capitalize on the darker aspects of a scene – without losing any of the color or detail.

Bird’s Eye Views

Seoul’s Namsan Tower sits an impressive 479.7 meters above sea level, offering visitors a scenic, near-panoramic view of the city which comes to life as the sun sets.

One of Namsan Tower’s best offerings to visitors is the opportunity it presents for you to capture shots of the sprawling city that lies below it – as well as of the striking tower itself against a diverse backdrop. But cloudy weather conditions add more disruptive photographic noise to pictures that obscures the details of a shot, especially for long-distance vistas.

Just by sliding Night Mode on in the Camera app, Galaxy S10+ users can take shots that bring out the vivid colors of this cityscape at night thanks to Night Mode’s intelligent noise reduction capabilities. Whether you are looking to photograph a landmark (first picture) or achieve a birds-eye view shot of a city (second picture), the Galaxy S10+’s Night Mode optimizes color and contrast balance to striking effect – even in less-than-ideal weather conditions.


^*Availability of the Galaxy S10 line’s new Night Mode feature may vary by market and product


* This article was originally published here