Leveraging Technology to enhance Mall & Shopping Centre Appeal: Exploring the AI-VR-AR options

If smart technology is responsible for taking customers away from the malls and shopping centres, it is the smarter version of this digital experience – artificial intelligence, predictive analysis, the Internet of Things – that is once again attracting customers to the ‘brick and mortar’ centres. Such are the platforms being developed that they can easily out-smart the online channels when it comes to engaging with customers.

The real challenge of bringing technology to the mall is to ensure that the same is used in a way that it moves beyond the ‘wow’ factor and adds more charm to the shopping experience. If mall operators and their tenants get the strategy right, the outcome will also add on to retaining customers for longer duration, higher retail purchases, and repeat visits. US based research major IDC estimates that by 2021 the smart-tech market will reach US $110 billion in revenue sales and the Retail industry will become the top spender on these smart technology platforms by 2021.

Tech trends in retail

As per the Retail Digital Outlook 20181 report by Indian IT major Infosys, cyber security (74%), big data analytics (66%) and enterprise cloud (62%) are being utilized the most by tech-savvy retailers today. The study further reports that investment in augmented reality (AR), PoS or in-store experience and analytics for aligning marketing spends should be the trend for coming three years.

Blurring lines between online and offline – “The era of the ‘offline to online’ movement got a leg up when aggregators like Amazon, Flipkart, Myntra, etc also started fancying online to offline movement by blurring the difference, thereby acknowledging that one channel might not be sufficient to keep the consumers satiated,” says Chirag Boonlia, Vice President – IT, Virtuous Retail, and adds: “Malls and the shopping centres are also toying with the multi-channel strategy. We have seen many retail players facilitating pick-ups at their stores irrespective of the sales channel the order might have come from.”

AI, AR & VR – Customer-centric technology platforms are generally labelled as Artificial Intelligence (AI) and two other variants in smart technology today are Augmented Reality (AR) and Virtual Reality (VR), both of which add similar value in retail by enabling personalized and engaging experiences. While AR consists in overlaying digital content upon the real world, enriching it with interactivity, VR immerses users into an alternate reality, completely disconnected from the actual world.

These technologies help extend the mall and shopping centre’s connection to the customer through product visualization, customer experiences, and brand differentiation ‒ which are obviously the most common use cases in retail. It is basically the rising attraction for experiential retail and retail-entertainment that is fuelling demand for these technologies.

As per findings of the Sonar report entitled ‘Frontier(less) Retail’ by J Walter Thompson Intelligence2, 70% of US millennials and 62% of millennials in the UK say they would prefer a brand or retailer using AI technology because they believe that these brands will be able to more accurately predict what they are looking for.

With special reference to preferences of Gen-Z shoppers (13-17 years old), the Sonar report reveals that:

  • 80% of them are more likely to visit a store that offers entertainment
  • 80% are more likely to visit stores offering AR and VR technologies
  • 79% are more likely to visit stores that offer interactive experiences to customize products

Artificial Intelligence

Artificial Intelligence (AI) products are mainly Sales and CRM based Applications and many of these have yielded increased ROI; it is also true that some others have been tried, but failed to meet expectations. AI is now increasingly being regarded as ‘essential’ to competitiveness of the mall and will have a dramatic impact on businesses by 2020, as revealed in a recent study by Tata Consultancy Services3. Almost 32% of the top companies who participated in this survey believe that AI’s greatest impact will be in sales and marketing or customer service, while 20% see AI’s impact being largest in non-customer facing corporate functions, including finance, strategic planning, corporate development, and HR.

Customer loyalty is an integral part of CRM. “The art of engaging with the consumers has undergone radical overhaul with shopping centres now collecting consumers data subtly, without consumer making an effort to share the same,” says Chirag of VR, and adds, “The various consumer touchpoints provide data disclosing sufficient insight into consumer’s behaviour.”

AI in fashion segment, FashionAI was first developed by Alibaba researchers with an aim to provide a recognizable interface for customers to use while trying on clothes. Using the system, customers can try clothes on, receive fashion tips and suggestions from the AI App, and then make purchase online. If a user wants to try something different, or add other items, a store attendant can be summoned with the press of a button.

Chatbots and Deep Machine Learning are the next big shift in customer service

Retail Chatbot – A Chatbot engages with customers, it is location-based and uses multiple data feeds over different customer touchpoints. It answers in natural language and in real time, complex questions around gifting, food recommendations, attractions and holiday events happening in the mall. World’s largest mall, the Mall of America deploys chatbots produced by Satisfi Labs that reside on the mall website, mobile app and facebook page and can also be added as an Amazon Alexa Skill.

Machine Learning (ML) – The ML advanced algorithms are trained on huge data sets and offer personalized recommendations based on customer aspects like behaviour, activities, profile data, etc. that are intuitive enough to drive shopping action. Such Deep learning allows the AI to make connections in real-time by accessing massive quantities of data and making ‘smart’ decisions.

On Amazon, for example, if the user is looking at a pair of ethnic designed slippers, it might prompt the algorithm to recommend other casual slippers; but if the user also searches for a Sherwani, he is likely to be recommended a complete set of ethnic wedding gear. Remembering the personal choices of each customer could never be possible by any sales staff, but AI makes that task very much possible, and easy as well.

Humanoid Robot Pepper – Japanese firm SoftBank and French robotic manufacturer Aldebaran partnered in 2010 to develop Pepper, a humanoid robot that can interact with customers and ‘perceive human emotions’. Pepper is used as a customer service greeter and representative in 140 SoftBank mobile stores in Japan. During trial of the product at apparel store the Ave, 98% increase in customer interactions, 20% increase in foot traffic and 300% increase in revenue was recorded. Such AI robots appear to at least initially boost store interest and sales, probably because customers feel excited to try out a session with it. The Mall of America also deploys Pepper.

Besides providing recommendations about food and events, the chatbot can also accomplish the following tasks:

  • Provide directions to stores, restaurants, attractions and services based on current location
  • Answer customer inquiries on deals, gifting recommendations and holiday events in the mall
  • Recommend the mall’s top gift items and brands and direct shoppers to the stores that carry the brands
  • Connect a shopper with a digital guest service representative in real-time if an ongoing AI conversation requires more information

Robot at Kiabi Fashion Store, Paris

Conversica ‘sales assistant’ – Conversica software is designed to automate and enhance sales operations processes by identifying and conversing with internet leads. The company claims that its authentic-sounding messages result in engagement rate of 35%. Boch Automotive of UK employs Conversica software, which it attributed to a 60 unit sale increase per month at one Toyota dealership.

IBM’s Watson – This AI platform from IBM provides several order management and customer engagement capabilities to eCommerce retailers. In the trials to replicate its role at brick-and-mortar stores it was noted that within two months, 70% of online orders were completed through physical stores.

Snap.Find.Shop for easy visual search – American department store chain Neiman Marcus uses a Snap. Find. Shop feature that allows customers to take pictures of anything they like and have the app display similar items from the store’s inventory. This helped the retailer increase customer engagement.

Memory Mirror for enhanced shopper experience – Neiman Marcus has also deployed another cutting-edge tech called Memory Mirror produced by MemoMi Labs Inc, which allows shoppers to digitally compare outfits by capturing a 360-degree view of themselves in an outfit and letting them send these images to friends via email and social media. This is a virtual dressing room, which even online shoppers cannot experience or match as of now.

The Memory Mirror also works on the cosmetics counter, recording a customer’s makeover session in the perfect light and note all the products used. The video can then be sent to his/her phone, to be shared and discussed later at home or with friends for their input.

A similar mirror takes video clips of customer wearing eyeglasses or sunglasses and allows comparing up to four different styles side by side. One can see what new frames look like even when it’s hard to see because you don’t have your own power glasses on.

Applications within Malls

In the US, Simon Property Group planned to roll out AI chatbots for its 208 malls across the US, at a time. In India, INOX Leisure Limited (INOX) is amongst India’s largest multiplex chains with approximately 125 multiplexes and 500 screens across 60 cities. At Mumbai’s R City Mall, Ghatkopar and newly renovated Metro Inox at Nariman Point, INOX enthralls visitors with amazing life-sized 3D AR characters and scenes. Similar experiences are already available in more than 40 countries.

What makes it more unique is that visitors can capture and share high quality photos and visuals of their favourite AR experience moments with friends and family. The AR experience allows visitors to get up-close and personal with animated creatures & experience.

DLF Cyberhub presents the unique “Phygital” experiences with Huber, a virtual concierge, where one can talk to Huber to find out when and where the next event is happening, one can reserve a table, checkout the menu, and so on. During this conversation, Huber gets to know about the customer and can make personalized recommendations.

“Foodbox at VR Mall Bengaluru is a case in point with Mobile App for food ordering, top ups, card balance check and order notifications. Apart from that there are browser based web apps available for food ordering & notifications, duly corroborated with offers, product showcase window and free Wi-Fi to keep the savvy consumers occupied,” says Chirag of Virtuous Retail.

Foodbox at VR Mall, Bengaluru

Augmented Reality

It’s like walking by a store and having relevant information being delivered to your phone based on your earlier shopping behaviour. Once you enter the store, a set of Augmented Reality (AR) gear/glasses help you compare the products, view details, and even scroll through reviews on social media simply by glancing at the merchandise in hand.

The horizon can be further widened via social sharing: one can tag specific merchandise with AR notes that can be shared with friends — you can be sharing what you want to buy for yourself or can be making recommendations for others.

Entertainment value in AR – With the AR glasses on, one can see images of specific stores with complete display of merchandise for the upcoming festive season. On the one hand this encourages future visits, and on the other hand attracts customers to the malls and shopping centres for a rich, interactive experience awaiting them.

Getting people to visit the stores and malls is a great challenge for retailers and mall managers. While malls emphasize on adding the entertainment component alongside retail and F&B, retailers too are seen bringing merchandising and entertainment together through smart customised displays. For example, if the customer wants to buy a pair of running shoes, the AR application of the store could suggest the best options based on customer’s gait and stride.

Speedy checkout – Mobile payments are a lot faster than chip cards. Going further, now the emphasis is on eliminating the checkout process altogether. Walmart is rolling out its Scan & Go app that allows scanning the items with smartphone while putting them in the cart, the total bill and receipt can thereafter be generated, bypassing the checkout line.

Amazon’s touted brick-and-mortar locations Amazon Go employs technology whereby customers need to use the app to check in, and thereafter the entire shopping experience is designed to be automated. Sensors track which objects customers pick up and put in their basket, and respective customers’ Amazon accounts are automatically charged after exiting the store – no check-out required.

Virtual Reality vis-a-vis Augmented Reality

AR enhances our perception of reality by super-imposing computer-generated images, video or graphics over our view of the real world. For example, a 3D simulated virtual dinosaur can be projected to appear next to you in the mall.

Virtual Reality (VR) is another smart-tech platform slightly different from AR, especially with regards to experiences4:

  • While VR creates more immersive emotional engagement, AR enables practical trial, visualization and interaction with product for better shopping decisions.
  • VR limits usability to viewer devises, AR provides easier scalability through multiple devises like web, smartphones, tablets.
  • VR is harder to integrate across channels while AR enables immediate purchase action, anytime, anywhere, through scan-to-shop Apps.
  • VR can create fully immersive, imaginary brand experience with focus on fun factor, while AR remains natural to the shopping and product environment, helps in better purchase decision.
  • In VR we interact with computer generated environments that ‘replace’ their real world whereas AR adds these experiences to the present world.

“Technology platforms are evolving at an unprecedented pace and the winner will be one who can leverage these to solve business problems,” says Chirag of Virtuous Retail, “A classic example is ubiquitous smart phones, this device can potentially be leveraged for solving umpteen plaguing issues.” Citing the case of weekend crowd at the shopping centres, with serpentine queuing at food courts or Parking, he says the problem is effectively addressed by mobile Apps/ Web Apps that can be used for seamlessly ordering the food, top up cards and also get food-readiness notifications. “This genuinely brings que-less, cashless and personalized experience to the savvy consumers and can potentially be extended to parking for an express exit instead of manually scanning the bills and payments.”

Make-up Bar at Kiabi fashion stores, Paris

Why is implementation slow?

Small offline retailers do not have the capability or scale to experiment with customer centric technology in order to decide the most suitable one. On the other hand ‘big box’ stand-alone retailers like Best Buy, Target, Walmart, and Future Group, Reliance Retail, Landmark Group, ABRL’s Mega Mart, etc are extremely slow to adopt cutting-edge technologies in customer services, even though they are implementing substantial business-critical applications in manufacturing and supply chain.

The future will likely be dictated only after cutting edge technology experiments of the big-box retail are proven effective. Every at-scale retailer is looking to these big players for hints on feasible next-steps, which will help them form their own ideas on technology strategy.

‘AI revolution’ for customer services in the shopping Centre retail space is therefore unlikely for another two to three years, but Entertainment Malls are very much capable of immediately adapting the same, based on experience of early adapters in malls and big-box retail.

The hesitation, however, still persists. “I see AR & VR as too premature to create a compelling case for investment, also from the ROI perspective; but there is a huge opportunity to tap onto the AI space,” says Chirag of VR, and adds, “Traditional data collection is dead and AI & Machine Learning platforms like Tensorflow, Watson, etc can be leveraged to get insight about the consumer to an extent that was extremely complicated till now. AR does have some scope in social gaming that can potentially be explored by shopping centres.”

To conclude

Experts are generally of the view that retailers should opt for chatbot platform that can centralize data and scale up the bot development. Through deep personalization the platform allows store and mall operators to use every information that is already known about a customer, and also everything that the customer shares during a series of interactions – finally, the right message can be delivered to the right user at the right time. This way, a frequenter of a neighbourhood mall gets a different content as compared to one who visits a luxury destination mall.

“Business Value realization is the key and at the end of the day, it is all about the ‘Business Problem and ROI’ (actual and/or Tacit), that drives decision-making in the boardroom; the ideal model would be to partner with the new-age and agile smart technology start-ups and try and solve one business problem at a time; also, bring every stakeholder on the same page before marching further,” says Chirag Boonlia of Virtuous Retail.

The AI-VR-AR chatbot platforms can also be used to engage with consumers proactively, thereby allowing for full engagement personalization. On the basis of customers’ past shopping behaviour, the mall operator can recommend new stores and offers. Different properties of the same mall developer can have differently programmed Apps. In effect, AI-VR-AR Chatbots are a major evolution in digital customer relationship management (CRM) for retail and mall brands and none can afford to ignore early implementation of the same. “The winner would be the one who could potentially strike a balance across the multiple platforms and curate an omnichannel and seamless experience for the ever savvy consumer”, concludes Chirag Boonlia.


1. Digital Outlook 2018, Retail Industry, Infosys
2. Frontier(less) Retail, J Walter Thomson – WWD Report, June 2016
3. Getting Smarter by the Day: How AI is elevating the performance of global companies. Global Trends Study by Tata Consultancy Services, 2017
4. https://catchoom.com/blog/virtual-vs-augmented-reality-experiences-in-retail/ 09 June 2017 By Helena Franco