AR

The latest MEMS and sensors advancements in markets ranging from displays to biotech and in key areas of manufacturing including sensor devices and fabrication came into sharp focus at the 2022 MEMS & Sensors Technical Congress (MSTC) in late April as 120+ industry experts gathered in Berkeley, CA.

MEMS actuators transform electronic signals into something that can be sensed or touched by the end user of an electronics device. A case in point: MEMS actuators such as print heads in inkjet printers transform electronic files into text or beautiful images. In 3D printers, actuators can produce real objects. Inside smart glasses, tiny MEMS mirrors can create virtual objects. Little surprise, then, that integrating these powerful devices into the end products is a multidisciplinary enterprise. STMicroelectronics has been successfully leading the deployment of dedicated MEMS actuator solutions with customer products in various market segments. SEMI spoke with Anton Hofmeister, group vice president and general manager of the MEMS Actuator Division at STMicroelectronics, about MEMS actuator trends. Hofmeister shared his views at the SEMI MEMS Imaging Sensors Forum as part of the virtual SEMI Technology Unites Global Summit. Watch the STMicroelectronics’ presentation on-demand until March 26, 2021. Registration is open. SEMI: What is the difference between MEMS devices that sense and MEMS devices that actuate? Hofmeister: MEMS sensors gather data from the world around us including motion, pressure and air temperature and transform them into an electrical signal. Actuators work the other way round. They receive an electrical signal and transform it into some well-controlled actuation such as ejecting a fluid, moving a membrane or deflecting a laser beam. SEMI: How can MEMS actuators’ integration be simplified to be embedded in new applications so they appeal to consumers? Hofmeister: The challenge of integrating MEMS sensors into devices has been simplified by demo kits and evaluation boards, which customers use to embed the sensor into a system. MEMS actuators are more difficult to integrate. They often power the core function of a system and therefore require deep system understanding. Reference designs are a big step forward in simplifying integration. My presentation at the SEMI MEMS Imaging Sensors Forum showcased some examples. MEMS micro-mirror projection for augmented reality (AR) glasses is an example of a complex system that requires multiple types of components to function. Together with several partners, STMicroelectronics recently announced the LaSAR Alliance, which will develop reference designs to enable the AR glasses market. SEMI: MEMS sensors and actuators are considered the backbone of many consumer products. Are MEMS actuators also mostly used in automotive? Hofmeister: The widest use of MEMS actuators has so far been in print heads for inkjet printers. In recent years, we have seen actuators adopted in emerging applications ranging from piezo heads for 3D printers to MEMS mirrors for laser beam scanning systems or 3D sensing solutions for consumer applications. The first high-volume application in automotive will likely be MEMS mirrors for LIDAR systems. SEMI: What market growth trends do you see for MEMS sensors and actuators? Hofmeister: The sensorization trend, which aims to collect data from homes, cities, factories, cars and personal devices, continues to drive the adoption of sensors and actuators for a wide variety of applications. While the last wave of MEMS growth was triggered by one end product – the smartphone – the next wave will be driven by multiple applications and use cases in industrial, medical, automotive and personal electronics. SEMI: How can technology unite us? Hofmeister: In recent months, we have all experienced vividly how vital technology has become. MEMS, and semiconductors in general, are an integral part of many products and services that make our lives easier. Communications technologies have been particularly important during this pandemic, whether using the personal devices as our interface to the digital world or the complex infrastructure that they operate through. I hope that my participation at the summit helped increase awareness of the new possibilities and opportunities that technologies like MEMS actuators have to offer to create products and services that further improve people’s lives. Anton Hofmeister is group vice president at STMicroelectronics, general manager of the company’s MEMS Actuator Division and managing director of its German subsidiaries. Hofmeister has been with STMicroelectronics for more than 30 years, working in Germany, France, the U.S. and Italy. He has held managerial positions in key account management, product and strategic marketing, advanced R D and general management. For the past 10 years, he has managed various product divisions in the MEMS sector. Hofmeister has also served as a board member of the Singapore-based molecular diagnostics company Veredus Laboratories. Serena Brischetto is senior manager of Marketing and Digital Engagement at SEMI Europe.

SEMI spoke with Eyal Shekel, senior vice president of Service Strategy and Excellence at Tokyo Electron Limited, about the impact of artificial intelligence (AI) on smart manufacturing and how other fab solutions for smarter process tools are advancing semiconductor manufacturing.Eyal shared his views ahead of his presentation at the SEMI Fab Management Forum, 17 February, as part of the SEMI Technology Unites Global Summit, 15-19 February 2021, an online event. Join us to meet experts from Tokyo Electron and other key industry influencers. Registration is open. SEMI: AI technology is considered a key enabler for smart manufacturing. What are the latest trends? Shekel: The advent of advanced nodes and extreme complex 3D semiconductor geometry has lengthened time to market and increased costs in areas ranging from equipment development and large-scale metrology usage to monitoring yield inhibitors.AI is becoming a critical tool in the area of material informatics to determine suitable materials and processing techniques in order to meet the needs of future devices. Together with new materials and processes, the development and implementation of virtual metrology will enable accurate and almost absolute real-time monitoring of our customers’ device wafers at each stage of the manufacturing process.SEMI: What are the benefits of data analysis in the process from R D and Ramp-Up to High-Volume Manufacturing? Shekel: The new research field of materials informatics enabled by AI provides tools to guide the highly efficient discovery and optimization of production processes. For example, TEL has developed methodologies for co-optimizing processes and materials for etch rates.To monitor and manage the yield of semiconductor fabrication processes, direct metrology measurements are important. However, it is difficult to monitor all production wafers due to the time and cost involved. With deep learning AI, it is now becoming possible to predict every wafer’s metrology measurements based on production equipment data and previously processed wafer metrology variables. This enables total quality management and run-to-run control, while simultaneously reducing production costs and cycle time.SEMI: Can you tell us more about TEL Service Advantage?Shekel: TEL Service Advantage is a TEL global support organization that allows customers to select a service plan that fits their needs. Through TEL Service Advantage, we can quickly respond to customer requests and technical advancements. TEL Service Advantage provides various plans to maximize equipment maintenance efficiency for customers and productivity from equipment manufactured by TEL. TEL Service Advantage plans can be combined to meet customer needs and achieve maximum results.A key enabling element of TEL Service Advantage is TELeMetrics™. TEL analyzes equipment data from various sensors using a remote connection and, based on that analysis, provides solutions to customer-specific problems around equipment throughput and predictive maintenance.SEMI: How is AI helping during the pandemic? Can you share a success story? Shekel: The pandemic forced severe travel restrictions worldwide, making it very difficult or even impossible in many cases to visit our customers, as it is still the case today. Standard communication devices like smartphones and email helped at the beginning when TEL intensified the remote support by our Total Support Centre (TSC).TEL continued to develop its Service Advantage program quickly, and started using additional advanced tools and methodologies such as the following: Deployed AR (Augmented Reality) to remotely assist our customer and TEL engineers Secured remote connections into TEL tools to investigate parameters and logs, or to change set-up Used remote training courses that connects trainers via video conferencing systems and training tools in the factories to skill up engineers located in a different parts of the world Used AR glasses for tool start-up and troubleshooting Expanded TEL database global technology with multi-tool on languages search capabilities A key project at a customer site in Europe offers an excellent success story. Using all the approaches above, we collaborated with the local team to put a tool into production with no major delays. This was highly appreciated by the customer and very important for us.SEMI: What do you predict for the future? Shekel: Global technology infrastructure continues to develop and expand rapidly. Elements like 5G networks, IoT and advanced sensing capabilities will lead to what we call General AI, which will be based on neuro-like infrastructure. The auto learning will spread across domains and rely on internal logic and reasoning to automate many tasks that are manual today. In our industry in particular, General AI will enable workers to focus more on data analytics and future advanced R D rather than ongoing operations.SEMI: How can technology unite us? What do you expect from your participation at SEMI Technology Unites Global Summit?Shekel: Technology united us in the last 150 years. The connectivity started with telegraph and telephone and was used to exchange information over wider distances. Nowadays, video conference capabilities, AR and improving communications technology makes it much easier to unite people who are geographically dispersed. This becomes obvious and valuable especially during this pandemic period. As a fact, we are able to continue to perform all our key activities – our tool support, training and customer relationships – even if we cannot be present in person.The SEMI Technology Unites Global Summit is a great chance to stay connected to people and customers that I would normally meet at the SEMICON exhibitions.It also offers the opportunity to network with many more people who I would not be able to meet otherwise. Moreover, I can watch speeches and presentations at any time! Normally I would miss some programs since exhibitions and events took place at the same time.Eyal Shekel, senior vice president of Service Strategy and Excellence at Tokyo Electron Europe Limited, is a 27-year semiconductor industry veteran. Upon his graduation as a Mechanical Engineer from the Technion (Israel leading technical institute), he joined Applied Materials. In 1997 he moved on to Tokyo Electron (TEL) in Europe, served as the Regional Service Manager of Israel and, soon after, was appointed the company’s General Manager. Since 2005 Eyal has been part of TEL Europe senior management. He oversaw the Service and Support Operations for TEL Europe as a senior vice president until 2019. In his current role, he co-leads TEL’s Global Service Committee in Japan.The SEMI SMART Manufacturing Initiative is a global effort to promote awareness of and interest in smart manufacturing with a focus on delivering industry-recognized best-in-class programs and services to enable members to maximize product quality and productivity while reducing costs. Activities are focused on building out core capabilities to enable smart manufacturing across the microelectronics supply chain. MADEin4 is a consortium of 47 partners from 10 countries connecting the full range of supply chain – from semiconductor equipment manufacturers and system-integrating metrology companies to RTOS and key applications such as the automotive industry. The MADEin4 Project develops next generation metrology tools, machine learning methods and applications in support of Industry 4.0 high-volume manufacturing in the semiconductor manufacturing industry. Serena Brischetto is senior manager of Marketing and Communications at SEMI Europe.

Jack McCauley understands the interplay between video game hardware and human interaction like few others in the industry. He designed the guitar and drums for Red Octane’s (later Activision’s) Guitar Hero video game series. As co-founder and chief engineer of Oculus VR, he designed the Oculus DK1 and DK2 virtual reality (VR) headsets and helped guide the company through its acquisition by Facebook in 2014. Now active in automotive technology, he builds cars at Black Lab, his private R D facility and hardware incubator in Livermore, California. And, in no small feat, he thinks he’s solved the head-tracking problems in augmented reality (AR)/VR headsets – which he’ll demonstrate during his keynote presentation, MEMS Applications in Augmented Reality, October 6 at MSEC 2020. SEMI’s first virtual MEMS Sensors Executive Congress. The event is October 6-8 and 13-15, 2020, and registration is open. I interviewed McCauley to preview his presentation. Register now for MSEC 2020.SEMI: What inspired you to become the first person to use a MEMS sensor in a gaming device?McCauley: When I started designing the Guitar Hero peripherals, I had intermittent problems with the motion tracking. I switched to a Freescale single-axis accelerometer, developed some IP around it, and that fixed the problem. That’s how I became an early customer of MEMS. SEMI: When you pioneered immersive VR gaming experiences at Oculus VR, tech industry analysts predicted widespread adoption of VR for gaming. What do you think happened?McCauley: There are a lot of reasons why VR hasn’t become the standard bearer for gaming. Gaming used to be a solitary activity, but as companies like Microsoft and Sony got behind multiplayer gaming, we realized many gamers found the social aspect more important than the visual aspect. Many gamers are content to play on a 2D screen or on multiple monitors because they’re playing against many people. The proliferation of internet connections worldwide has also promoted the kinship and social aspect of gaming.SEMI: Do you think VR has a place in other applications?McCauley: I think it has a lot of potential in real estate, VR movies, and engineering and design, among other areas. The automotive designer Henrik Fisker, for example, created whole vehicles in a game-engine model. If you wanted to buy one of his cars, let’s say, you could change the color and upholstery, for example, and then view it in a VR environment. SEMI: One of the biggest obstacles to VR adoption is the motion sickness some people experience during game play. What would you do to fix that?McCauley: The vestibular system in the brain, which uses the inner ear, is crucial to helping you balance. If there’s a mismatch between what your eyes see and your brain is perceiving, you’re likely to feel dizzy. I’ve built a VR headset that uses a MEMS pico projector with micromirrors and a small laser for position tracking as well as for facial tracking and modeling. But the platform’s not for sale.Still, many of the technical advances that we’ve made in VR are helping us with AR development. The increasing power of mobile chipsets and GPUs, the decreasing geometry for individual transistors and the way specific chips are processed, screen interfaces that will drive a 4K panel at a high frame rate, plus MEMS devices inside the eyewear for rotations and tracking are all helpful innovations.SEMI: When designing cars in your own lab, you’re doing a lot of work with AR. What do you think of AR’s commercial viability?McCauley: I know there are well-funded AR programs in place at major companies. That’s because mobile-device companies want an omnipresent phone in front of your face. I thought Google Glass, for example, was brilliant, but it was way too early for that product, and there was too much hype behind it.McCauley's latest R D project is a vehicle that incorporates augmented features and a computerized display. The vehicle is a custom built, environmentally friendly super-car with enhanced driver safety and high vehicle performance. AR is appealing because it lets people see through a screen – and have objects appear on that screen – while they are moving through space. My son actually came up with one of the ideas I’m implementing in a car I’m designing. We were driving in Spain, and he suggested that instead of using Google Maps to show me driving directions – which would force me to look down at an infotainment display – a sign could appear on AR glasses that would instruct me how to drive to Italy. That’s just an example of how we’ll use AR. SEMI: After you sold Oculus VR to Facebook, you began investing time and resources into engineering education. Why did you make that choice?McCauley: I’m originally from a blue-collar family, and then I got an education at Berkeley. That made a major difference in my life. When I sold Oculus, I donated to education-focused charities primarily, because an education can lift an entire family out of poverty. Let’s say your family are farm workers, but you get a degree in engineering and land a job at Apple. That could produce a ripple effect. As other members of your family and people in your community see the benefits of your education, they’re more likely to get an education, too. SEMI: What would you like MSEC attendees to take away from your presentation?McCauley: I appreciate what the MEMS industry has done for VR because if Oculus didn’t have a nine degrees of freedom (9DoF) IMU, no one would have bought our company. A new application will come along sooner or later that will require a different type of MEMS technology, and I have total confidence that the MEMS industry will deliver what’s needed. For more information on McCauley’s R D projects or on his position as Innovator in Residence at UC Berkeley’s Jacobs Institute for Design Innovation, visit his website. MEMS Sensors Industry Group® (MSIG), a SEMI technology community that connects the MEMS and sensors supply network in established and emerging markets, enables members to grow and prosper. Visit us today.Jack McCauley is an Innovator in Residence at the Jacobs Institute for Design Innovation, where he mentors students, lectures in courses focused on product design and design for manufacturing, and leads research and development projects focused on applications of augmented, virtual, and mixed reality for design professionals and students.McCauley graduated from Berkeley Engineering with a B.S. in Electrical Engineering and Computer Science in 1986, and credits the time he spent at Berkeley as an undergraduate with helping to ignite his career. Maria Vetrano is a public relations consultant at SEMI.

Today’s mobile devices are smaller, more power-efficient, and have more capability than we could have imagined just a decade ago. Offering ever-increasing levels of user functionality, mobile devices are now ubiquitous, and are rapidly becoming the primary mechanisms through which we interact with the digital world, our physical environment, and one another. An unintended side effect of our dependence on the current crop of mobile devices is that they are driving us to distraction.A major industry dynamic will shake things up for the better. Sensors are getting smaller and more efficient, and they’re offering attractive new functionality, giving us the ability to monitor our air and water quality, assess potential toxins in our food sources, and analyze personal health conditions, to name a few use cases. At the same time, the realization of flexible hybrid electronics (FHE) through new materials and production processes, better integration with other electronic components, more efficient energy production and consumption, and pervasive wireless connectivity are fueling the next generation of devices and experiences. What can we expect from tomorrow’s mobile devices — and how can we manage them, instead of having them manage us?SEMI’s Nishita Rao caught up with Mike Wiemer, Ph.D., VP of Engineering, CTO and co-founder, Mojo Vision, to preview his February 25 keynote, The Art of the Possible, at FLEX|MEMS Sensors Technical Congress (MSTC) 2020, February 24-27 at the DoubleTree by Hilton in San Jose, California.Join us at FLEX|MSTC to meet Mike and other industry influencers advancing innovation in FHE and MEMS sensors. Register now to connect with him at FLEX|MSTC or visit him on LinkedIn.SEMI: Mojo Vision has conducted its own research on human interaction with mobile devices. Why is this important?Wiemer: Our mobile devices have given us access to the information we need and want, improving many aspects of our lives. But our devices have also influenced our relationships and attention to our environment in negative ways. We believe that the next mobile computing platform must improve this situation. Instead of pulling us away from the moment, our devices need to embrace more human-centric engagement while still letting us access information that improves our quality of life. Mojo Vision has worked to understand this problem through our own studies and research so we can better develop an approach to address it. SEMI: How are key technical trends driving size, efficiency and capability advancements in mobile devices?Wiemer: Tiny low-power sensors are enabling ever-smaller feature-rich mobile devices that run longer on a battery charge. Smartwatches are a good example. Just a few years ago, smartwatches were not that much more than small screens on our wrists. Today, we have GPS, EKG/health monitoring, and cellular wireless interfaces all inside the same form factor.As this trend continues, we at Mojo Vision predict that our devices will continue to shrink and become even more personal: They’ll be more continuously worn and matched to our own needs and behaviors. This trend towards invisible personal devices is something we’re trying to accomplish with our solutions at Mojo Vision.SEMI: What is Mojo Vision’s concept of “Invisible Computing?” Wiemer: Our vision of Invisible Computing is based on the idea that our wearable devices should be invisible to those around us, encouraging more human interactions. These wearables should be invisible and unobtrusive to users themselves. Our Mojo Lens, which contains a full display and sensors housed inside a contact lens platform, exemplifies this vision. Using proprietary microelectronics and the world’s densest microdisplay to layer digital images and information seamlessly, Mojo Lens is redefining augmented reality. Our mobile devices today continue to increase the quantity and magnitude of interruptions. We think that shouldn’t happen. As a socially invisible device that delivers contextual, relevant content, the Mojo Lens lets us go about our daily lives, naturally interacting with other people while simultaneously enjoying the benefits of augmented reality. We think Invisible Computing can change our relationship with our devices, as well as seemingly give us superpowers. For more information, download the Mojo Vision report, Device Distraction: Understanding the Problem, Re-Thinking the Solution.SEMI: Can you tell us more about Mojo Lens?Wiemer: At its foundation, Mojo Lens is a nanoLED display, radio and sensor platform, integrated using flex technologies, and placed on your eye to provide important information. Mojo Lens can elevate or suppress this information to decrease reliance on your other devices.Unlike your smartwatch or smartphone, which react to you in a binary manner because they don’t have enough information to make autonomous decisions, Mojo Lens understands the context of your experience. That’s because it’s based on our Invisible Computing platform, which can understand your activity. Mojo Lens recognizes if you’re engaged in a conversation, driving or having a coffee, and it reacts with information accordingly.Mojo Lens could act like a real-time interpreter, for example. When someone speaks to me in a language I don’t understand, I should see “subtitles.” Or if I’m having a conversation with someone, Mojo Lens wouldn’t interrupt me with a notification at that moment. For the 92% of Americans who are interrupted by their devices during conversations every day, this prioritization can boost productivity. More importantly, it can improve the quality of our connections with the people around us.Mojo Vision’s microLED platform offers a world-record pixel pitch of over 14,000ppi and pixel density of over 200Mppi², making it the smallest, densest display for dynamic — or moving — content. SEMI: What would you like FLEX|MSTC attendees to take away from your presentation?Wiemer: It feels like the speed at which people are defining important problems and tackling them is increasing every year. And there are so many important problems to solve: space travel, autonomous driving, electric vehicles, alternative energy, quantum computing, lifespan extension, increased food production, brain-computer interfaces, AR/VR. All these problems seem impossible and “crazy,” until some group of people comes along to put a framework in place that can address them. Interestingly, these frameworks aren’t necessarily new. Rather, they build upon existing technologies and capabilities.MEMS sensors and FHE are good examples. From smart textiles, flexible displays and biological sensors to miniature radars, MEMS sensors and FHE technologies are essential building blocks. Many of the big problems we can imagine today will be solved by stacking today’s MEMs and FHE technologies in imaginative new ways. So what do we do next? I’d like to encourage FLEX|MSTC attendees to first define the problem to solve and then define the technology — rather than starting with the technology solution. Mike Weimer is a serial entrepreneur and proven science and technology leader in complex systems development and integration. Before co-founding Mojo Vision as CTO, Weimer co-founded and served as president at Solar Junction, a high-efficiency solar cell company (acquired) where he and his team set two world records for the highest-efficiency solar cells ever made by humans.After Solar Junction, Wiemer joined New Enterprise Associates (NEA) as an Entrepreneur in Residence where he sourced new investments and helped portfolio companies to develop their business and funding strategies. He is a board director at Stratio Corporation and an advisor at Stanford’s StartX Accelerator. He holds a B.S., M.S., and Ph.D. in Electrical Engineering from Stanford University.For more information, visit Mojo Vision.Interested in engaging with the MEMS sensors supply chain? MEMS Sensors Industry Group is a SEMI technology community that enables professionals in the MEMS and sensors industry to accelerate business results by addressing common challenges and opportunities.Nishita Rao is marketing manager for technology communities at SEMI.

Have you ever wondered if you could “feel” what it’s like to revisit your favorite vacation spot while sitting on a couch in your living room? How about walk through a restaurant overlooking the water’s edge as you enjoy a savory dish, while still sitting on that couch?If you follow trends in consumer electronics, you probably imagined a virtual reality (VR) headset that uses a visual interface to simulate the ambiance of the restaurant as you use a voice interface to scroll through the restaurant’s menu. While the tech world has made great progress in evolving the visual and voice interfaces of VR, immersive virtual food-tasting also requires a digital interface that supports a sense of smell and taste. In fact, the National University of Singapore is conducting research on the topic (see the video), and Project Nourished claims to enable “eating and drinking in a whole new way – by hacking vision, gustation, olfaction, audition and touch – with or without caloric intake” – through VR.We’ve already come a long way in our quest to replicate human senses such as touch, vision (via biometric authentication) and voice to build user interfaces to interact with the digital devices around us. In fact, every invention that permanently changed the consumer electronics landscape in the last few decades has in turn brought to life one of these user interfaces (UI). For example, smartphones proliferated touch, video games such as Nintendo Wii and Sony Xbox brought gesture, and most recently, smart speakers and VR headsets have increased the adaptation of voice and vision.Complexities of UI DesignUI design is a complicated task that builds upon years of research in neuroscience, cognitive thinking and engineering. It must also account for individuality because users interact differently with their digital devices. Some, like me, use their left hand predominantly when interacting with a gaming console. Some have a heavy accent, which can make speech recognition difficult, while those with a hearing disability may prefer touch over voice as a user interface. Application, context and proximity of the device to the user also affect UI. For example, a user interacting with a smartphone at home has the option to touch or speak to the device whereas voice is the safest means to communicate with a car’s infotainment system while driving.Consumers often bring their digital devices wherever they go, but still expect a consistent user experience. Therefore, a natural user experience is the key to UI adoption. A multi-sensory approach combining voice, vision and/or touch could prove the most practical solution. For example, if I were to access my account at a bank ATM, I would prefer visual- or touch-interface authentication for security reasons, but I would still want to use a hands-free voice interaction to switch between the different menus on the machine. In this case, a combination of UIs could provide a more natural multi-sensory experience, albeit one that needs a careful design.UI technology development and adoption are largely influenced by the top four players in the consumer electronics industry – Apple, Amazon, Google and Samsung. Apple pioneered the touch interface with the invention of keyboard-less smartphones, and the rest of the industry followed suit. The introduction of Google Glass kickstarted the VR/augmented reality (AR) segment and opened new applications in the gaming and multimedia entertainment segments. While VR headsets work for gaming – and more recently for selling products and experiences – they are large and cumbersome devices that are uncomfortable to wear for extended periods. These are major hurdles for designers to solve. Voice, on the other hand, offers a hands-free user interface that is a more natural and frictionless compared to alternative UIs.A voice UI needs nothing but a voice command to interact with digital devices. However, it comes with its own complexity of varying user speech characteristics such as accent or volume. More importantly, the need to suppress various background noises for efficient use of voice UIs is critical. While edge computing and/or cloud-based artificial intelligence (AI) are critical technologies to enable battery life and performance of smart home devices, the overarching goal of conversational AI is still far from reality.From a business standpoint, the winner of the race for voice UIs must improve AI capabilities while supporting a strong ecosystem of partners. Amazon, for example, is king of this strategy. The e-commerce giant is building an Alexa Voice Service (AVS) ecosystem by way of its Alexa Fund companies and third-party integrations (partners) to realize its goal of proliferating voice everywhere. These partnerships enable the ecosystem to build end-to-end speech systems that can literally take voice interface products everywhere and promote, among other things, hardware startups that are disrupting the MEMS market with products such as environmentally robust piezoelectric microphones.Energy harvesting near-zero-power always-listening microphones, used in partnership with the AVS ecosystem, are enabling voice UI products to expand into battery-operated applications such as hands-free TV remotes, smart garbage cans, Bluetooth speakers, headsets, hardware appliances and automobiles. A good example of a unique voice UI launched at CES 2019: Housewares designer simplehuman’s voice-activated smart garbage can uses Vesper microphones and AVS. Watch the video.While the future might bring additional digital interfaces, along with multisensory experiences using vision, gesture and touch, voice UI is at the forefront of current technological innovation. Soon, Alexa might help cook dinner without intervention, even turning off the stove when food is burning through the use of a scent-detection sensor integrated with a microphone array. Voice UI continues to astound us with its possibilities, and we’re excited for the journey ahead.With more than 12 years of experience working in speech and voice applications for wireless devices, Udaynag Pisipati is a senior field applications engineer at Vesper. He holds a master’s degree in electrical engineering from University of Missouri and an MBA from Santa Clara University. A firm believer in speech as a natural user interface for human-machine interaction, Pisipati’s areas of interest include everything related to speech processing, including microphones/speakers, signal processing and machine learning.Vesper is a member of MEMS Sensors Industry Group (MSIG), SEMI technology community.

The 3D optical sensing market is once again surging – and it’s all thanks to Apple. What will we see in the next wave of end products enhanced by this technology, how will other market segments approach – and eventually use – 3D optical sensing, and which suppliers stand to gain the most from this very vital technology?Although 3D sensing, facial recognition and optical authentication systems have become only recently hot topics in the consumer electronics market, these mechanisms first made their appearance nearly a decade ago in November 2010. Following that debut, Microsoft soon launched the Kinect system in its Xbox 360 gaming console, marking a milestone as significant as Nintendo’s launch of its Nintendo Wii remote controller in 2010, which catapulted MEMS motion sensors into the high-volume consumer market.The Kinect system used a triangulation-based camera that Israeli developer PrimeSense Ltd. created and then licensed to Microsoft; Apple liked the technology so much that it acquired PrimeSense in 2013. The first version of Kinect applied the Structured Light (SL) method, a depth-sensing principle featuring an infrared (IR) laser projecting dots onto the scene, with a monochrome CMOS sensor measuring the differences in the acquired pattern. The second version of Kinect used the Time-of-Flight (ToF) principle.Kinect for Xbox360 was not only a successful consumer product; it also sparked a new market, thanks to the relatively low cost of the 3D sensing solution. By using the same hardware for Xbox 360 as in its first version of Kinect, Microsoft allowed developers to design their pet projects in the Kinect environment. Adding hand gestures controls to a PC, creating a user-controlled virtual dynamic light (see Kimchi and Chips’ demo), and developing an inexpensive hologram generator (see “Princess Leia” video from the MIT Media Lab) are just a few examples of ecosystem developers and DIYers applying their creativity to Kinect.Apple Goes 3D with Face ID3D optical sensing has expanded from gaming consoles to the smartphone. In 2017 Apple presented its Face ID camera system for the iPhone X, which they launched to celebrate the 10-year anniversary of the iPhone. Face ID is the result of a longer term strategy for Apple, the byproduct of several company acquisitions to expand know-how in 3D sensing and augmented reality (AR)/virtual reality (VR). Between 2015 and 2018, Apple acquired the camera-module maker LinX (2015), the AR startup Vrvana and the imaging sensor firm InVisage Tech (both in 2017), and AR glasses’ designer Akonia Holographics (2018).For a company that has always innovated on its own terms, Apple’s idiosyncratic approach called for deployment of the Structured Light method combined with a ToF device. The result is an amalgamation that utilizes the best features of the two mechanisms, even if the combination is one that is expensive. Apple’s addition of a near-infrared illuminator to its ToF device enhances the system’s effectiveness under most light conditions while also improving the reliability of Face ID; the overall outcome is a more satisfying user experience. The ToF component, which STMicrolectronics supplies, makes use of so-called single-photon avalanche diode (SPAD) receivers that can work with any target material and color, although a higher target illumination is required to obtain good accuracy.The other core components of the Face ID system are the Vertical Cavity Surface Emitting Laser (VCSEL, from Lumentum) and a dot projector (from ams/Heptagon), assembled together in an optical package. Apple’s expensive but reliable approach explains the company’s inclusion of the Face ID system in its latest smartphone and tablet offerings – across the iPhone Xs, Xs Pro and Xr as well as in the latest iPad Pro models. Apple’s Face ID uses facial recognition for authentication on a range of iPhone and iPad Pro models. Image courtesy of Apple. Chinese Phone Makers Get into the GameMeanwhile, other mobile handset manufacturers are rumored to be working on Face ID-like systems or have already presented similar solutions, albeit through a variety of approaches. Some have chosen to use standard ToF devices while others have adopted an SL tactic. In many of these designs, which happen to target Android systems, OEMs generally include a fingerprint sensor as a fallback biometric option to their own nascent 3D facial recognition systems. The fingerprint sensor operates in either standalone mode or integrates into the display.Chinese handset maker Oppo, for instance, uses the SL method on its Find X model with algorithms coming from Megvii. Oppo claims its equivalent of Apple Face ID is faster. I have heard that Vivo has been working on a ToF camera since mid-2018, which it claims provides greater accuracy and security in end-applications such as secure payments and unlocking the phone.Chinese technology giant Huawei’s first 3D facial sensor appeared in its Mate 20 Pro flagship mobile phone. Aside from providing facial biometrics, the front-facing 3D sensor doubles as a 3D scanner, enabling users to digitize live objects that they can then manipulate in 3D AR applications. While still a novelty, the application highlights the use of 3D light sensors beyond that of biometrics. Xiaomi’s Mi Explorer Edition smartphone features a complex SL 3D module to enable 3D facial scanning although it looks like a clone of the Apple solution.Overall, the importance of facial recognition is no longer a matter of dispute, given that Apple’s rivals are now developing counterpart offerings of their own. Leaked code from the next revision of the Android operating system (revision Q), now under development by Google, has confirmed as much. Big and Getting BiggerIHS Markit forecasts that global revenue for ToF sensors in the 3D optical sensing market will surpass $500 million in 2019, up from $370 million last year. We also predict that the ToF market will grow in the coming years, spurred by combo solutions integrated with other light sensors in the same package. This will lead to a cheaper bill of materials (BOM) compared to the BOM for the SL method.At the same time, IHS Markit forecasts that the total market potential for light sensors will be worth much more, reaching $1.5 billion by year 2022. That’s because after a solid start with gaming consoles, 3D sensing has matured and consolidated in the massive smartphone arena.A segment of 3D Sensing’s future growth will come from other use cases and applications that are emerging outside consumer electronics and mobile. These include people-counting and -tracking in consumer and industrial applications, landing-aid and obstacle-avoidance functions in drones, and car-trunk (boot) opening with foot gestures, as well as gesture recognition and passenger detection in automotive. IHS Markit predicts steady growth for ToF and other light sensors. All told, the ToF approach appears to have a greater chance than the SL method in gaining a larger market share, leading to a cheaper and smaller BOM along with reduced integration costs in system assembly and calibration.Sometime this year, Apple and other handset OEMs may include a ToF-based 3D camera on the back of the iPhone to support more immersive gaming experiences and new AR/VR applications. This will further boost the 3D sensing market.To be sure, other mature technologies are available as valid alternatives to optical 3D sensing, including ultrasonic, mmWave and radar. These alternative technologies may gain part of the total market now commanded by 3D sensing, in use cases such as obstacle-avoidance or in-cabin presence detection.To learn about 3D Optical Sensing and Light Sensors from IHS Markit, go to: https://technology.ihs.com/606483/light-sensors-for-consumer-mobile-report-2018Manuel Tagliavini, a principal research analyst at IHS Markit, covers MEMS and sensors technology.Manuel Tagliavini joined IHS Markit in 2017. His key areas of focus are MEMS and sensors for mobile and consumer technologies. He is responsible for the tracking of sensors in handsets, tablets, laptops, and sports and fitness products.Prior to IHS Markit, he spent over 10 years with STMicroelectronics, working in various roles including product engineering, program management, and marketing and business development in the company's MEMS division.Tagliavini earned an Executive Master of Business Administration at SDA Bocconi School of Management and a Master of Science in Electronic Engineering from the University of Parma, both in Italy.Stay tuned with the technological advances and market trends in the MEMS Sensors ecosystem. Join MEMS Sensors Industry Group (MSIG), the SEMI technology community that connects the MEMS and sensors supply network in established and emerging markets, allowing members to grow and prosper.

As group vice president of the Analog MEMS Group and general manager of the MEMS Sensor division at STMicroelectronics, Andrea Onetti brings nearly three decades of experience in MEMS, sensors and audio systems to his leadership role at one of the world’s most successful electronics and semiconductor manufacturers. During his keynote at FLEX and MEMS Sensors Technical Congress 2019, February 18-21 in Monterey, Calif., Onetti will address the criticality of sensor accuracy in advancing automotive, industrial and consumer applications. SEMI’s Maria Vetrano spoke with Onetti recently to give FLEX/MSTC attendees a preview of his presentation. SEMI: What are some promising advancements in sensors for autonomous cars? Onetti: The avionics industry is already successfully applying sensors for autonomous operationl. Inertial navigation systems (INS) support the operation of planes during flight, both after takeoff and before landing. Unfortunately, the technology in these navigation systems is expensive and not scalable, and they are hampered by reliability limitations in an automotive environment.Following the steady progress that we have made with MEMS inertial sensors in consumer applications, we are on the cusp of realizing greater accuracy in temperature and time – finally delivering the performance required for autonomous driving. Because we can scale in production – we’re now manufacturing more than a billion units a year – we can select the cream of this production crop for adoption in cars. Consequently, we should see Level 3 and Level 4 autonomous driving for consumers very soon.SEMI: How are companies using sensors to monitor and track their assets in industrial applications? Onetti: Predictive maintenance and asset tracking are the two main verticals in Smart Industry. The adoption of multiple sensors for condition monitoring is helping to detect the faulty operation of equipment and to detect early signs of issues that are otherwise difficult to capture. Ultrasonic microphones can detect leaks in a pipe at an early stage, accelerometers with high bandwidth can act as micrometers, and accurate temperature sensors can catch overheating. Similarly, in asset tracking, we use temperature monitoring in combination with inertial sensors to detect problems during the transport of goods. Shock sensors with extremely high full scale (up to 8000g) can tell whether a lightweight envelop has been dropped. Pressure sensors can switch off a radio system when a cargo plane takes off and can mute smart trackers in compliance with flight regulations. We really can do almost anything! A full slate of ST sensors and microcontroller units (MCUs) enable WEG’s small but powerful motor sensor, which listens to a motor, feels its pain, and shares that information with engineers, operators and others to diagnose problems before they happen. Image courtesy of STMicroelectronics. High-accuracy motion, environmental and proximity sensors are crucial to VR/AR. Image courtesy of STMicroelectronics. SEMI: How will sensors advance user experiences in consumer electronics, such as VR/AR systems?Onetti: Virtual reality (VR) and augmented reality (AR) are great examples of promising consumer technologies that will become pervasive as performance of inertial sensors improves. First, we need super accuracy in time and temperature to provide the right experience to users. To achieve this level of accuracy, we need a major step forward in performance, and that includes power consumption and miniaturization. Fortunately, we are constantly making progress in the high-accuracy motion, environmental and proximity sensors that are critical to these systems. While the scale is vastly different between VR/AR and automotive, the requirements for AR/VR systems are pretty similar to those that will enable autonomous cars. A growing variety of sensors (environmental, microphone, proximity, motion) – combined with a sensor hub in an MCU – are central to VR controllers (above) and VR head mounted displays (below). Images courtesy of STMicroelectronics. SEMI: We don’t hear much about the criticality of higher accuracy in sensors. Why is improving accuracy in sensors especially important – and what role do calibration routines play in achieving higher accuracy?Onetti: A sensor is more than just the performance of the relevant function. It is also the intrinsic accuracy that it brings. This accuracy is tuned by calibration, which is typically an expensive process done at the end of product manufacturing or – better still – during earlier stages of manufacturing.Today more applications require sensors with higher accuracy, which necessitates investing more time in calibration, leading to higher cost.MEMS technology can help by offering solutions with intrinsic higher accuracy, which reduces the cost of calibration for product manufacturers. This naturally delivers major benefits to OEMs and, ultimately, their customers.SEMI: What would you like FLEX and MSTC attendees to take away from your presentation?Onetti: As attendees explore the wide variety of available sensor solutions for their end products, I would ask them to prioritize the role of accuracy in sensor selection – because improved accuracy means higher quality data, and higher quality data means better decisions with reduced need for data processing.While designers understand the role of calibration routines in qualifying individual components for specific applications, it is the continuous evolution of MEMS technology that offers the best possibility of breakthrough reductions in time and cost of these calibration routines. This makes MEMS sensors more attractive and affordable than similar sensor components based on different technologies. Andrea Onetti will present Accuracy Enables MEMS Sensor Pervasion at FLEX/MSTC on Tuesday, February 19 at 11:00 am.Register today to connect with him at the event. To learn more about STMicroelectronics, click here. Maria Vetrano is a public relations consultant at SEMI.MSTC FLEX 2019 is organized by MEMS Sensors Industry Group (MSIG) and FlexTech.Maria Vetrano is a public relations consultant at SEMI.

Do you email your doctor when you have a tickle in your throat? Wear a fitness tracker or use an app to monitor your sugar levels, exercise or nutrition? If so, congratulations! You are a part of the rapid growth of digital medicine.Since you’re on the leading edge of this trend to enhance the efficiency of healthcare delivery, you might enjoy learning more about how the medical industry is transforming healthcare in this collection of podcast episodes and video. These are my top picks and just the tip of the proverbial iceberg in how modern medicine is taking today’s technology and applying it to best practices for remote patient monitoring, medical diagnosis, rural healthcare and more.Enjoy! And let me know if you find any others worth the listen!1. Inside Angle: 3M Health Information Systems - Telemedicine: Enhancing Access to Improve OutcomesAccess to healthcare can be a matter of life or death. For some patients, this may mean taking a day off work and driving for hours because services are not available in their hometown.Inside Angle host Dr. Gordon Moore interviews Barb Johnston, co-founder and CEO of HealthLinkNow, about the implementation of telemedicine programs. They discuss how technology impacts telemedicine adoption and related regulations and benefits clinicians and patients in the telepsychiatry and mental health industry. 2. NPR’s The Salt: What’s On Your Plate – This Chef Lost 50 Pounds and Reversed Prediabetes With A Digital ProgramThis short audio clip and article dives into lifestyle and wellness apps designed to motivate users to eat healthier, exercise more and – in some cases – save them from a preventable disease, like diabetes.Wellness apps like Omada Health rely on smartphones, e-coaching, electronic nudging and other methods to encourage users to make and, more importantly, stick to changes that can improve their health. And it’s catching on as other organizations, such as The Centers for Disease Control and Prevention, recognize the difference lifestyle-change programs designed to prevent diabetes are making. 3. Red Hot Healthcare: Episode 57 - Bluer Skies for TelemedicineLast year alone, venture capitalists poured $7 billion into telemedicine. As an emerging concept, telemedicine has a long road ahead until it’s fully integrated and adopted into modern medical practices. Nathaniel Lacktman, partner and health care lawyer with Foley Lardner LLP, discusses legal issues and hurdles surrounding telemedicine today with Red Hot Healthcare host Dr. Steve Ambrose.Lacktman pointed to healthcare providers such as Mercy Virtual Care Center, Mayo Clinic and MDLive that are likely to lead the telemedicine race. This podcast episode is a great listen for entrepreneurs and medical professionals who are interested in learning about compliance and business considerations for the implementation of telemedicine. 4. Digital Health Today: Episode 56 – Eren Bali on Building the World’s Largest Connected Care NetworkEren Bali, CEO and co-founder of Carbon Health, is out to change our fragmented traditional healthcare system with his aim to build the world’s largest connected care network.For Bali, it all started with his sister’s experience consolidating his mother’s health records – scattered over 20 different systems. And from that challenge, his idea to create a universal network that aggregates patient medical records was born.Bali and Digital Health Today host Dan Kendall discuss Bali’s launch of Carbon Health, the new medical records system’s first implementation in a San Francisco primary care clinic, and how providers and patients can get on board with this model. 5. Digital Health Today: Episode 58 – Brennan Spiegel Gets Real About Virtual RealityThis episode of Digital Health Today centers on how Virtual Reality (VR) is being used to enhance patient care. Host Dan Kendall speaks with Dr. Brennan Spiegel, Director of Health Services Research at Cedars-Sinai Health System and Professor of Medicine and Public Health at UCLA, about how immersive technology like VR and Augmented Reality (AR) can improve patients’ experience as well as alleviate pain, anxiety, depression, addiction and more. In a particularly interesting segment, Dr. Spiegel calls the hospital a biopsychosocial jail cell and underscores its importance in not just treating physical ailments but, more wholistically, also addressing the psychological and social wellbeing of patients. 6. TED2017: Raj Panjabi – No one should die because they live too far from a doctor What do you do when access to a doctor means rowing a boat for hours? Millions of people around the world lack access to health care because they live in a remote town or village.In this TED talk, Raj Panjabi, physician in the Division of Global Health Equity at Harvard Medical School, Brigham and Women’s Hospital, and co-founder of Last Mile Health, offers a solution to the problem of healthcare access in the form of the Community Health Academy, a global platform to train and connect community health workers by leveraging devices like smartphones to bring preventative healthcare to even the most far-flung regions of the world. 7. GeekWire’s Health Tech Podcast: How AI is making humans the ‘fundamental thing in the internet of things’Can AI predict which patients are at risk for chronic diseases using their old medical records? This episode of Health Tech Podcast dives deep into the future of healthcare with a look at the potential for AI -- “augmented intelligence” or “assistive intelligence” – to improve patient outcomes, the focus of Ankur Teredesai, a data scientist at the University of Washington and co-founder and CTO of health AI startup KenSci.Clare McGrane, host of GeekWire’s Health Tech Podcast, speaks with numerous experts in this field. They compare precision health to a modern car constantly monitored by microprocessors. The minute something is wrong, you get a warning to take the car to a shop to resolve the issue. The same concept can be applied to humans and deliver big healthcare impacts as research in AI and machine learning continues to evolve. 8. NPR’s Shots: Can Home Health Visits Help Keep People Out Of The ER?In Washington D.C., the city with the highest per capita 911 call volume in all of the U.S., Mary’s Center is piloting a program to provide primary care telemedicine to patients who cannot, or in some cases, do not want to visit the clinic. NPR covers the story of Medicaid patient Dennis Lebron Dolman, who is currently receiving a mix of home visits and virtual treatment.Besides providing healthcare access to rural regions, telemedicine has the potential to reduce emergency room visits in cities as well as improve the health of patients in the long run. Learn more at SEMICON West’s Smart MedTech TechXPOT on Digital Medicine and Remote Patient Monitoring. Hosted by NBMC on July 12, 2018, from 10:30 AM to 12:30 PM, the event will feature healthcare industry leaders exploring state-of-the-art healthcare practices and the future of medical technology. Registration is now open!Amy Ly is a Technical Programs Marketing Coordinator at SEMI.