smart speakers

The seemingly simple act of commanding consumer devices by voice is a choice that nearly 118 million Americans now make every day, according to a recent report from eMarketer, the digital marketing research firm.While the voice interface is convenient for users, its implementation comes at the potential loss of individual privacy. The reason? Always-on, always-connected voice-first devices such as Amazon Alexa and Google Home require a wall plug and an internet connection to powerful cloud processors, making it possible for cloud companies — however benignly — to collect data on personal habits, location and conversation that were never intended for sharing. Move processing to the edgeTo address concerns over user privacy, device designers are attempting to do more of the audio processing within the consumer device, rather than sending users’ voices into the cloud. Moving more processing to the edge is a trend across the Internet of Things (IoT) industry, and not just for voice data but for other types of sensitive or proprietary data as well.Yet designers have realized limited success because the conventional approach to always-listening edge processing is notoriously inefficient: It digitizes and processes 100% of incoming sound data even though up to 90% of the data is irrelevant noise. This digitize-first approach wastes vast amounts of system power digitizing and analyzing the audio signal as it searches for a wake word when there isn’t even speech present, making it impractical for use in small, battery-operated devices.Workarounds don’t workTackling this power issue is critical to keeping private data secure. Unfortunately, it’s also exceptionally difficult. Design engineers have tried workarounds to decrease power consumption in an always-listening system, including duty cycling and reducing the power of each individual component in the audio signal chain that handles the data. The reality is that these kinds of approaches don’t address the root cause of the problem: too much data.To truly tackle the problem, we need to change our approach to a system solution, not a component solution. By moving to a more efficient edge architecture that intelligently minimizes the amount of data that moves through the system, we can focus the system’s energy resources on analyzing voice and not on searching for a wake word in irrelevant noise. Analyze, THEN digitize It’s time to move away from the digitize-first approach that has dominated voice wake-up device architecture since the invention of voice-first applications.Inspired by the way the human brain efficiently filters incoming information, differentiating, for example, a dog bark from a baby’s cry, an ultra-low-power analog machine learning technology is changing this paradigm. For the first time, device designers can use low-power analog machine learning to detect which data are important for further processing and analysis prior to data digitization.Leveraging an analyze-first architecture, a new analog neuromorphic semiconductor platform allows the higher-power-processing components in the system to stay asleep until voice has actually been detected, and only then does it wake them to listen for a possible wake word.Delivering a post-microphone audio chain that draws as little as 25µA of current when always-listening and collecting preroll data, this analyze-first architecture allows designers to extend battery lifetime significantly. That’s the difference between smart earbuds that run for weeks instead of hours or a battery-powered smart speaker that runs for months instead of weeks.More importantly, it’s the difference between the current always-listening devices that indiscriminately record and send all sound data to the cloud, and one that has the localized intelligence to select and send only the relevant data, reducing the user’s vulnerability to the loss of private data.Balance convenience with privacyThe trade-off between making our lives easier and keeping our personal information private is a choice that we are asked to make throughout our day in a hundred different ways. Bringing more audio processing capability to the mobile device without draining the battery is the first step toward delivering more secure voice-first solutions. But to succeed in this effort, we must shift to a bio-inspired architecture that determines which data are important and requires further processing at the earliest point in the signal chain. Once we move to the analyze-first approach, only a small fraction of the tens of zettabytes of data collected by the forthcoming generation of always-on IoT devices will require further processing in the device and in the cloud.A better balance between cloud and edge processing is a better balance between convenience and privacy, and that’s a win for everyone.About the AuthorTom Doyle is CEO and founder of Aspinity. He brings over 30 years of experience in operational excellence and executive leadership in analog and mixed-signal semiconductor technology to Aspinity. Prior to Aspinity, Tom was group director of Cadence Design Systems’ analog and mixed-signal IC business unit, where he managed the deployment of the company’s technology to the world’s foremost semiconductor companies. Previously, Tom was founder and president of the analog/mixed-signal software firm, Paragon IC solutions, where he was responsible for all operational facets of the company including sales and marketing, global partners/distributors, and engineering teams in the US and Asia. Tom holds a B.S. in Electrical Engineering from West Virginia University and an MBA from California State University, Long Beach. For more information, please visit https://www.aspinity.com/Technology.Aspinity is a member of MEMS Sensors Industry Group (MSIG), a SEMI technology community, that enables the MEMS and sensor industry to address common challenges, innovate and accelerate business results.

MEMS technology has changed human interaction with electronic devices. Introduced in the 1990s, the first mass-market MEMS devices were used for inkjet printheads and automotive airbag crash sensors. Today, MEMS are ubiquitous, with billions of the tiny devices adding intelligence and interactivity to smartphones, smart speakers, wearables, automobiles, biomedical devices, remote monitoring and event detection systems, and countless other applications. Integrating MEMS with Flexible Hybrid Electronics (FHE) is an important step in the evolution of this miniaturized intelligent sensing technology, paving the way for its use in new classes of flexible, conformal devices.The integration of the two technologies promises to breed new applications in small form factors but also presents challenges inherent to FHE design and fabrication processes. SEMI’s Nishita Rao caught up with Nathan Pretorius, prototyping and automation engineer, NextFlex, to discuss MEMS-FHE device integration challenges and opportunities ahead of his February 26 presentation, Integrating MEMS Devices in FHE, at FLEX|MEMS Sensors Technical Congress (MSTC) 2020, February 24-27, 2020, at the DoubleTree by Hilton in San Jose, California.Join us at FLEX|MSTC to meet Nathan and other industry influencers advancing innovation in FHE and MEMS sensors. Register now to connect with him at FLEX|MSTC or visit him on LinkedIn.SEMI: Why is integrating MEMS devices into FHE systems important? What new use cases might it enable?Pretorius: The main value proposition of integrating MEMS devices into FHE is that it allows MEMS devices to exist in a different form factor than was possible previously, giving us high-quality MEMS sensors on the flexible and conformable platform of FHE.Ease of application, flexibility, lower cost and rapid iteration on a design are just some of the benefits of FHE devices. And because there are few robust FHE sensors that overlap with MEMS’ capabilities, when you combine the two, you get a lot of compelling uses. That’s why NextFlex is working with agencies and companies to evaluate MEMS’ integration, including using bare MEMS die with microfluidics and promoting new ways of attaching and packaging MEMS die for use with FHE. SEMI: Why is FHE an ideal platform for integrating various types of sensors?Pretorius: MEMS integrated with FHE devices are ideal for rapid design and deployment of data-gathering sensor nodes — which we can iterate for specific applications. A few examples include on-body health monitoring devices for bio-fluids analysis, medical pressure sensors for monitoring blood pressure, and peel-and-stick sensors nodes for infrastructure monitoring. In terms of design and production, FHE devices support rapid prototyping, allowing for instantaneous design-iteration cycles. This speeds design-to-production over traditional rigid PCBs and copper flex because the feedback cycle time between design, manufacturing and testing is shorter, accelerating time to market. What’s exciting about FHE technology is that a variety of sensors or components, including MEMS, can be designed into the base system to easily customize it for a specific application. In addition, our experience shows that when compared to a traditional rigid PCB, an FHE board reduces manufacturing steps and device weight by two-thirds and, perhaps most importantly, converts the device to a thin, conformal shape that makes possible products in new form factors. SEMI: What are the primary challenges to integrating MEMS with FHE? What is NextFlex doing to help device manufacturers address these challenges? Pretorius: There are a few challenges, some of which are device-specific. Most recently, I’ve been focusing on inertial and timing devices, including accelerometers, gyroscopes and resonators. There are a few technical challenges involved in the process of getting the devices from the wafer to an FHE substrate. The wafer processing is very important, especially the dicing and thinning steps. After thinning and dicing, the die is placed onto the FHE substrate. The stresses caused by bonding to the substrate have to be understood and characterized. After placing the die, you then have a calibration step, which is normally performed after the device is packaged. With a MEMS die placed onto directly onto an FHE substrate, calibration then must be done.Finally, the device encapsulation is important, since on an FHE substrate the hard-to-soft material transition is very important to mitigate stresses to rigid component interfaces. We have also been looking at how to work with devices that have damping vents. Flexible encapsulants are inherently more permeable to gases and water vapor than hard encapsulants, so studying the encapsulation of MEMS devices on FHE is another area of interest. NextFlex has been working in a supporting role to evaluate best design practices and best attach and integration methods. In addition to our ongoing collaborative programs, NextFlex is developing the FHE manufacturing ecosystem to include system and component manufacturers and designers, product developers, and materials and equipment providers.SEMI: How do we facilitate closer collaboration between the FHE manufacturing ecosystem and MEMS suppliers such as MEMS device manufacturers, product developers, and materials and equipment providers?Pretorius: It’s important to include manufacturers early in the design process so we can identify challenges up front. That’s why NextFlex spearheads technology road-mapping efforts that include representatives from across the manufacturing ecosystem. We use the roadmaps to prioritize challenges that we can address effectively through collaboration, focusing the industry on solving problems through Project Calls that reveal integration challenges and results from real devices and that tell us how the materials and equipment actually perform with a real device.NextFlex keeps the information flowing, holding quarterly project update webinars to share results. As current devices are optimized for the process in which they will be used, we learn a lot from the project performers who make FHE system demonstrators — and we share that information with the member community. SEMI: Can you point to an example of a successful MEMS-FHE device integration?Pretorius: MEMS-FHE integration is still in the early stages, but we are working on several projects including a DARPA Seedling project for which we have integrated MEMS sensors into FHE systems for testing and evaluation. We plan to continue this work by integrating MEMS and FHE devices using methods that support mass production.SEMI: What would you like FLEX|MSTC attendees to take away from your presentation?Pretorius: We would like to see the FHE community work more closely with MEMS device manufacturers. For example, NextFlex often works with manufacturers to gain access to bare die, which is still a significant hurdle in making devices.The best way to speed things along is to get involved. We encourage FLEX|MSTC attendees to join NextFlex. As a prototyping and automation engineer at NextFlex, Nathan Pretorius explores new print methods for prototyping and automation using novel materials and processes. Pretorius currently focuses on how best to apply software scripting and machine learning to streamline FHE processes. Prior to joining NextFlex, he researched the strengths of roll to roll and screen printing on printed electronics designs, including capacitive touch interfaces, FHE passive component design, and antennas. Nathan holds a Bachelor of Science degree in Graphic Communications from Clemson University. FLEX|MSTC is organized MEMS Sensors Industry Group (MSIG) and FlexTech, SEMI technology communities focused on the growth of MEMS sensors and the flexible electronics supply chain, respectively.Nishita Rao is marketing manager for technology communities at SEMI.