Robotics & Automation News

Where Innovation Meets Imagination

Smartphones have created a whole new world of opportunities, says Hillcrest Labs boss

Even for Apple, selling 60 million units of anything would be a big deal. For most small and medium sized companies, just six units might be the start of a wonderful voyage in the world of enterprise. 

For Hillcrest Labs, the journey started more than a decade ago, and over that time, the company has reached the milestone of having sold 60 million units.

Having started out selling software, the company moved into bundling that software into chips, which have become more elaborate over time and found an increasing number of applications.

In an exclusive interview with Robotics and Automation News, Chad Lucien, senior vice president of business development at Hillcrest Labs, tells the company’s story. 

Chad Lucien, SVP of business development at Hillcrest Labs
Chad Lucien, SVP of business development at Hillcrest Labs

“Hillcrest really started as a software company,” says Lucien. “We have a product line that now spans from software to single-chip solutions to module solutions.

“We started our business more than 15 years ago, designing technology for the TV industry, and one of the core inventions at that time was a MEMS-based, motion-sensing remote control.

“To put that in perspective, you could think of a remote control like a Nintendo Wii remote that is much more accurate and doesn’t require external sensors to operate.

“That, the foundation of that technology, and the expertise we gained working with inertial sensors has enabled us to – over the years – branch into other applications, including robotics, which is really one of the more recent additions to our portfolio in the past couple of years.

“Our company has shipped more than 60 million units – that’s software and hardware combination units – into the marketplace. The majority of that volume has been in the consumer electronics market.

“And just in the past six months, back in December, our company was acquired by InterDigital, which is a technology and intellectual property company that’s primarily focused on standards-essential wireless technologies, like 3G, 4G, and 5G, and in the IoT market.

“Hillcrest Labs is now a division, or subsidiary, of InterDigital, and our focus remains the same. We’re selling sensor data processing software and hardware into the robotics market, the TV market, the augmented and virtual reality market, and making the products available for the broader internet of things devices.”

It’s all letters and numbers to me 

hillcrest BNO080
Hillcrest Labs’ BNO 080

As well as still offering software solutions, Hillcrest Labs has a number of chips that come with its code – principally the BNO 080, and the newly launched FSM 300. It also has a range of other components that and some complete products.

“The FSM 300 is a sensor module that is targeted at consumer electronics and IoT applications,” explains Lucien. “It’s a sensor module that includes an accelerometer, gyroscope and a magnetometer.

“The key aspect here is that we are able to deliver what has traditionally been an industrial grade sensor performance with a product that can meet the price of consumer products and lower-cost IoT devices.

“That’s really made possible because of Hillcrest’s core business. Our business is supplying software and components to the consumer electronics industry as well as various applications in the IoT market to enable high-performance, high-quality sensor processing in a variety of products.”

Hillcrest recently launched the FSM 300 at the Sensors Expo and Conference, the annual event in San Jose, in the US.

“We go to Sensors Expo with two purposes,” says Lucien. “One is a sales event. So, our VP sales who works on my team, he attends the event, he sets up customer viewings, and he uses the time to identify new, potential prospects.

“The other thing we do at Sensors Expo is, we usually send one or two of our research and development team to really understand what the technology is in the sensor market and the surrounding ecosystem, identify potential technology partners, and also identify potential applications of our technology that we should be investigating further.”

The difference between the chips

fsm300_square
Hillcrest Labs’ FSM 300

Chips are becoming more complex, more powerful and more application specific, but there’s still room for flexibility in the products Hillcrest supplies.

Lucien says: “The BNO 080 is a chip. So it’s a single component that has three sensors in it and a microprocessor and our software.

“The FSM 300 is quite analogous to the BNO 080. It has a very similar set of software. Actually, from a robotics and autonomous vehicles perspective, it has the same software, but there are some variations when it comes to other applications.

“The difference is that the FSM 300 comes in a module form factor, which means it can be designed in a little bit more easily than the chip itself, and it comes pre-calibrated.

“Which means that, in the factory, we calibrate the sensors, so the performance of the FSM 300 out of the box will be higher quality, particularly in a robotics scenario.”

“In terms of that robot use case, the BNO 080 has that same software in it. The advantage with the FSM 300 is that it becomes pre-calibrated better once you install it in the device.

“The BNO 080 would require the manufacturer to run additional stacks in the production process.”

It’s all about the robots

hillcrest robotic vacuum crop

Since our website is all about robotics and automation, it’s helpful when someone explains how their solution addresses a particular application space in that sector.

“Being an autonomous or semi-autonomous machine, the robot needs to know how to navigate its environment, and there’s lots of different ways to navigate the environment.

“You can use GPS, particularly if you’re outdoors, you can use lidar, laser scanning or cameras to get a visual view of the environment that you’re operating in.

“But the other piece that you also need is an inertial component, which is really what helps the robot maintain its heading – helps the robot to drive straight, or drive on the path that you’re intended to drive on.

“Other systems, like a camera or a laser system, are part of a SLAM system.”

SLAM stands for simultaneous localisation and mapping, which is basically the system that tells the robot where it is in the image world and how to navigate in that world.

So, as Lucien explains, in a basic SLAM system, you have a camera which will typically look up at the ceiling and then out in front of you, and it provides a rudimentary map – it can see where the walls and the ceiling meet, it can see the patterns in the ceiling, and it basically can get a sense as to the shape of the room and where it is in that room.

Lucien continues: “Then there is an inertial system, which is what we provide – based on our BNO 080 or the FSM 300. And that is effectively providing the heading, which means the robot might [metaphorically] wonder, ‘Which direction am I pointing?’

“And as the robot is driving – and let’s say you have a vacuum cleaner that’s vacuuming your living room – as it’s driving along one wall in your living room, as it makes a turn at the other end, the inertial system ensures that the robot turns the full 180 degrees to come back the other way.

“And when it comes back the other way, it makes sure that it continues to drive straight, because the sensors on their own and the camera system are imperfect. So, if you don’t have a good system in place, that robot will begin to drift, so you’ll end up with a robot that kind of wanders around your room rather than drive in a specific pattern – there and back.

“So that’s really the key component of the inertial system.

“And also, as the robot drives on uneven surfaces in your home – so, you might have hardwood flooring with a rug, and if one wheel or two wheels goes up on the rug, if you do not have the right set of sensors for accurate fusion, inertial sensor processing, when the robot goes up on the rug, it will not realise that it’s done that and it will think that it’s turning and might compensate for that turn and turn again in the other direction.

“So that is really the key here. We have designed a solution that is reasonably turnkey – it’s still a chip or a module. You can put that in your robot, it allows it to drive straight even in adverse conditions, and it maintains its heading better than the other solutions that we have seen our customers using in the market today.”

Everyone needs a robotic vacuum cleaner

For cheapskate journalists, everything seems expensive. So, buying a small, robotic vacuum cleaner for $1,000 is basically unthinkable. But maybe if it was sold for a lower price – say, one or two hundred dollars, but closer to one?

“That’s about the price point that a lot of people in the industry say that’s going to drive the mass market volume – $299 or less,” says Lucien.

“You can buy a couple of models at that price point, but you’re not getting the highest technology solutions.

“Like anything, you’ll see the prices come down as the volumes go up and as the technology matures.”

Building on software foundations

hillcrest vr goggles crop

Even though software is still the foundation and ongoing driver of Hillcrest’s business, hardware has become increasingly important.

Having started with basic chips, the company has found that clients are asking for more complex hardware.

The BNO 080, Hillcrest’s early foray into hardware, has three sensors and a microprocessor – in a 5 by 4 millimetre package size. The capabilities of the BNO 080 are also available on the new FSM 300, but on a PCB form factor.

“What we found is that there are some market categories that particularly like to buy a module,” says Lucien.

“Sometimes it’s because they want to be able to swap multiple vendors in and out to enable to complete the socket more effectively.

“Some of them like the module because it’s just easier to prototype with and develop with. If they already have a system design, putting the module in often means they don’t have to redesign their main PCB.

“They can add a sister board or a carrier board to attach the module and not have to redesign the whole system.

“So the module has a lot of benefits, especially for certain customers ad certain markets.

“But with some customers, as their product become higher volume, we’re able to move them from a module to a chip, which would save them some cost, and move them from a chip to a software solution, which gives them more flexibility.

“So, large manufacturers who are shipping significant volumes often like to control the entire system, and we have the flexibility to deliver the software in those cases.

“But in many of the robotics cases and the more niche IoT use cases, the module is really the ideal solution because it allows very fast time to market.”

Much more than a calculator 

hillcrest ZTE-axon
Hillcrest Labs’ technology has been integrated into ZTE phones

For most people who aren’t experts, chips might seem like general-purpose, commodity items, which simply process data – it doesn’t matter what type of data.

But with more companies – from large semiconductor companies Intel, Nvidia and others to tech giants such as Apple and Google – developing chips that do specific work, like artificial intelligence, it makes you wonder where the market is going.

Are we going to see more specialised chips that come pre-loaded with specialised functions for specific industries and applications, or is that all just hype?

“You’re definitely going to see more specialised chipsets that are designed for specific target markets or a specific application,” says Lucien.

“The advantage in those solutions is going to come in large part from software – it’s the specific software that’s going to enable the application.

“But what we’re seeing is that the microprocessor suppliers – as in traditional MCU suppliers, using ARM Cortex-M based systems, as well as microprocessors that are using CSP cores – things from Synopsys or keyed-in flash Tensilica or Ceva are also becoming much more prevalent.

“We’re not necessarily seeing a trend towards companies like Hillcrest designing its own silicon, but we are seeing that the ARM and the MCU suppliers like STMicroelectronics or Silicon Labs or Microchip, as well as the silicon IP providers, like Synopsys and Kadin, are really focusing on designing in specific functions into the hardware to enable sensor processing to enable artificial intelligence, to enable energy harvesting… they’re really focused on the application sets.

“I don’t see as much a need for us to design custom chips because the chip suppliers are doing that work and they’re focused on the application as well, but I definitely do see special purpose solutions.

“We started to go down that path with the BNO 080 and FSM 300, but we added in application-specific functions into the device, and we’re still somewhat treating it as a general purpose.

“And if you’re a robot company, you can buy the BNO 080 or FSM 300 and you can use the robot navigation features.

“If you’re a VR device manufacturer, you can also buy the same hardware, but you will use the features that are specific to VR controllers and VR headsets.

“Over time, it’s likely that those will split up and we’ll have very tailored, specific products for each of those market places.

“And that’s just a trade-off of cost and performance.”

The beginning of an explanation

chip wallpaper
Illustration of a chip. Courtesy of Best Collection of Wallpapers

But what is it that differentiates, say, an AI microprocessor from a standard microprocessor – after all, both of them are made of silicon transistors which can be either in the on or off state.

“A lot of it comes down to the math functions – and I can only talk at a super high level here,” says Lucien. “But the ability to compute certain types of algorithms is what’s driving a lot of the changes.

“So being able to have an instruction set that allow you to compute complex algorithms more quickly, that enable you to send the data out at a very fast rate…

“A lot of applications, for example, require you to sample sensors at 100 hertz.

“But in the VR world, the expectation is that when you’re sampling the sensors, and outputting the sensors fusion, for tracking your head movements, you’re outputting at a kilohertz.

“So, being able to hold the samples in, and process them and then output them, at that rate, requires you to have math functions that can scale, that can enable you to compute faster.

“And in some cases, they also require you to perform the math function without consuming a lot of RAM because while flash memory is cheap, the chips have limited RAM.

“So a lot of the design is to manage those things. So, it’s managing realtime data flow, managing complex calculations, figuring out how to make sure that the power consumption is super low – that’s how I understand it.

“And that’s why we’re seeing now more signal processing instruction sets going into these microprocessors than you’ve been seeing in the past.”

So there is a reason behind it, a logic, and it’s not just hype?

“Yes, there’s good reasons… Some of the math functions that we might compute in software today can ultimately be designed into hardware itself, so they’re going to be fixed in the ROM of the hardware, which ultimately makes the product cheaper.”

Hillcrest’s chips contain their own firmware. “In our particular case, our software is the firmware. Our software is running in the flash of the chip and I guess the RAM as well. We’re not at the level where we’re in the ROM of the device – it’s not baked into the ROM, it’s programmed into the flash. And then obviously we use the RAM when we’re doing the compute.”

From Lucky Goldstar to apparent obscurity

hillcrest Scoop-Pointer
Hillcrest’s pointer for presentations

For Hillcrest, Asia has proved a good market. Millions of its chips are in all sorts of devices in the region, particularly inside television sets – the smart, new ones.

“Our entry point to the sensor processing space was in the motion control, or motion remote control, business,” recaps Lucien.

“We’ve had quite a bit of success over the past 10 years, licensing our software to the TV market. Many smart TVs – not as many in the US and Europe as there are in Asia, but – many TVs use our motion technology to enable remote control.

“So you can basically use your remote control as a 3D mouse. Rather than navigating the screen of your smart TV with the arrow keys and the buttons, you have a cursor on the screen and you simply move your remote around in very small, comfortable motions and move and control the cursor around the screen just like you would use your mouse to control the arrow on your computer screen.

“If you’ve ever seen the LG smart TVs, they have something called the magic remote, which is enabled by Hillcrest technology.

“So that really formed the basis, and since that time, we’ve licensed software into the smartphone market, into the wearables market, into VR and AR. We’re mainly selling components today, more recently in robotics.

“We’ve had some good success selling our inertial system, like the BNO 080 and its predecessor into the robot vacuum cleaner market, which for an autonomous robot is actually one of the higher volume markets out there.

“The autonomous robot vacuum cleaners are selling in the single-digit millions of units today globally and growing pretty quickly, so that’s been an exciting market for us.

“But it’s also led us down the path of working with companies who are building robotic lawnmowers, home assistance that kind of follow you around the house, and companies that need IMUs [inertial measurement units] for auto steering of equipment like agricultural vehicles – farm equipment.

“Now we’re started to see things that I would’ve previously said were more obscure, but finding out they’re less obscure. So, things like antenna positioning – companies that have antennas mounted on buildings or in fields and they want to make sure they’re oriented properly and pointed in the right direction, and they want to know if they’ve shifted off of their orientation because of a wind gust or something ran into it, sensors are necessary to really help to stabilise antennas.

“There are also applications in asset tracking – being able to use the sensors in combination with other technologies to create a geo-fence so you can make sure equipment stays within a pre-defined zone.

“So those are the areas – in broad brush terms – that we are focused on expanding, and we’re looking at the FSM 300 as a way for us to service a much broader set of these new markets and then fine-tune our solutions for larger ones.”

Field of sensors

iphone sensors
Illustration of some iPhone sensors, courtesy of iNotes4You.com

The availability of cheaper, more sophisticated sensors is what has driven the new mobile computing revolution, and is likely to be the basis of an entirely new technological world, where new companies produce new devices that take everyone in new directions.

But how did this happen?

Lucien offers some background. “The sensors that we work mostly with are MEMS-based inertial sensors, like accelerometers, gyroscopes, magnetometers, as well as environmental sensors – so, sensors that are detecting changes in pressure, and are detecting ambient light, proximity, temperature, humidity and those types of things.

“I think the real key, top-level business driver for why you’re seeing more use of these sensors and better quality is the smartphone market. The smartphone market has been an enormous contributor to the advancement of these technologies.

“Really, before the smartphone market came into existence, these kind of sensors – gyroscopes, for example – were used in aircraft guidance systems and lots of industrial and military type of installations.

“But they ranged from the hundreds of dollars a unit to thousands of dollars a unit for a particular sensor. Now those sensors are more capable, they’re higher quality, have better precision, they don’t have as many negative anomalies, and so on.

“But the new sensors that are going into the consumer grade products have gotten significantly better over the past 10 years, and it’s really only been in the past 10 years that we’ve seen consumer-grade gyroscopes, for example, at a price point that could go into a typical consumer product.

“When we started developing our motion technology, a gyroscope cost maybe $6 per axis. Today, you can buy a three-axis gyroscope in a single package for less than a dollar.

“That is largely driven by the fact that Samsung and Apple and the smartphone industry have been putting these technologies into their phones for many years and have driven down the costs.

“The other early driver was the Nintendo Wii. The Nintendo Wii implemented an accelerometer and then subsequently a gyroscope, and then shifts 100 million units, which I think really helped to kick off the marketplace.”