The autonomous future of industrial vehicles: Interview with John Deere business manager Jesse Haecker

John Deere is, by most accounts, the world’s largest manufacturer of agricultural machinery – tractors, combine harvesters and so on. According to figures published by Statista, the 185-year-old American giant generated revenues of almost $32 billion in 2020. 

The next three companies on the list are, in order, Kubota ($15 billion), CNH Industrial ($11 billion), and AGCO ($9 billion). So Deere is clearly in the lead now, but one of those three companies – CNH Industrial – may be in a good position to grow its market share going forward because it launched a quite striking looking autonomous tractor a couple of years ago.

While CNH’s machine may look too futuristic for most farms, possibly leading people to believe it won’t take off any time soon, John Deere has nonetheless responded to the whole autonomous trend by launching its own driverless tractor (pictured below), which was showcased at the recent CES event.

In this interview, Robotics and Automation News speaks to Jesse Haecker, business manager for planting, spraying and nutrient application at John Deere.

Haecker did not give away any details about the autonomous tractor the company launched just a few months later, but it was becoming quite obvious – especially after the $305 million acquisition of Blue River – that John Deere was looking to make substantial moves into the burgeoning autonomous market, and perhaps further expand the technology into other markets, such as construction.

So here it is, the interview with Jesse Haecker. Questions were asked by Abdul Montaqim. Interview took place a few months ago.

Abdul Montaqim: What’s the nature of your job at John Deere?

Jesse Haecker: I’ve been with Deere for 23 years. My current position is business manager for the spraying and planting jobs globally.

What that entails is that, essentially, my team is responsible for the future portfolio planning and introduction of new products to the market and then also responsible for the business results of those those products.

AM: I suppose your job has become more busy or your department has become more busy as machinery has become more computerized. Can you give us a sort of potted history of how things have gone from being mechanical to increasingly computerized in the last few years?

JH: Yeah, so my group is a little bit new. John Deere went through a reorganization within the last six months to put even more focus on technology. So my team is largely new.

But when you think about the evolution of technology on equipment, this has been a journey that John Deere’s been on for over 20 years. The idea of using GPS to to guide vehicles down a path, to get equipment down a path, has been around for 15 years.

We’ve had different manifestations of robotics for the past 15 to 20 years. So the use of technology and computer algorithms and those sorts of things on equipment is not new.

But what Deere has done in the last three to five years is accelerate this, and it continues to accelerate more and more every year.

Abdul Montaqim: What do you attribute the acceleration to? Is it because AI and computing infrastructure has become more available, smaller and cheaper, and more capable of running AI systems? And what kind of things does AI do?

JH: I’d say in terms of drivers there are two things: one is macro trends, and the other is the advancement of technology in general.

So when you think about macro trends, we all know that the global population by 2050 will be around 10 billion, which is a huge increase over the current 7 billion we’re at right now.

But if you drive that down to food, we need to produce 50 percent more food to feed that population as diets change and we need to be able to make the farmer 60 more productive overall than they are today.

Okay, so that’s one of the macro drivers.

Another one of the macro drivers is the reduction of labor. Fewer people are on farms, which means individual farmers need to do more with a lot less.

Then the last thing is increased environmental pressure. Environmental stewardship, sustainability and so on.

These macro trends are driving companies like John Deere to do things differently, to do things better, to do them more productively.

And on the flip side, technology has evolved to be able to unlock that potential, whether it’s advancement in robotics, advancement in sensors, artificial intelligence combined with machine learning and so on.

Those would be the things that I would say is causing this acceleration.

There’s a lot of examples I can give you, but I’ll point to two that are both recent product releases Deere has gone through.

One was last summer. We introduced a new combine (harvester) to the market that harvests grain out of the field. That combine, at the macro level, is 10 to 20 percent more productive than its predecessor within the same frame. So the same envelope, the same relative size, but 10 to 20 more productive.

And as you zoom in on that piece of equipment, as that machine goes through the field, there’s approximately 180,000 kernels of grain of corn, for example, going through that machine.

We leverage camera technology, AI and machine learning, in this system. There’s 180 kernels per second going through that machine and we use AI computer vision machine learning to analyze that grain as it comes by and make real time adjustments on the combine to ensure that we’re getting clean grain samples and that we’re not damaging that grain.

Another example that we just announced within the last month is around our see-and-spray technology where we use 36 cameras across 120 foot boom and those cameras can recognize weeds that are less than one inch tall and and activate one of 96 robotic nozzles to spray just that weed versus spray across the whole machine.

AM: What do you think of the broader trend in the market, the agricultural sector, in terms of the development of AI systems, IoT systems and the vehicles themselves becoming more autonomous and intelligent? Could startups or established companies threaten Deere’s dominant position using these technologies?

JH: I think the trend in agriculture is towards such technologies – and it’s not just John Deere. We are not the only company that has recognized this trend.

But John Deere invests more in research and development than anybody in the industry, especially in this space, because we are committed to it.

We believe that this is the most important way to help reach those goals by 2050 – of the amount of food production that needs to be increased.

There’s lots of folks that are interested in the agricultural space and and we’re committed to advancing this technology in agriculture.

The agriculture industry has always been competitive, whether it was 20 years ago, 50 years ago or a hundred years ago, right across the globe – the ag industry has always been competitive.

The difference today is that the competitive dynamics have changed from more of an iron product to a highly integrated, highly technical product with lots of different technologies.

So, on one hand, our traditional competitors are recognizing the same opportunities to help grow more food and to help reduce the task burden on customers that we are. On the other hand, you’ve got startups that are focused on different technology niches.

From our standpoint, we’re focused on the vision and the mission of feeding 10 billion people by 2050, and helping farmers become more profitable, more sustainable, and reducing their task burden.

And we’re placing bets in a lot of places. You probably are aware that, in the 2015-17 time frame, we acquired Blue River Technology out of California, which was a computer vision machine learning startup. It was focused on the core technology of see-and-spray.

At the Agri-Technica event, we featured an electrified tractor. We featured some other autonomous concepts that are quite a bit different than the equipment we have today.

So we’re focused on moving the industry forward, given our position in the industry, and we’re investing in a lot of unique technologies and a lot of unique areas to get there.

AM: Software is your area of expertise. What kind of challenges or do you face? What’s more difficult – hardware or software? Or do they both link together in a way that’s inseparable?

JH: They both offer unique challenges, to be honest, and the applications of them are different.

I’ll give you a couple of different examples. One of the main challenges in ag around hardware and sensor technology is the harshness of the environments that we operate in – you’re not in an enclosed building, like in a factory, you’re not in a semi-controlled environment.

For years, if you dropped your cell phone into the lake, you put it in a bag of rice and hope it comes back to life. That’s a table stakes environment – rain on sensors, dust, dirt abrasion, impact. These are real environments that we develop in, so that that does pose challenges in ag on the hardware side. We have to ruggedize the hardware and make it live through the harsh environments. So you got those challenges.

When you get to the software side of it, there’s a lot of diversity in ag. So let’s think about a machine learning algorithm.

This is real interesting because our brains can look at a weed or a piece of grass or something that’s out there and, regardless of where the sun’s at, we recognize it’s a weed.

But if you look at it into the sun and look at it away from the sun, the color changes drastically, our brains think it’s green. It’s because we’re using other pieces of our mental model to do that.

So when you think about the software side of it, and the sheer diversity of things that we’re trying to see and recognize, it’s not a controlled factory environment where we’re looking for a weld that doesn’t look perfect. It’s literally a very dynamic environment.

So the machine learning models that we train and the things that we do from a software standpoint become very complex very fast.

So I would say both of them pose their challenges.

AM: What do you think of IoT systems for farming? So, for example, sensors all over the field spotting weeds and other problems, and using drones for things like spraying weeds and maybe even planting seeds. Do they compete with vehicle- or tractor-based systems for farming, or do they work together?

JH: I think that’s a great question around technology capability versus scalability.

So one question is, ‘Is the technology – of putting stationary infield sensors in and taking a different approach – capable of actually doing the work that our customers do?’

Yeah, I would say the technology is probably capable.

But when you get to the scalability question, I mean, just to put this into perspective, I’m going to use an American football field analogy – I apologize for not using European football, that’s my bad – but if you think about an American football field.

We would apply nutrients and pesticides to a field. Today, we can cover a football field in 20 seconds.

So when you think about the scalability, we can we can put seeds in the ground on that same football field in 60 seconds – 50,000 to 200,000 seeds in 60 seconds.

So, if you think about the idea of putting 50,000 to 200,000 seeds in that size of area in 60 seconds, this idea of drones and stationary sensors becomes a challenge. It becomes a significant challenge

John Deere has been around since 1837, and we’ve kind of learned how to enable customers to get over as much farm ground as possible within the short weather windows, do multiple tasks at the same time, and do so super accurately, super sustainably, and super efficiently, so that’s one of the things we consider.

Every second that a farmer makes a pass through the field you know it’s not just turning the steering wheel. There’s dozens of things going on around them.

We like to say farmers are the the ultimate multi-taskers and the ultimate micro-managers.

When I mention that you’ve got 46,000 seeds to 200,000 seeds going into the ground every minute, the farmers’ goal is to micro-manage every one of those seeds to get the most potential.

And at the same time, they’ve got lots of things going on around them. So the idea of leveraging robotics and all these technologies is super critical for those farmers to enable them to do a better job.

A typical farm, for a US customer, will have 750 million plants on it.

AM: Has John Deere developed any drones for farming?

JH: No, we haven’t developed any drones.

But to build on one of your earlier comments, it really boils down to sensor fusion. There’s things that we can sense on vehicles as you go through a few fields. Then there’s potentially other inputs from other sensors or other pieces of information from elsewhere.

At John Deere, we have a cloud-based application called Operations Center, which is our digital tool for customers to monitor their operation and to collect data about the operation, and analyze that data.

Operations Center is a tool that’s open to other providers through APIs. It’s able to fuse other pieces of data into that system.

The platform is connected through modems onto our equipment and we can seamlessly exchange information from that vehicle up to that cloud platform and fuse the data coming off of machines with data that would come from from other places in the ecosystem.

AM: Does John Deere have any fruit-picking machines? Or is it developing any, perhaps using similar technical principles to combine harvesters? I’ve read about some startups developing fruit-picking machines or robots – they look very sophisticated. 

JH: We don’t have fruit-harvesting equipment in our portfolio. The closest thing to that would be cotton harvesters that pick cotton off of plants, which is a very different robotic challenge than stripping grain from a plant.

You can picture it going up to a cotton tree and plucking that cotton off that tree – this is more of a mechanical means than maybe than a complete robotic means. But we’ve been able to mechanically grab cotton individual cotton off of plants since the late 1940s with technology.

So that would be the the closest thing that we’ve got to what you’re describing in terms of kind of the robotic action or the arms picking fruit off of trees or those sorts of things.

AM: John Deere sells a lot of vehicles, but 99 percent of your vehicles are driven by humans. I imagine that, in the future, all those vehicles will become autonomous because the technology doesn’t seem so difficult for a company like yours to acquire, develop or implement. 

JH: I would go back to my comments earlier. If driving was the only thing the operator did, we’re basically there.

We’ve got technology today: auto track GPS guidance that can guide you down a path, plus or minus a half inch, that we’ve been able to do for 10 years; we’ve got technology that automatically turns the vehicle on the end of the field to go back the other direction; and we’ve got technology that now will – after you make the first pass through the field – it’ll automatically create the path for every other operation that goes through the field.

So you know you can literally pull into a field and hit a button and we’ve got enough technology to drive the vehicle.

The challenge then becomes all the other things that happen when you’re driving through the field. Even before guidance can drive the steering wheel and drive the machine, customers spend more time looking behind the them and their vehicle at all the things that are going on – to optimize the job of putting seeds in the ground, spraying weeds, getting crop out of the field, and so on.

So the journey in ag is less around thinking of autonomous vehicles like cars – we’re really close to being able to do that today. In fact, very capable of doing that. The challenge is the six to 12-plus other things that are going behind a typical farm vehicle.

In agriculture, the next years will be focused on reducing the task burden of the operator through automation of tasks that leverage robotics technology and AI and machine learning and sensors and cameras and so on.

We will be combining all those things to reduce the number of tasks that the operator does in addition to the driving part of it.

The other thing to recognize is that your typical farmer is the CEO of his or her company: they’re managing logistics – inputs coming to the field and things going off the field; they’re marketing grain and selling grain; they’re human resources managers; and so on. They’re doing all these things.

So today, with the automation and technology we have brought to ag equipment, that cab of that tractor combine becomes their mobile office.

So they’re literally on their phone selling grain, they’re checking weather, they’re doing all those things as they’re going through the field.

And what automation and robotics have been able to do for that customer is to allow them to make that cab their office, in addition to managing the other things.

So that was a long way to not directly answer your question, but I think the reality is that the runway in ag is a tremendous amount of automation and reducing the task the customer has to do.

In addition, that cab has become their mobile office, where they conduct the business of running their entire company. So self-driving is essentially, like, we’re basically there.

The focus is on all the rest of the things that the operator needs to do, the customer needs to do.

With all the connectivity that we put in the cab they’ve literally chosen to make that their mobile office to run their business.