Autonomous cars have been arriving in bits and pieces for years. Bristling with sensors and microprocessors, cars have long been lightening the driver’s workload and improving safety

October 25, 2013

On the Road to Autonomous, a Pause at Extrasensory

By PAUL STENQUIST

The self-driving car, whenever it arrives, is sure to be hailed as a breakthrough. The truth is, autonomous cars have been arriving in bits and pieces for years. Bristling with sensors and microprocessors, cars — especially at the top end of the price spectrum — have long been lightening the driver’s workload and improving safety with features like parking assist, lane-departure warning, adaptive cruise control and various crash-avoidance technologies.More bits of autonomous driving technology are arriving in the new model year.

“We’ve given the vehicles senses,” Ralf G. Herrtwich, director for driver assistance in the advanced engineering group of Daimler, the parent company of Mercedes-Benz, says in a video presentation for reporters. “They can see and do so much more.”

The car Dr. Herrtwich speaks about is the 2014 Mercedes S-Class, which arrived at United States dealerships this month. The car makes a significant step toward the ability to drive autonomously, using many types of sensors to gather data about driving conditions, including a stereo camera that lets it “see” by providing 3-D images, akin to human vision.

The new S-Class — an S550 and S63 AMG are first to arrive in the United States — offers a package of technologies, some of which are extra-cost options, that Mercedes calls Intelligent Drive. Among its many components are systems that can steer, brake, cover blind spots and watch for pedestrians and animals. One function monitors the road ahead to ensure that passengers are not jolted by potholes.

Dr. Herrtwich points out that the car’s computing power, in the form of processors and algorithms, is highly developed.

“Our Intelligent Drive Systems these days are capable of analyzing complex traffic situations, and they have developed an understanding for what to do,” he says.

It is a network of sensors that makes the analysis possible. Most perform multiple functions, generating data for a number of driver-assistance systems.

Included are numerous cameras, mounted front, rear and side. Among them are two forward-facing infrared cameras for night vision, and a stereo camera — actually two cameras in a single housing — that provides 3-D imaging crucial to the system’s decision-making powers.

The 3-D camera is a critical element in a driving aid for the S550 called Magic Body Control, which expands on existing Mercedes suspension technology. The two lenses of the stereo camera each record seven images per second of the road ahead. The images are combined to generate a three-dimensional view of the pavement. If a bump or hole is seen, a microprocessor adjusts the car’s suspension to better manage the rough spot.

The stereo camera’s images are key to this ability, as a 3-D view provides perspective. In humans, the brain processes the images our eyes provide and compares them with our databank of experience, enabling us to gauge distance. The S550’s microprocessors and algorithms analyze the images from the stereo camera to determine whether it’s seeing a roadway bump that might cause discomfort and gauge how distant it is from the car. That information, in combination with vehicle speed data, enables the microprocessor to determine when to adjust damping and activate hydraulic pistons that act on the suspension’s springs.

Other data-gathering devices include two short-range radar units that monitor an area near the rear fenders. Multimode radar keeps tabs on the road from directly behind the car to 260 feet distant. Ultrasonic sensors detect objects alongside the car.

Front-facing radar keeps track of a wide area to within 200 feet of the car, and forward-facing cameras, in combination with multirange radar, provide data needed by the Distronic Plus adaptive cruise control and steering assist. Here, cameras watch the edge markings of the lane and the car ahead. Radar locks onto the car directly ahead and a second car beyond that.

To reach that second car, the radar signal is directed under the car directly ahead. The microprocessor regulates electronic steering to keep the car centered in the lane and manages the speed as required by traffic.

Steering-assist technology is capable of guiding the car through slight bends and gentle curves without driver input. But the system will not override the driver, and it will issue a reprimand on the dashboard should the wheel be released for more than 10 seconds.

Lane-keeping and blind-spot assist are also meant to help keep the car in the correct lane. Should, for example, the driver start to change lanes when another vehicle is overtaking, the steering wheel will pulse in warning and the brakes will be applied on one side of the car to move it back toward the center of its lane.

Cameras and radar work together to provide brake-assist functions, including collision prevention assist, at speeds above 4.5 m.p.h. By monitoring the vehicle ahead and the second vehicle ahead, the system calculates when there is a danger of collision. Visual and audible warnings alert the driver.

If subsequent braking is inadequate, the system increases pressure. If the brakes are not applied, the system will do so. The car can be stopped in time to avoid a collision at speeds below 31 m.p.h., and the effects of a collision are mitigated at speeds below 45 m.p.h.

The system can also detect cross traffic at an intersection and apply the brakes to reduce the risk of a side-impact collision.

If the radar unit in the rear bumper detects a vehicle closing fast and determines that a rear-end collision is imminent, the brakes are applied to help prevent a secondary crash into the car ahead, and the seat belt pretensioners are activated.

The S550 can detect pedestrians in front of the car by analyzing data from the stereo camera and radar. If a pedestrian is in danger, the driver is warned, and brake assist is provided as necessary. If the driver fails to brake, the computer takes over and brings the vehicle to a halt. The same speed thresholds for crash prevention and mitigation apply.

The system distinguishes between a pedestrian and, for example, a stop sign or a tree bending in the wind, using complex algorithms to analyze data from the sensors, Mercedes says.

Stuart Klapper, a managing director at Autoliv, which provides radar and camera components to automakers, said in a telephone interview that his company spent $50 million developing algorithms for pedestrian recognition.

“You have to have a lot of confidence that it’s truly a pedestrian,” he added.

While Autoliv did not provide the stereo camera for the 2014 S-Class, Mr. Klapper said it would develop hardware for the car’s next generation. Autoliv produces the radar equipment and the infrared cameras for night vision.

According to Mr. Klapper, the S550’s night-vision system is unique. Rather than use a single infrared camera, as most systems do, Mercedes employs a far-infrared camera and a near-infrared camera. The far-infrared camera sees radiation far removed from the visible spectrum and detects small temperature differences on and near the road ahead, such as those caused by the presence of an animal or a person.

The near-infrared camera operates close to the spectral band that regular cameras use and renders a clear image of the detected hot spot. Because it’s very sensitive to temperature difference, the far-infrared camera can detect people and animals that a near-infrared camera might miss.

Once the computer decides that a living thing of considerable size is in or near the road, it issues visual and audible warnings. The heads-up speedometer image on the windshield changes to an infrared picture of the animal or pedestrian.

If the microprocessor’s analysis of the infrared image determines that a pedestrian rather than an animal is in the road, the control microprocessor communicates with the vehicle’s adaptive headlight system and aims a spotlight beam at the person, to better inform the driver and to warn the pedestrian. The spotlight alert is not used with animals because some animals react unpredictably to light.