In order to understand the patent portfolio of TuSimple, we have divided its patents into 10 clusters shown below. This division has been very extensive and covers almost every aspect of the research that the company has been doing.
As evident from the graph, we can see that the majority of its patent families are focused on the autonomy of the level 4 ADAS trucks.
With approximately 50% of its patents in autonomous innovation, the company is building its monopoly in the self-driving trucks market.
TuSimple Patent Portfolio- Autonomous Driving (95 patent families)
The autonomous vehicles cluster as expected covers the majority part of the patent portfolio.
TuSimple is using multiple cameras with other sensors for developing a commercial-ready Level 4 (SAE) fully autonomous driving solution for the logistics industry.
The company is covering all the aspects of the fully autonomous truck systems ranging from obstacle detection to efficient fuel-saving methods.
The below exhibit shows the categorization of the patent families according to the functions of autonomous vehicles.
SLAM (29 Patent Families)
Most of the patents are focused on the usage of cameras for labeling objects and lanes and constructing maps for effective SLAM. Some patents include the synergy of the image data and the sensor data for vehicle positioning along with the awareness of other parameters on the road.
Also disclosed are some patents which achieve vehicle tracking information by the combination of information from a roadside assistance system via a V2X communication method.
Cruise Control (16 patent families)
The patent families in cruise control mention methods that help the vehicle in:
- Proximate vehicle detection
- Taillight state recognition
- Speed control mechanisms
- Emergency braking methods
The patents also disclose the ways of monitoring the surrounding environment for traffic analysis and maintaining a motion of the truck connected to its trailer.
Patents on switching driving modes based on traffic conditions, and for calculating the angle between the trailer and the tow vehicle are also in the cluster.
Lane Detection (9 Patent Families)
Using visual information to detect the lanes for ADAS invites several problems:
- The picture quality may differ as per different weather conditions
- Repeated camera shaking may hinder the detection
The patents in this cluster address these issues. It includes the fusion of sensor data for performing the localization of roads to produce lane markings and hence providing better perception methods.
Also, the patent discloses the methods of image processing for detecting the road edges and ensuring supreme safety while driving.
Since TuSimple uses a plurality of cameras for detecting the various road situations, it also has disclosed some methods for the automatic calibrations of cameras for the determination of potholes, lanes, etc on the road.
Vehicle maintenance and security (9 Patent Families)
The cluster contains patents on optimized route plans for efficient fuel consumption, precise data collection & maintenance for sensors, and handling emergency situations via automatically fixing failures.
Parking (2 Patent Families)
Trucks in ports need to be parked accurately for efficient port operations. The patents in this cluster disclose the parking server and parking methods that help in autonomous position-based parking by monitoring the braking speed and time of the vehicle.
TuSimple Patent Portfolio- Camera Technology (49 patent families)
Like human drivers, autonomous trucks’ perception systems are challenged by a wide variety of light conditions. TuSimple is working on cutting-edge camera technology that sees better than human eyes i.e. for better perception in any condition- night or day, rain or shine.
TuSimple’s patent portfolio contains 49 patent families on camera innovation for better perception through trained machine learning algorithms.
The patents address the complex imaging challenges such as instant light changes when entering and exiting tunnels, flaring during sunrise and sunset, and headlight glare.
They also revolve around the pixel-level interpretation of the visible environment through the image acquisition, transmission, and processing method. This ensures improved vehicular driving and the safety of truck fleets even at night.
Also, some patents disclose the positioning, calibration, and synchronization of the cameras that are used for a better perception of the environment.
Since trucks need to perceive the condition of the road up to a long distance in order to make better driving decisions TuSimple uses a plurality of cameras mounted on its vehicle that are able to sense the environment as long as 1000 meters.
Moreover, there are several patents in this cluster that aim at improving camera resolution, the enhancement of the exposure parameters of the image taken, and correcting the abnormality of the camera system.
TuSimple Patent Portfolio- Autonomous Freight Operation (45 patent families)
TuSimple entered into the trucking industry for safe and efficient freight operations through its Autonomous Freight Network (AFN). AFN involves 4 parts:
- Self driving trucks
- Digitally mapped routes
- Freight terminals
- And a system that will help customers monitor autonomous trucking operations and track their shipments in real time.
The Freight Operations cluster is divided into the below 5 subclusters:
Port infrastructure (20 Patent Families)
Conventionally at a port, the container transportation operations such as loading and unloading require human intervention.
- Ships need to be steered and controlled by humans to berth at specified locations.
- Hoisting apparatus and transportation vehicles need to be driven operators to perform operations such as container loading, container unloading.
- Yards or warehouse centers also need human intervention to arrange locations at which containers are to be placed.
The present patent cluster aims at solving these issues that require a lot of human resources.
TuSimple in its patent portfolio focuses on the complete automation of the port through a central control system. The central control system helps in the automatic management and control of the harbor, warehouse, shipment area, checkpoint control, container hanging devices, etc.
Also, patent families are covering automatic parking systems, vehicle positioning systems, etc with the help of cameras located in the port for V2X communication and generating an electronic map (which contains the entire whereabouts of the port) for making efficient and safe travel decisions.
Autopilot (8 Patent Families)
The goods delivery trucks demand a driver to indulge in a lot of activities like loading and unloading containers, paying custom duty, and the like.
In this cluster the patents target the automatic supply of goods, carrying out the custom duty checks, automatic loading and unloading of the goods via the autonomous vehicle, thereby reducing human cost and improving the efficiency of work in the freight operations.
Platooning (7 Patent Families)
The patent cluster discloses inventions related to connected trucks. It covers vehicle-to-vehicle communication in a connected truck fleet to detect traffic and other situations on the road.
The communication among the vehicles in the fleet happens for maintaining a synchronized motion of the fleet, where cars follow each other based on the driving decision of the leader.
Freight ops Management (6 Patent Families)
Patents that incorporate the automatic fueling methods, vehicle tracking and controlling, and vehicle scheduling methods are included in this cluster.
TuSimple Patent Portfolio- Cloud Computing (14 patent families)
With the continuous progress in autonomous technology, robot sensors are generating increasing amounts of real-world data. Autonomous vehicle research is highly dependent on the vast quantities of real-world data for the development, testing, and validation of algorithms before deployment on public roads.
However, the cost and effort for processing and analyzing these data, including developing and maintaining a suitable autonomous vehicle platform, regular calibration, and data collection procedures, and storing the collected data, is very high.
The present cluster consists of 14 patent families that are focused on the collection, storage, and management of gigantic amounts of data in a distributed environment. They aim at maintaining a high-speed transfer of data (high throughput) to the computing terminal used for the training of the DNN.
The patents involve the security, efficiency, and ease of use of the data produced by different sensors for simulation-related purposes.
There are some patents that include the effective resource utilization for GPU computing, that can enable verification tests to be completed in a lesser time.
Running these tests in-house would take weeks each time new code is rolled out, but thanks to the massive computing power of AWS GPU-powered compute in Amazon EC2 P3 instances, the tests can be completed in a matter of hours, minimizing the turnaround time before a truck can get back on the road.
~Dr. Xiaodi Hou, Co-Founder, TuSimple
TuSimple Patent Portfolio- Artificial Intelligence (17 patent families):
The artificial intelligence system designed by TuSimple helps in taking accurate decisions. It includes guiding the vehicle for taking the most fuel-saving route. All the decisions are real-time in accordance with maps and the current condition of terrain with real-time road conditions.
17 patent families in the artificial intelligence cluster are on problems of computer vision for large model parameters, accuracy and speed of the neural network, and the decision making of the autonomous vehicles based on real-time scenarios like object detection, lane detection, and the like.
The cluster includes ways to optimize the neural networks by several machine learning methods for limiting the model parameters and still maintaining the accuracy and speed of the neural network.
For example, US20190279089A1 relates to a method of network pruning in which the network parameters are reduced while maintaining the accuracy of the existing network. This can thus allow low latency and hence better and efficient real-time applications of the self-driving vehicle.
Another patent US20180365564A1 discloses a training method where the knowledge distillation method is used for training a neural network. The patent broadly focuses on the reduction of model parameters along with the achieving of high-speed computation.
TuSimple Patent Portfolio- Vehicle parts (16 patent families):
16 patent families on vehicle parts in TuSimple’s patent portfolio covers the extensions and systems to be deployed and mounted on autonomous vehicles.
TuSimple has partnered with the automotive supplier ZF to mass-produce vehicle technology such as sensors on a commercial scale.
The patents in this cluster work towards the creation or improvement of cooling systems, sensor adjustment systems, power supply control systems, etc.
TuSimple Patent Portfolio- Image Processing (06 patent families)
TuSimple is an autonomous driving company working towards the hauling of real commercial cargo through its advanced perception algorithms. However, there are several problems associated with the processing of the images which include low efficiency in the labeling of different instances associated with autonomous driving.
The patents in this subcluster talk about the efficient labeling and processing of images from a camera and the data received from the LIDAR.
The cluster discloses the deep learning algorithm and tools for improving the performance of the autonomous vehicle in obstacle detection, lane detection, cruise control, trajectory planning. Along with this they also mention the effective localization and mapping methods based on the data from the image captured with the help of sensors assigned to the vehicle.
Also in some instances, the patents involve the automatic calibration and improved focusing methods of the camera for better perception and efficient driving decisions in real-time.
TuSimple Patent Portfolio- Design Patents (02 patent families)
The cluster consists of the below three design patents:
- USD876525S1: Discloses the design for an automatic clearing system for the sensor.
- USD896862S1: Discloses the design of a camera to be used in the truck for autonomous driving.
- USD903598S1: Discloses the design for an aviation plug.