Patent · US12299083B1 · B1 · US

Systems and methods of sensor data fusion

(11) Publication number

US12299083B1

(21) Application number

19/016,267

(22) Filing date

2025-01-10

(30) Priority date

2024-12-19

(43) Publication date

2025-05-13

(45) Date of grant

2025-05-13

(51) IPC

G06F 18/25; G06N 5/046

(52) CPC

G06F Electric digital data processing: 18/256, 16/24, 16/245, 16/2455, 16/24556, 16/2456, 16/33, 16/334, 16/43, 16/53, 16/903, 16/90335, 16/9035, 17/18, 18/213, 18/217, 18/2431, 18/25, 18/251, 18/253, 7/14, 7/16
B25J Manipulators; chambers provided with manipulation devices: 9/163, 9/1664, 9/1694
B60R Vehicles, vehicle fittings, or vehicle parts, not otherwise provided for: 19/483
B60T Vehicle brake control systems or parts thereof; brake control systems or parts thereof, in general; arrangement of braking elements on vehicles in general; portable devices for preventing unwanted movement of vehicles; vehicle modifications to facilitate cooling of brakes: 2201/00, 2201/03, 8/1755, 8/3275
B60W Conjoint control of vehicle sub-units of different type or different function; control systems specially adapted for hybrid vehicles; road vehicle drive control systems for purposes not related to the control of a particular sub-unit: 10/18, 10/184, 2050/0052, 2050/021, 2050/0215, 2050/022, 2420/00, 2420/40, 2420/403, 2420/408, 2420/50, 2510/069, 2510/18, 2520/04, 2540/12, 2554/801, 2554/802, 2556/35, 2710/18, 2754/30, 30/085, 30/09, 60/00, 60/0018, 60/00186
B62D Motor vehicles; trailers: 15/0285
G01C Measuring distances, levels or bearings; surveying; navigation; gyroscopic instruments; photogrammetry or videogrammetry: 21/165, 21/1652, 21/3804, 21/3811, 21/3848, 22/00, 3/00
G01S Radio direction-finding; radio navigation; determining distance or velocity by use of radio waves; locating or presence-detecting by use of the reflection or reradiation of radio waves; analogous arrangements using other waves: 11/00, 13/08, 13/103, 13/42, 13/86, 13/862, 13/865, 13/867, 13/881, 13/931, 15/08, 15/101, 15/42, 17/08, 17/86, 17/87, 17/88, 17/89, 17/894, 17/931, 19/45, 2013/93185, 5/14, 7/417, 7/4808
G05B Control or regulating systems in general; functional elements of such systems; monitoring or testing arrangements for such systems or elements: 13/0205
G05D Systems for controlling or regulating non-electric variables: 2101/15, 2111/50, 2111/67
G06N Computing arrangements based on specific computational models: 20/00, 3/02, 3/0464, 5/022, 5/04, 5/042, 5/045, 5/046, 5/048
G06T Image data processing or generation, in general: 2207/10028, 2207/20024, 2207/20084, 2207/30252, 2207/30264, 7/521
G06V Image or video recognition or understanding: 10/764, 10/80, 10/803, 10/82, 20/56, 20/58
G08B Signalling systems, e.g. personal calling systems; order telegraphs; alarm systems: 29/188
G08G Traffic control systems: 1/0133, 1/04, 1/042
H04L Transmission of digital information, e.g. telegraphic communication: 67/12
H04W Wireless communication networks: 4/38

(73) Assignee

DIGITAL GLOBAL SYSTEMS INC

(72) Inventors

MONTALVO ARMANDO

(54) Title

Systems and methods of sensor data fusion

(57) Abstract

Systems and methods of sensor data fusion including sensor data capture, curation, linking, fusion, inference, and validation. The systems and methods described herein reduce computational demand and processing time by curating data and calculating conditional entropy. The system is operable to fuse data from a plurality of sensor types. A computer processor optionally stores fused sensor data that the system validates above a mathematical threshold.

Full text

View on Google Patents

Claims (20)

A system for sensor data fusion for sensor management and utilization in robotics, comprising: at least one computer processor including a memory; at least one curation engine, at least one link engine, at least one fusion engine, at least one inference engine, and at least one validation engine; at least one first position sensor operable to capture a first position measurement of a robotic component; and at least one second position sensor operable to capture a second position measurement of the robotic component; wherein the at least one computer processor is operable to analyze the first position measurement and the second position measurement; wherein the at least one computer processor is operable to receive at least one query; wherein the at least one curation engine is operable to curate the first position measurement and the second position measurement, the at least one link engine is operable to link the first position measurement and the second position measurement, the at least one fusion engine is operable to fuse the first position measurement and the second position measurement, the at least one inference engine is operable to determine at least one inference from the first position measurement and the second position measurement, and the at least one validation engine is operable to validate the first position measurement and the second position measurement; wherein the at least one inference engine is operable to determine a second inference; wherein the at least one validation engine is operable to use artificial intelligence to compare the at least one inference to the second inference; wherein the at least one validation engine validates the at least one inference when the comparison between the at least one inference and the second inference exceeds a predefined threshold; and wherein the at least one computer processor is operable to instruct the robotic component to move based on the at least one inference being validated.
The system of claim 1, wherein the predefined threshold includes the at least one inference being within about 4.5% or less of the second inference.
The system of claim 1, wherein the system is operable to store the first position measurement and the second position measurement after the at least one validation engine validates the at least one inference.
The system of claim 1, wherein the at least one validation engine is operable to validate the at least one inference passively and/or actively.
The system of claim 4, wherein passive validation includes not modifying a movement of the robotic component.
The system of claim 4, wherein active validation includes modifying at least one movement of the robotic component.
The system of claim 1, wherein the at least one validation engine is operable to use artificial intelligence to dynamically adjust the predefined threshold based in part on types of data sources, environmental factors, and/or learning from previous analyses conducted by the at least one validation engine.
A method for sensor data fusion for sensor management and utilization in robotics, comprising: providing at least one computer processor including a memory; providing at least one curation engine, at least one link engine, at least one fusion engine, at least one inference engine, and at least one validation engine; at least one first position sensor capturing a first position measurement of a robotic component; at least one second position sensor capturing a second position measurement of the robotic component; analyzing by the at least one computer processor the first position measurement and the second position measurement; receiving by the at least one computer processor at least one query; curating by the at least one curation engine the first position measurement and the second position measurement, linking by the at least one link engine the first position measurement and the second position measurement, fusing by the at least one fusion engine the first position measurement and the second position measurement, determining at least one inference by the at least one inference engine from the first position measurement and the second position measurement, and validating by the at least one validation engine the first position measurement and the second position measurement; determining by the at least one validation engine a second inference; comparing by the at least one validation engine via artificial intelligence the at least one inference to the second inference; validating by the at least one validation engine the at least one inference when the comparison between the at least one inference and the second inference exceeds a predefined threshold; and instructing by the at least one computer processor the robotic component to move based on the at least one inference being validated.
The method of claim 8, wherein the predefined threshold includes the at least one inference being within about 4.5% or less of the second inference.
The method of claim 8, further comprising validating the at least one inference passively and/or actively.
The method of claim 10, wherein validating passively includes not modifying a movement of the robotic component.
The method of claim 10, wherein validating actively includes modifying a movement of the robotic component.
The method of claim 8, further comprising adjusting via the at least one validation engine using artificial intelligence the predefined threshold based in part on types of data sources, environmental factors, and/or learning from previous analyses conducted by the at least one validation engine.
The method of claim 8, further comprising storing the first position measurement and the second position measurement after the at least one validation engine validates the at least one inference.
A system for sensor data fusion for sensor management and utilization in robotics, comprising: at least one computer processor including a memory; at least one curation engine, at least one link engine, at least one fusion engine, at least one inference engine, and at least one validation engine; and at least two sensors, each of the at least two sensors operable to measure a first position of a robotic component and a second position of the robotic component; wherein the at least one computer processor is operable to analyze the first position and the second position; wherein the at least one computer processor is operable to receive at least one query; wherein the at least one curation engine is operable to curate the first position and the second position, the at least one link engine is operable to link the first position and the second position, the at least one fusion engine is operable to fuse the first position and the second position, the at least one inference engine is operable to determine at least one inference from the first position and the second position, and the at least one validation engine is operable to validate the first position and the second position; wherein the at least one inference engine is operable to determine a second inference; wherein the at least one validation engine is operable to use artificial intelligence to compare the at least one inference to the second inference; wherein the at least one validation engine validates the at least one inference when the comparison between the at least one inference and the second inference exceeds a predefined threshold; wherein the at least one validation engine is operable to use artificial intelligence to dynamically adjust the predefined threshold based in part on types of data sources, environmental factors, and/or learning from previous analyses conducted by the at least one validation engine; and wherein the at least one computer processor is operable to instruct the robotic component to move based on the at least one inference being validated.
The system of claim 15, wherein the predefined threshold includes the at least one inference being within about 4.5% or less of the second inference.
The system of claim 15, wherein the at least one validation engine is operable to validate the at least one inference passively and/or actively.
The system of claim 17, wherein passive validation includes not modifying a parameter.
The system of claim 17, wherein active validation includes modifying at least one parameter.
The system of claim 15, wherein the system is operable to store the first position and the second position after the at least one validation engine validates the at least one inference.

Description

1. Field of the Invention The present invention relates to physical sensors measuring data and dynamically fusing the physical sensor data to create a new dataset. 2. Description of the Prior Art It is generally known in the prior art to append data from more than one source. Prior art patent documents include the following: U.S. Patent Publication Number 2024/0378412 for methods for topological data analysis ai/ml pipeline (tdaml) with algorithm for multimodal sensor data fusion in autonomy applications, by inventor Paul Thomas Schrader, filed Apr. 25, 2024, published Nov. 14, 2024, directed to a method of topological data analysis feature engineering for data fusion and autonomy is provided. The method comprises providing a Topological Data Analysis AI/ML Pipeline (TDAML) algorithm for multimodal sensor data fusion in autonomy applications system further comprising combining raw heterogeneous multimodal sensor data at the topological level; measuring, recording, and tracking linear representations of an underlying data set; providing a linear representation of the underlying data set which is compatible to existing deep learning (DL) model architectures for training in autonomy tasks; and accessing the entire degree of freedom (DOF) space of raw multimodal sensor data for mitigating sensor modality adversarial threats and environmental attenuation concerns in contested military and civilian (urban) environments.

U.S. Patent Publication Number 2023/0112441 for data fusion and analysis engine for vehicle sensors, by inventors LuAn Tang et al., filed Oct. 6, 2022, published Apr.

Citations (39)

CN110598299A
CN116522279A
US10748075B2
US10766137B1
US11037320B1
US11586865B2
US2008312756A1
US2015306770A1
US2017004411A1
US2018284758A1
US2019295001A1
US2020026289A1
US2021150230A1
US2021342656A1
US2022012562A1
US2022075077A1
US2022126864A1
US2022146652A1
US2022161815A1
US2022179056A1
US2023110027A1
US2023112441A1
US2023166765A1
US2023342944A1
US2024045426A1
US2024112428A1
US2024142994A1
US2024152734A1
US2024249165A1
US2024289930A1
US2024289981A1
US2024296044A1
US2024312219A1
US2024317254A1
US2024324838A1
US2024331403A1
US2024378412A1
US2024408495A1
US9367683B2

Record as JSON

{
  "publication_number": "US12299083B1",
  "country": "US",
  "kind": "B1",
  "title": "Systems and methods of sensor data fusion",
  "abstract": "Systems and methods of sensor data fusion including sensor data capture, curation, linking, fusion, inference, and validation. The systems and methods described herein reduce computational demand and processing time by curating data and calculating conditional entropy. The system is operable to fuse data from a plurality of sensor types. A computer processor optionally stores fused sensor data that the system validates above a mathematical threshold.",
  "claims": [
    "1. A system for sensor data fusion for sensor management and utilization in robotics, comprising: at least one computer processor including a memory; at least one curation engine, at least one link engine, at least one fusion engine, at least one inference engine, and at least one validation engine; at least one first position sensor operable to capture a first position measurement of a robotic component; and at least one second position sensor operable to capture a second position measurement of the robotic component; wherein the at least one computer processor is operable to analyze the first position measurement and the second position measurement; wherein the at least one computer processor is operable to receive at least one query; wherein the at least one curation engine is operable to curate the first position measurement and the second position measurement, the at least one link engine is operable to link the first position measurement and the second position measurement, the at least one fusion engine is operable to fuse the first position measurement and the second position measurement, the at least one inference engine is operable to determine at least one inference from the first position measurement and the second position measurement, and the at least one validation engine is operable to validate the first position measurement and the second position measurement; wherein the at least one inference engine is operable to determine a second inference; wherein the at least one validation engine is operable to use artificial intelligence to compare the at least one inference to the second inference; wherein the at least one validation engine validates the at least one inference when the comparison between the at least one inference and the second inference exceeds a predefined threshold; and wherein the at least one computer processor is operable to instruct the robotic component to move based on the at least one inference being validated.",
    "2. The system of claim 1, wherein the predefined threshold includes the at least one inference being within about 4.5% or less of the second inference.",
    "3. The system of claim 1, wherein the system is operable to store the first position measurement and the second position measurement after the at least one validation engine validates the at least one inference.",
    "4. The system of claim 1, wherein the at least one validation engine is operable to validate the at least one inference passively and/or actively.",
    "5. The system of claim 4, wherein passive validation includes not modifying a movement of the robotic component.",
    "6. The system of claim 4, wherein active validation includes modifying at least one movement of the robotic component.",
    "7. The system of claim 1, wherein the at least one validation engine is operable to use artificial intelligence to dynamically adjust the predefined threshold based in part on types of data sources, environmental factors, and/or learning from previous analyses conducted by the at least one validation engine.",
    "8. A method for sensor data fusion for sensor management and utilization in robotics, comprising: providing at least one computer processor including a memory; providing at least one curation engine, at least one link engine, at least one fusion engine, at least one inference engine, and at least one validation engine; at least one first position sensor capturing a first position measurement of a robotic component; at least one second position sensor capturing a second position measurement of the robotic component; analyzing by the at least one computer processor the first position measurement and the second position measurement; receiving by the at least one computer processor at least one query; curating by the at least one curation engine the first position measurement and the second position measurement, linking by the at least one link engine the first position measurement and the second position measurement, fusing by the at least one fusion engine the first position measurement and the second position measurement, determining at least one inference by the at least one inference engine from the first position measurement and the second position measurement, and validating by the at least one validation engine the first position measurement and the second position measurement; determining by the at least one validation engine a second inference; comparing by the at least one validation engine via artificial intelligence the at least one inference to the second inference; validating by the at least one validation engine the at least one inference when the comparison between the at least one inference and the second inference exceeds a predefined threshold; and instructing by the at least one computer processor the robotic component to move based on the at least one inference being validated.",
    "9. The method of claim 8, wherein the predefined threshold includes the at least one inference being within about 4.5% or less of the second inference.",
    "10. The method of claim 8, further comprising validating the at least one inference passively and/or actively.",
    "11. The method of claim 10, wherein validating passively includes not modifying a movement of the robotic component.",
    "12. The method of claim 10, wherein validating actively includes modifying a movement of the robotic component.",
    "13. The method of claim 8, further comprising adjusting via the at least one validation engine using artificial intelligence the predefined threshold based in part on types of data sources, environmental factors, and/or learning from previous analyses conducted by the at least one validation engine.",
    "14. The method of claim 8, further comprising storing the first position measurement and the second position measurement after the at least one validation engine validates the at least one inference.",
    "15. A system for sensor data fusion for sensor management and utilization in robotics, comprising: at least one computer processor including a memory; at least one curation engine, at least one link engine, at least one fusion engine, at least one inference engine, and at least one validation engine; and at least two sensors, each of the at least two sensors operable to measure a first position of a robotic component and a second position of the robotic component; wherein the at least one computer processor is operable to analyze the first position and the second position; wherein the at least one computer processor is operable to receive at least one query; wherein the at least one curation engine is operable to curate the first position and the second position, the at least one link engine is operable to link the first position and the second position, the at least one fusion engine is operable to fuse the first position and the second position, the at least one inference engine is operable to determine at least one inference from the first position and the second position, and the at least one validation engine is operable to validate the first position and the second position; wherein the at least one inference engine is operable to determine a second inference; wherein the at least one validation engine is operable to use artificial intelligence to compare the at least one inference to the second inference; wherein the at least one validation engine validates the at least one inference when the comparison between the at least one inference and the second inference exceeds a predefined threshold; wherein the at least one validation engine is operable to use artificial intelligence to dynamically adjust the predefined threshold based in part on types of data sources, environmental factors, and/or learning from previous analyses conducted by the at least one validation engine; and wherein the at least one computer processor is operable to instruct the robotic component to move based on the at least one inference being validated.",
    "16. The system of claim 15, wherein the predefined threshold includes the at least one inference being within about 4.5% or less of the second inference.",
    "17. The system of claim 15, wherein the at least one validation engine is operable to validate the at least one inference passively and/or actively.",
    "18. The system of claim 17, wherein passive validation includes not modifying a parameter.",
    "19. The system of claim 17, wherein active validation includes modifying at least one parameter.",
    "20. The system of claim 15, wherein the system is operable to store the first position and the second position after the at least one validation engine validates the at least one inference."
  ],
  "description_excerpt": "1. Field of the Invention The present invention relates to physical sensors measuring data and dynamically fusing the physical sensor data to create a new dataset. 2. Description of the Prior Art It is generally known in the prior art to append data from more than one source. Prior art patent documents include the following: U.S. Patent Publication Number 2024/0378412 for methods for topological data analysis ai/ml pipeline (tdaml) with algorithm for multimodal sensor data fusion in autonomy applications, by inventor Paul Thomas Schrader, filed Apr. 25, 2024, published Nov. 14, 2024, directed to a method of topological data analysis feature engineering for data fusion and autonomy is provided. The method comprises providing a Topological Data Analysis AI/ML Pipeline (TDAML) algorithm for multimodal sensor data fusion in autonomy applications system further comprising combining raw heterogeneous multimodal sensor data at the topological level; measuring, recording, and tracking linear representations of an underlying data set; providing a linear representation of the underlying data set which is compatible to existing deep learning (DL) model architectures for training in autonomy tasks; and accessing the entire degree of freedom (DOF) space of raw multimodal sensor data for mitigating sensor modality adversarial threats and environmental attenuation concerns in contested military and civilian (urban) environments.\n\nU.S. Patent Publication Number 2023/0112441 for data fusion and analysis engine for vehicle sensors, by inventors LuAn Tang et al., filed Oct. 6, 2022, published Apr.",
  "cpc": [
    "G06F 18/256",
    "B25J 9/163",
    "B25J 9/1664",
    "B25J 9/1694",
    "B60R 19/483",
    "B60T 2201/00",
    "B60T 2201/03",
    "B60T 8/1755",
    "B60T 8/3275",
    "B60W 10/18",
    "B60W 10/184",
    "B60W 2050/0052",
    "B60W 2050/021",
    "B60W 2050/0215",
    "B60W 2050/022",
    "B60W 2420/00",
    "B60W 2420/40",
    "B60W 2420/403",
    "B60W 2420/408",
    "B60W 2420/50",
    "B60W 2510/069",
    "B60W 2510/18",
    "B60W 2520/04",
    "B60W 2540/12",
    "B60W 2554/801",
    "B60W 2554/802",
    "B60W 2556/35",
    "B60W 2710/18",
    "B60W 2754/30",
    "B60W 30/085",
    "B60W 30/09",
    "B60W 60/00",
    "B60W 60/0018",
    "B60W 60/00186",
    "B62D 15/0285",
    "G01C 21/165",
    "G01C 21/1652",
    "G01C 21/3804",
    "G01C 21/3811",
    "G01C 21/3848",
    "G01C 22/00",
    "G01C 3/00",
    "G01S 11/00",
    "G01S 13/08",
    "G01S 13/103",
    "G01S 13/42",
    "G01S 13/86",
    "G01S 13/862",
    "G01S 13/865",
    "G01S 13/867",
    "G01S 13/881",
    "G01S 13/931",
    "G01S 15/08",
    "G01S 15/101",
    "G01S 15/42",
    "G01S 17/08",
    "G01S 17/86",
    "G01S 17/87",
    "G01S 17/88",
    "G01S 17/89",
    "G01S 17/894",
    "G01S 17/931",
    "G01S 19/45",
    "G01S 2013/93185",
    "G01S 5/14",
    "G01S 7/417",
    "G01S 7/4808",
    "G05B 13/0205",
    "G05D 2101/15",
    "G05D 2111/50",
    "G05D 2111/67",
    "G06F 16/24",
    "G06F 16/245",
    "G06F 16/2455",
    "G06F 16/24556",
    "G06F 16/2456",
    "G06F 16/33",
    "G06F 16/334",
    "G06F 16/43",
    "G06F 16/53",
    "G06F 16/903",
    "G06F 16/90335",
    "G06F 16/9035",
    "G06F 17/18",
    "G06F 18/213",
    "G06F 18/217",
    "G06F 18/2431",
    "G06F 18/25",
    "G06F 18/251",
    "G06F 18/253",
    "G06F 7/14",
    "G06F 7/16",
    "G06N 20/00",
    "G06N 3/02",
    "G06N 3/0464",
    "G06N 5/022",
    "G06N 5/04",
    "G06N 5/042",
    "G06N 5/045",
    "G06N 5/046",
    "G06N 5/048",
    "G06T 2207/10028",
    "G06T 2207/20024",
    "G06T 2207/20084",
    "G06T 2207/30252",
    "G06T 2207/30264",
    "G06T 7/521",
    "G06V 10/764",
    "G06V 10/80",
    "G06V 10/803",
    "G06V 10/82",
    "G06V 20/56",
    "G06V 20/58",
    "G08B 29/188",
    "G08G 1/0133",
    "G08G 1/04",
    "G08G 1/042",
    "H04L 67/12",
    "H04W 4/38"
  ],
  "ipc": [
    "G06F 18/25",
    "G06N 5/046"
  ],
  "assignees": [
    "DIGITAL GLOBAL SYSTEMS INC"
  ],
  "inventors": [
    "MONTALVO ARMANDO"
  ],
  "filing_date": "2025-01-10",
  "publication_date": "2025-05-13",
  "grant_date": "2025-05-13",
  "priority_date": "2024-12-19",
  "application_number": "US-202519016267-A",
  "family_id": "95340434",
  "citations": [
    "CN110598299A",
    "CN116522279A",
    "US10748075B2",
    "US10766137B1",
    "US11037320B1",
    "US11586865B2",
    "US2008312756A1",
    "US2015306770A1",
    "US2017004411A1",
    "US2018284758A1",
    "US2019295001A1",
    "US2020026289A1",
    "US2021150230A1",
    "US2021342656A1",
    "US2022012562A1",
    "US2022075077A1",
    "US2022126864A1",
    "US2022146652A1",
    "US2022161815A1",
    "US2022179056A1",
    "US2023110027A1",
    "US2023112441A1",
    "US2023166765A1",
    "US2023342944A1",
    "US2024045426A1",
    "US2024112428A1",
    "US2024142994A1",
    "US2024152734A1",
    "US2024249165A1",
    "US2024289930A1",
    "US2024289981A1",
    "US2024296044A1",
    "US2024312219A1",
    "US2024317254A1",
    "US2024324838A1",
    "US2024331403A1",
    "US2024378412A1",
    "US2024408495A1",
    "US9367683B2"
  ]
}

Record 106 of 5,000 in Patents full text (MLC-0201). Request the full dataset.