Patent: Algorithm and Devices for Calibration and Accuracy of Overlaid Image Data

Abstract

View on Google Patents

Abstract

The algorithm of the present disclosure works in conjunction with a user device (e.g., a mobile phone) capable of acquiring and displaying an image, and a server. First and second portions of the algorithm can be resident on the user device and the server, respectively. The algorithm can receive the image, as well as other information from the device such as the location and compass heading of the device, and time of day. The algorithm can obtain information relating to an object in the image, and display the data set on or near the object in the image. The data set can contain information relevant to or about the object. The algorithm can automatically detect whether the data set is in the right location, and if not, calculate and apply an offset to the data set. The algorithm can also store data from multiple users and analyze this data over a period of time to determine the accuracy of the same.

BACKGROUND OF THE DISCLOSURE

 

1. Field of the Disclosure

The present invention relates generally to algorithms, devices and methods for improving the accuracy of sensor data in electronic devices through various embedded, application and cloud-based solutions. In particular, the present disclosure provides an algorithm and associated devices that provide better and more reliable representation of perspective, location, direction, overlaying of content, and spatial relationships for devices.

 

2. Description of the Related Art

It is known that the accuracy of individual sensors and associated data in a mobile device are prone to inaccuracies. So much so, in fact, that enabling valuable user experiences based on this sensor data can frequently be difficult if not impossible. For example, it might be possible to gain access to adequate sensor data to determine a specific user’s device location from GPS, and also accurately determine the precise time, but based on environmental or other conditions, it’s difficult to accurately determine the orientation and perspective of a user’s device.

As a result, for location based applications that rely on directional accuracy, perspective or other heading and angle-dependent solutions are prone to inaccuracy often providing a poor user experience, particularly with content being overlaid against a real-time image and as a result, incorrect information can be provided to the user about their surroundings.

While it is possible for some of the sensors in mobile devices to be calibrated, every sensor is somewhat different and therefore, it becomes the responsibility of the user, the application or the developer to implement workarounds that account for and resolve these inaccuracies for any particular device to enable an accurate output or augmented output through an offset calculation.

 

SUMMARY OF THE DISCLOSURE

In order to improve the accuracy of a network of community sensors, this invention provides for individual calibration and offset solutions. Additionally, it provides a cloud-based or service-based solution that will ingest individual sensor data from each device and associated sensor, along with the user calibration offset data. With the service or cloud-based algorithm of the present disclosure better accuracy is provided across all attached devices and sensors. While all of the calibration and sensor offset parameters can be stored directly on the device and made available to an application or operating system, one of the main advantages of this invention is that it enables the individual device, sensor and offset data to be shared and stored centrally, processed and shared across all devices and related sensors in near real time.

Thus, in one embodiment, the present disclosure provides a system for acquiring an image and overlaying a data set on the image. The system comprises: a user device, the user device comprising an image acquisition sensor, a display, and communication port; a server, wherein the server communicates with said the device via the communication port; and an algorithm comprising a first portion and a second portion, wherein the first portion is resident on the user device and the second portion is resident on the server. The image acquisition sensor acquires the image and displays it on the display. The algorithm acquires the image from the image acquisition sensor, determines the nature of an object in the image, and supplies the data set at a location in the display, wherein the data set contains information relating to the object. The data set comprises information pertaining to the object in the image. The algorithm determines if the location of the data set aligns with the location of the object, and if not, automatically calculates and applies an offset to the data set, so that the data set aligns with the object in said image.

 

CLAIMS

 

What is claimed is:

  1. A system for acquiring an image and overlaying a data set on the image, the system comprising:
    • a user device, the user device comprising an image acquisition sensor, a display, and communication port;
    • a server, wherein said server communicates with said user device via said communication port; and
    • an algorithm comprising a first portion and a second portion, wherein said first portion is resident on said user device and said second portion is resident on said server,
    • wherein said image acquisition sensor acquires the image and displays said image on said display,
    • wherein said algorithm acquires the image from said image acquisition sensor, determines the nature of an object in the image, and supplies the data set at a location in the display, wherein the data set contains information relating to said object,
    • wherein the data set comprises information pertaining to the object in the image, and
    • wherein said algorithm determines if said location of the data set aligns with the location of said object, and if not, automatically calculates and applies an offset to the data set, so that the data set aligns with said object in said image.
  2. The system of claim 1, wherein said user device further comprises at least one of a location sensor, a compass, and a time source, that provide a location of said user device, a heading of said user device, and a time, respectively.
  3. The system of claim 2, wherein said algorithm receives at least one of said location, said heading, and said time from said user device.
  4. The system of claim 3, wherein said algorithm determines said location of said data set based on at least one of said location, said heading, and said time from said user device.
  5. The system of claim 1, wherein said first portion of said algorithm determines said offset.
  6. The system of claim 1, wherein said second portion obtains said information in said data set from a remote source of structured or unstructured data.
  7. The system of claim 1, wherein said algorithm creates an image histogram based on a brightness of said image, and determines said location of said data set and said offset based at least in part on said image histogram.
  8. The system of claim 1, wherein said user device is a plurality of user devices, each of which is in communication with said server.
  9. The system of claim 8, wherein said second portion of said algorithm obtains said location and offset from each of said plurality of user devices.
  10. The system of claim 9, wherein said second portion of said algorithm analyzes said location and offset from each of said plurality of user devices, and transmits adjusted location and offset data to each of said user devices based on said analysis.
  11. The system of claim 9, wherein said second portion of said algorithm acquires weather data from a weather database, and combines said weather data with said location and offset data to determine said adjusted location and offset data.
  12. The system of claim 9, wherein said second portion of said algorithm acquires astronomical data from an astronomical database, and combines said astronomical data with said location and offset data to determine said adjusted location and offset data.
  13. The system of claim 8, wherein said algorithm prompts a user of each of said plurality of devices to name an unidentified object within said image.
  14. The system of claim 13, wherein said second portion of said algorithm stores said name supplied by each of said users, and analyzes said names supplied by said users for statistical relevance.
  15. The system of claim 8, wherein said second portion of said algorithm obtains data said plurality of user devices regarding an unknown object, and tracks historical trends relating to said unknown object over a period of time.
  16. 16. The system of claim 1, wherein said algorithm acquires at least one of a time of day and weather conditions from a remote source, and determines said location of said data set based on at least one of said time of day and said weather conditions.