Thales: Trustworthy AI still need humans in the loop

As AI gains more ground in mission critical environments, Dr Marko Erman, Senior Vice-President & Chief Scientific Officer at Thales believes AI must deliver levels of reliability, transparency, and trust that surpass those of commercial systems.

As businesses continue to invest in more AI use cases, mission critical environments are also seeing increasing adoption of AI, given the capabilities of the technology to manage almost anything today.

At a recent AI demo presentation by Thales, the technology vendor showcased three use cases in critical industries whereby AI is playing an increasingly significant role. The first demo involved the use of AI in air traffic management. The next-generation air traffic management software relies on an AI algorithm that learns to predict real approach times based on historical data, aircraft types, weather conditions and altitudes.

Another use case demo involved is the use of AI in avionics industrial manufacturing. With Thales producing key components for aviation and with its largest production center in Asia in Singapore, the facility has adopted AI in its testing processes and in monitoring the health and usage of its maintenance equipment.

The third use case demo is the use of AI to enhance radars performance. The demo illustrated how AI can strengthen Thales' systems to deliver dependable situational awareness across sea, city, and airport environments.

While all three use cases demonstrated strong capabilities and enhancements, the reality is a human is still needed to oversee its operations. This includes ensuring that decisions made or suggested by the AIs are not only accurate but also realistically possible.

For Dr Marko Erman, Senior Vice-President & Chief Scientific Officer, Thales, this is where the whole notion of trustworthy AI comes into the picture. As AI gains more ground in mission critical environments, he believes AI must deliver levels of reliability, transparency, and trust that surpass those of commercial systems.

“At Thales, we believe AI should always complement, not replace, human expertise. In situations where lives are at stake, humans must remain at the center of decision-making. We must ensure that AI recommendations remain valid, explainable, and actionable in real-time. Operating in strategic fields such as defense and aerospace, Thales continues to uphold its proven track record of the highest standards of safety, security, and performance,” he said.

When asked if the world is moving towards an era whereby AI can fully manage situations without the need of human intervention, Dr Erman said this would really depend on the use case.

“There isn’t a generic answer. If you build a system where validity is proven and the AI reliably performs its intended function, the role of the human depends on the use case. For scenarios that require millisecond-level reactions, humans must be involved during the planning and development stages, rather than just during real-time execution,” he said.

Dr Erman also pointed out that as each AI use case is very specific in mission critical environments and situations, the first conversation Thales has with a customer is whether there is a role of the human in their system.

Trustworthy AI and data sovereignty

According to Dr Erman, trustworthy AI starts with trustable data. In most countries today, trustable data means the country has control of the data, whereby the data resides in the country. He explained that this is a conversation that is driving the decisions countries make globally when it comes to AI, especially when it comes to sensitive data. Data sovereignty requirements mean the data needs to be hosted either on the cloud or on-premises locally within a country’s borders.

“The issue does not lie with the geopositioning of the cloud structure, but how data within it is protected. This is one of the few commercial solutions Thales provides. Countries and organizations require different levels of protection, and it is not only about encryption, which most cloud providers already offer, but about control. For some of our customers, Thales supplies the cybersecurity solution while the customer retains full operational control of the encryption keys,” added Dr Erman.

Dr Erman also mentioned that the homomorphic encryption of the data is ideally what matters when it comes to data sovereignty. Homomorphic encryption allows for computations to be performed directly on encrypted data without the need for decryption, producing an encrypted result that, when decrypted, is identical to the result of performing the same computations on the original plaintext data. This cryptographic technique enables secure processing of sensitive data in untrusted environments, such as the cloud, ensuring the data's confidentiality throughout the entire computation process

“Data flow typically starts from analytics, before moving to AI. During the data processing stage, where the identification of correlations and functions happen, data would inevitably be revealed. At that point, it leaves its secure cryptographic zone before being re-encrypted. This stage introduces a moment of vulnerability, even if the data is processed in a protected cloud or embedded environment,” he said.

Given the speed of innovation, Dr Erman believes that as things change, there will be a need for a more visible, tangible solution.

“The correlation functions used in encryption technologies are not that complex. It is really just plus, minus, and multiplication, rather than complicated logarithmic functions. We can imagine, perhaps even dream, that one day we will have homomorphic encryption for data. It is not widely available commercially today, but progress is being made.,” he concluded.