Filter articles by:
Date published
From:
To:
Article keywords
Article type

ºÚÁϳԹÏÍø’s OpenSI researchers develop framework to integrate and interpret AI tools

Larissa Fedunik

25 November 2024: A team of researchers at the University of Canberra’s Open Source Institute (OpenSI) recently published a proof-of-concept framework that helps decision-makers interpret artificial intelligence (AI) decisions.

CoSMIC (the Cognitive System for Machine Intelligent Computing) is a platform that integrates various AI tools, including Large Language Models (LLMs). This allows it to interpret and explain AI predictions or results to human agents in plain English.

Like all OpenSI projects, CoSMIC is open source – meaning that its source code is openly published and available to edit.

“This means that people from the community can implement their own solutions on top of it,” said AI researcher Dr , Research Chair of OpenSI.

The CoSMIC will be presented at the Australasian Conference on Information Systems (ACIS) 2024 by the OpenSI researchers, including Dr Kuhn, research fellow Dr Danny Xu and PhD student Muntasir Adnan.

The conference will bring together researchers working on digital technologies that promote a sustainable society and will be hosted at the University over 4–6 December.

The concept demonstration described in the paper uses a chess engine: a chess-playing AI that calculates chess moves with more precision than a human player. The outputs of the chess engine include probabilities of potential moves, which have not been easily interpretable to humans – until the team used CoSMIC to integrate the engine with LLMs.

“The chess engine calculates the chess moves, and the LLM creates the response for the user interface, so it gives you suggestions in English about what your next moves should be,” said Dr Kuhn.

AI models such as neural networks process data in a way that mimics the human brain, and are frequently described as “black boxes” because of their complex mechanisms and opaque decision-making processes. CoSMIC’s strengths lie in its ability to make these kinds of “black box” decisions understandable.

“We can use different services that talk to each other, and reach a decision that isn’t necessarily understandable by humans,” said Mr Adnan.

“This decision can be passed on to any open source LLM, which can summarise it and give you understandable feedback. The whole decision-making process is therefore very transparent.”

The team evaluated their system with different LLM families, including GPT-4o (ChatGPT), to show that the framework can be successfully integrated with a range of LLMs. This makes CoSMIC highly adaptable for a range of computational tasks.

“With CoSMIC, you can use a model like ChatGPT – which is very large and knows a little bit about everything – or you can use a small model locally on your computer that’s not necessarily very large, but trained to do some specific task,” said Dr Kuhn.

Existing applications like Microsoft AutoGen have similar properties to CoSMIC in terms of integrating different services, including LLMs. However, AugoGen can currently only support code execution in a local environment, such as a developer’s computer, where they can test and refine their code before sending to a web server.

The CoSMIC framework is novel because it only uses LLMs for the user output, as opposed to other AI services.

“An LLM itself is solely based on language – it’s not as intelligent as it seems,” said Mr Adnan. “Our approach uses an array of AI services to get the job done.”

This means that CoSMIC can be used for a variety of industry applications beyond the chess engine demonstration, from medical diagnostics to financial forecasting. “The services that can be added are limitless,” said Mr Adnan.

An example would be the use of CoSMIC in AI fraud detection tools, which are frequently used in banking to flag transactions as fraudulent. Developers could use CoSMIC to interact with an LLM to explain why the transaction has been flagged, which would be extremely useful for human auditors.

The team is currently developing a tool with CoSMIC to help developers generate the best possible code. Code that is clearly explained will be highly valuable for users, particularly when they build on the code for their own applications.

“We’re trying to make every decision made by generative AI explainable, so that when you generate a piece of code, it’s not just generated randomly – the framework needs to explain why,” said Mr Adnan. “That way any software developer and any client can use it.”

While LLMs like ChatGPT can help developers write code, they usually contain a lot of errors, known as bugs. OpenSI’s code generation tool addresses this by breaking up generated code into smaller sections, and running tests in a loop. If there is a bug, the code is sent to the LLM for improvement, and tested again in an iterative cycle until the desired level of accuracy is reached.

“It should be able to point out the shortcomings of its initial solution, and come back to me and say, ‘OK, if you try this way, that will probably solve the problem’,” said Mr Adnan. “This makes the whole code generation process more reliable.”

The next step for the team is working on a custom user interface for CoSMIC.

“We aim to make a friendly, well-documented interface for non-software developers,” said Dr Xu. “One of our student interns is working on this, and we hope to have a demonstration ready in the next month.”

They also plan to introduce and merge more services for customers and collaborators.

“We would like to establish a broad collaboration with stakeholders, including other research institutes, internships, clients from government or industry – all are welcome,” said Dr Xu.

Photos by Tyler Cherry