The goal of this session is to exchange views on how organizations can build Trustworthy AI systems.
Starting by demonstrating PyThia, our solution for ensuring Trustworthy AI we would like to ignite discussions and compare experiences and perspectives on the below topics:
a) What are the pros and cons on using predefined checklists for evaluating how an organization governs an AI system?
b) Is there a golden standard for Transparent AI that can rule all explanations?
c) We’re evaluating the fairness of an AI model in a quantitative way, is this enough?
d) How is it possible for an organization to find a balance between building Trusted AI versus optimizing for efficiency and maximize their business benefit?
It is our aspiration that this session will at least help our team gather feedback, for participants to share knowledge and ideally for consensus or a shared vision to surface.
We plan to follow the same approach independently of the number of participants, that is encourage discussion and exchange of views in order to create a list with best-practices. Now, if there are many participants (say 30) we plan to leverage the use of Q&A/Instant Messaging capabilities for gathering quick feedback. If there are only a few participants, then we may give emphasis on live discussion using the Video/Audio conferencing tooling. For starters we will definitely need the later as we plan to show a demo of our solution for Trustworthy AI named PyThia.
What is the goal and/or outcome of your session?:The goal for our session is stimulating participants exchanging ideas and experiences on how organizations can ensure their AI systems are Trustworthy. Also, for our team to collect feedback on the technology we're building on this domain, PyThia.
It would be ideal if at the end of the session there is consensus on what can be the best-practices for:
a) Ensuring an organization has the necessary maturity and accountability processes for building and operating AI models,
b) Monitoring how fair an AI system is over-time,
c) Defining what is the most appropriate and reliable way for explaining the decisions of an AI system.
d) Finding the balance between Trust and optimizing an AI model for efficiency or for maximizing for business benefit.
Finally, using a whiteboard we can keep notes and elaborate on them in order to come up with the list of agreed best-practices.
Some ideas that we may want to explore in order to continue the work from our session are:
a) Create a digital meetup on Trusted AI and especially in the area of F.Acc.T. (Fair Accountable Transparent) AI,
b) Co-develop solutions/tooling for ensuring the Trustworthiness of AI. Our startup is already building PyThia and we're keen in developing partnerships, collaborations and friendships with peers and people who think alike,
c) Seek for a joint publication at a known community venue (e.g. Towards Data Science) or if there is potential continuing some experimentation that may lead to a publication at an academic venue.
Yiannis is the founder of Code4Thought a technology startup who ensures AI is Fair, Accountable and Transparent. He has over 17 years of experience in evaluating large scale software systems.