Using local models and agents, and customising them for your needs

Rather than sending off all your data to a foreign server, in this workshop you'll learn how to

  • Run open weight models locally with open source tools (llamafile, Apache 2)
  • Use open source agents with these models (pi, MIT)
  • Customise those agents for your own usage (skills and pi extensions)

Given the current data sovereignty concerns, and risks of relying on loss making vendors, I want to teach you that you can use your existing hardware to try out cutting edge models. This is a stark contrast to the current status quo of sending over all your data to foreign LLM inference companies, where you've no control over their software stack and can't improve it.

I also want you to see that you can contribute to making these tools better for everyone. I hope that, by the end of the session, you will publish you Pi extensions under an open source license so that other people can get the benefit of your work!

The speaker's profile picture
Richard Finlay Tweed

Richard Tweed is the security, compliance and infrastructure person at Tessl. He's the OWASP Project Lead for Github-Workflow-Updater-Extension and the lead maintainer of kube-audit-rest.

Mastodon https://infosec.exchange/@RichardoC
LinkedIn https://www.linkedin.com/in/richardftweed/