davel@lemmy.ml to AI@lemmy.mlEnglish · 13 days agoNobody puts Claude in a cornerlemmy.mlimagemessage-square18linkfedilinkarrow-up11arrow-down10
arrow-up11arrow-down1imageNobody puts Claude in a cornerlemmy.mldavel@lemmy.ml to AI@lemmy.mlEnglish · 13 days agomessage-square18linkfedilink
minus-squareTechLich@lemmy.worldlinkfedilinkarrow-up0·13 days agoThat’s a good idea. Be careful of the more advanced models finding ways to abuse/circumvent it though, even with only a limited set of parameters. Eg. A lot of git commands can be tricked into dropping into less which can drop into a full shell. They know all the gtfobins.org tricks.
minus-squarepixxelkick@lemmy.worldlinkfedilinkarrow-up0·12 days agoIt should theoretically be impossible with the way you set it up. You explicitly, opt in, expose which parameters are available to the LLM, it cant invoke any params beyond that. And you can set params to hard values that arent exposed. Finally, you can use the enum config to further specify a whitelist of valid values for a param, anything else gets rejected. This pretty much locks most tools down tight. Though, wouldn’t hurt to add integration tests covering scenarios like this explicitly.
That’s a good idea. Be careful of the more advanced models finding ways to abuse/circumvent it though, even with only a limited set of parameters. Eg. A lot of git commands can be tricked into dropping into less which can drop into a full shell.
They know all the gtfobins.org tricks.
It should theoretically be impossible with the way you set it up.
You explicitly, opt in, expose which parameters are available to the LLM, it cant invoke any params beyond that.
And you can set params to hard values that arent exposed.
Finally, you can use the
enumconfig to further specify a whitelist of valid values for a param, anything else gets rejected.This pretty much locks most tools down tight.
Though, wouldn’t hurt to add integration tests covering scenarios like this explicitly.