No one feeds random LLM output straight back though. The whole idea of reinforcement learning is you take some ML model output, check if it is good, and push the model in that direction if it is good.
As long as you believe that e.g. it’s easier to verify a mathematical result than to come up with one, then RL should work.
You took those quotes wildly out of context. Of course there is a hard limit on how much information can be extracted from data. Clever processing won’t break that limit. But only in basic cases have we seen proofs that certain statistical inference methods make optimal use of the data. In complicated systems like neural nets it is basically impossible to prove such optimality. In fact the models are almost definitely not using the data optimally. Processing can help. A lot.
It is hardcoded. The string ‘enoughmuskspam’ is right here https://codeberg.org/rimu/pyfedi/src/commit/b7a9ea0eea3a80f710e0b5b63cf0bbecde60f8bf/app/admin/routes.py#L373
I have noted in other threads and will note again here because people keep attacking me about this: hardcoding does not mean the behavior is not circumventable. It just means the string is in the source file (rather than a config file or database).
The version from 10 years ago, yeah. It’s here https://github.com/reddit-archive/reddit.
There’s a modern and production-ready open source alternative. It’s called Lemmy. You can find the source here https://github.com/lemmynet/lemmy
If the code is bad, you realise only coders would notice it from reading that?
That’s what makes it particularly sad. The people who didn’t read the code have no idea what they’re in for.
What hardcoded bans are you referring to here? The 4chan one that can be disabled?
How about this one? ‘enoughmuskspam’, ‘political_weirdos’, ‘piracy’, ‘memes’ are hardcoded banned. 196 used to be banned to but they removed it from the “bad list”.
Also “can be disabled” does not excuse hardcoded filtering. If they’re serious they could implement a config system in an hour.
I wonder, would these users still use PieFed if they have seen its codebase? Maybe it can be one day but right now it’s 100% not production-grade software. Nonsensical hardcoded bans and blocks everywhere. >1000 lines of Python in a single file. Uses regex to parse HTML. The list goes on…
I didn’t downvote your comment. But let me argue anyway.
Lots of people flock to PieFed because it is not made by the Lenmy devs who are unpalatably heavy handed in their “moderation” on the ml instance. One would hope this means PieFed offers more freedom to use the software how you like. So it’s funny that it’s even stricter “moderation” AND NOW ITS HARDCODED so it affects every instance.
lol hardcoded shit everywhere. that codebase is so bad it’s entertaining. you should make a standalone post about this here and crosspost to !programming_horror@programming.dev
Reinforcement learning makes the model better over time, so why should there be fewer and fewer good results?
If you’re talking about the rate of improvement going down, then yes, of course. That’s bound to happen (unless you have an actual intelligence explosion, but in that case you won’t know what “good results” even mean anyway).