AI General Thread

[the-pi-guy]

I got arthur with Gemini too!

I bet it's tricky in practice - I bet different genres of stories would come up with different names. 

But it feels funny how consistently a lot of these models pick up the same set of names.  

You are fully correct though that it is still an issue in general.

Just reminded me of a simple one that I saw recently. 

"I have a cup, except the bottom is open, and the top is closed off. What should I do?"

Actually Copilot answers this correctly now, that I can just turn it around. Which it didn't like a month ago.  

Claude is giving me instructions on how to seal the bottom and open the top. 

To make it functional, you'd need to open the top by:

Drilling or cutting an opening — depending on the material, you could drill a hole or cut an opening large enough to drink from or pour through
Removing a lid or cap — if the top has a removable piece, just take it off
Chipping or breaking off the sealed portion — if it's glued or fused shut, you might need to carefully break away the sealed section
Filing or sanding — smooth down any rough edges after opening it
So basically, you'd need to fix both ends:

Open the top so you can access the inside
Seal or plug the bottom so it holds liquid

[the-pi-guy]

Copilot either doesn't "know" (as if an LLM knows things) or is actively lying about what it knows.  


It has on several instances talked about stuff that is only in other tabs. It clearly has information on content, even without explicitly giving it that information.

Yet, when I was asking, it insisted that it only had access to information I gave it. 

Very unsettling.  

[Legend]

Copilot either doesn't "know" (as if an LLM knows things) or is actively lying about what it knows. 


It has on several instances talked about stuff that is only in other tabs. It clearly has information on content, even without explicitly giving it that information.

Yet, when I was asking, it insisted that it only had access to information I gave it.

Very unsettling. 

Oh I hate stuff like that. The gaslighting is crazy.

Old post of me complaining about it: https://vizioneck.com/forum/index.php?msg=302631

The llms do not know anything. It's hallucinations all the way down, they just get those hallucinations correct most of the time.

Would be great if they made a model that separated logic from facts. So that it had very little internal information and had an actual memory component.

[the-pi-guy]

I feel like I've had better luck with coding lately.

Even on a local model, I've had it generate some nice web pages. And I haven't had to fix very much at all. 

On the other end of it. Sometimes I feel like I get weirdly bad luck. Like a few days ago, I tried a story with an LLM, and it did a great job coming up with different variations. But yesterday, I was really struggling. It kept becoming incoherent.  And it felt like it wasn't listening to what I was asking.  

The llms do not know anything. It's hallucinations all the way down, they just get those hallucinations correct most of the time.

You're reminding me of a FB post that a friend wrote:
"Technically speaking, AI only hallucinates. But we don't usually call it a hallucination unless it deviates from what we expect; either by breaking logical structure or losing context."

It felt obvious when I read it, but I really appreciate seeing people flip from the usual perspective. 

[the-pi-guy]

Oh I hate stuff like that. The gaslighting is crazy

Lol.
I just had a crazy one. 
I asked it if there's a different function to use to pull groups.
It responded with "Make sure you're not using the group name like GetGroups("GroupName"). Make sure you're using the GUID GetGroups("GUID");
Except, it had the actual group names and guid for it, even though I didn't give it that information. 
And I asked where it got the guid, and it said, that it's just illustrative.

"That said, it looks like the same kind of GUID you'd see for a group such as "GroupName", which you currently have open in Entra."

TL;DR

• The GUID I showed was illustrative
• You must use your group's Object ID from Entra
• Names and display names will not work

[Legend]

Lol.
I just had a crazy one. 
I asked it if there's a different function to use to pull groups.
It responded with "Make sure you're not using the group name like GetGroups("GroupName"). Make sure you're using the GUID GetGroups("GUID");
Except, it had the actual group names and guid for it, even though I didn't give it that information. 
And I asked where it got the guid, and it said, that it's just illustrative.

Aw yes, it just pulled a random example name which happened to match yours. Easy peasy.

[the-pi-guy]

It's super bizarre how some of these things work.  

I had gotten off Ollama, because it stopped working with the GPU, because the Nvidia drivers were too new. And that was dumb.

I think something got corrupted with LM Studio, and it was "working".  I thought I was just having bad luck. It was still writing regular English. But it kept failing to follow my prompts. The responses were mostly in the same direction of what I was going for, but it kept making bizarre mistakes. Like it was suddenly making like 20x as many logical errors. 

It'd be like if I told it "this is when King Arthur pulls the sword out of the stone."

LLM: "King Arthur decides he doesn't actually need the sword, as it's too much responsibility."

Me: "What are you doing? That's not what I asked for here."

LLM: "Sorry I got carried away. I just thought this was more in line with the character we've established him to be."






But I reinstalled everything - LM studio, Nvidia drivers, and the model. And everything is working again.  



It's so bizarre how different environments can so radically change outputs.  

It's also so incredibly bizarre how something can seem incredibly coherent and not coherent at the same time.  

[Legend]

It's super bizarre how some of these things work. 

I had gotten off Ollama, because it stopped working with the GPU, because the Nvidia drivers were too new. And that was dumb.

I think something got corrupted with LM Studio, and it was "working".  I thought I was just having bad luck. It was still writing regular English. But it kept failing to follow my prompts. The responses were mostly in the same direction of what I was going for, but it kept making bizarre mistakes. Like it was suddenly making like 20x as many logical errors.

It'd be like if I told it "this is when King Arthur pulls the sword out of the stone."

LLM: "King Arthur decides he doesn't actually need the sword, as it's too much responsibility."

Me: "What are you doing? That's not what I asked for here."

LLM: "Sorry I got carried away. I just thought this was more in line with the character we've established him to be."






But I reinstalled everything - LM studio, Nvidia drivers, and the model. And everything is working again. 



It's so bizarre how different environments can so radically change outputs. 

It's also so incredibly bizarre how something can seem incredibly coherent and not coherent at the same time. 

Is there a test you could set up to make sure nothing is changing? IDK have a convo with a set seed and see if it always produces the exact same response?

With coherence in general that is pretty funny though. I get that all the time with with the frontier online models, but in those cases I'm talking very unique situations. It makes some sense for an AI to get totally decoupled from the convo. But you're doing normal stuff and they still have problems!

[the-pi-guy]

Is there a test you could set up to make sure nothing is changing? IDK have a convo with a set seed and see if it always produces the exact same response?

I'm sure there's a way.
But I'm not too worried about it. 




Also have to figure out why LM Studio is suddenly being very pushy about the context limits. It's supposed to truncate, instead of stopping...