AI General Thread

[Legend]

For some reason, running local models feels goofier than it should be. And I'm not sure why. Like for some reason, prompt adherence is worse when I'm using different Android app.

Not sure if they're formatting the requests differently, or passing different default values for the model.

Different sampling method/temperature?

[the-pi-guy]

Different sampling method/temperature?

Most of the apps let you change the temperature, top K, top P. 

I'm leaning towards formatting being different, which would be harder to fix as an end user.

[the-pi-guy]

LM Studio has so many options you can mess with. 

You can even set how the context window is managed (truncate middle, rolling).  

[Legend]

LM Studio has so many options you can mess with.

You can even set how the context window is managed (truncate middle, rolling). 

Is there a way to run two models in parallel so that the next token can be sampled using both their outputs? Like averaged or multiplied or whatever?

[the-pi-guy]

Is there a way to run two models in parallel so that the next token can be sampled using both their outputs? Like averaged or multiplied or whatever?

Not that I know of.

[the-pi-guy]

Something that I've really wanted to do is combine two bots.

I don't think the bots are actually smart enough to do this.
I just tried something with copilot, and it kind of can do it as a single instance. But it felt sloppy.

But it would be cool to have two bots - for example have one of them be a Dungeon Master and have one be a player. Where the latter one has no knowledge of the other's thoughts.

You could probably do it with a single instance, but I think chances are good that the model would get confused. 

The way I'm thinking that I would try it, would still technically be a single instance, but just having a program to carefully cull different parts of the context to simulate each bot.

[the-pi-guy]

I have a dumb question. And maybe it's just dumb because Copilot is telling me that some of these systems are actually just inferring and they're not generative.  And maybe that's not actually a difference that makes any sense. Basically because upscaling a 480p image to a 1080p image doesn't feel generative. It's just inferring what would be there. 

What would be the line between generative and nongenerative/inference AI in different instances? 

I feel like if I ask a model to create a completely new image, that is very obviously generative AI.

What if I give an AI half an image and ask it to fill in the other half? That still feels pretty generative. 

But what if I give an AI half an image in a checkerboard pattern. That feels kind of different. 

But what if I give the AI 1/16th of an image in a kind of checkerboard pattern. 


The image generator obviously would have more degrees of freedom the further away from the filled in image it has. But to some extent that feels like the only difference. 

[Legend]

Good question Pi.

I'd say I'd agree with your point of degrees of freedom. But we could refine it slightly and say it is not about degrees of freedom themselves, it's about being able to make something unexpected.

If you can see the result and think "that is not the result I was expecting" but it did everything technically correct, then that is generative imo. Maybe it puts a chess piece in the corner of the image and surprises you.