AI General Thread

[the-pi-guy]

Been messing around with video generation.

I can make a 10 second video at 480p in like 8-10 minutes on my local machine. I think I'm using what is supposed to be one of the better local models.

It's in such a weird place.

There's lots of stuff that looks good at a glance, but it tends to break in motion.

I generated like 10 videos.

I would say like 6 of them were decent.  The others had some body horror.

There was 1 that was clearly broken, but it was broken in a way that didn't matter - if that makes sense. 


Something that felt bizarre, is that it felt like it couldn't handle transitions, and not even in the sense that stuff looks broken; but that it didn't even seem to try.  

Like I feel like if I asked it to make a video of someone picking up a baseball and throwing it, I feel like it wouldn't get to the second step. It would make an okay video of someone picking up a baseball and end there. 

no where near good enough for me.  I spot complete fabrications several times a day.  makes me not trust the answers when I'm less familiar with the topic.

Yeah, but I mean when we're trying to make fabrications.

[Legend]

Yeah, but I mean when we're trying to make fabrications.

Had a weird issue with openai yesterday and fabrications.

I told it to make a photorealistic picture of a person swimming in the cambrian period with all those creatures of the time. It made more of an illistration, exactly like most cambrian artwork.

I told it to fix it and make it photorealistic, and it refused! Said it was against the rules because it might make someone believe it was a real picture and that humans lived back then.

[kitler53]

Yeah, but I mean when we're trying to make fabrications.

i've spent less time with trying to use AI to create images and no time trying to make videos.   

i think i've told this story here before but,..  my 1 experience with image generation was attempting to get an image to use for the block party i was organizing.   i thought AI might might do a great job at making that task less work than what i've done in prior years which is cobble together some clip art.

i was soooo very wrong.   for sure i'm inexperienced at this but the images AI generated were horrible.  terrifying really.   and soooo much repetition.    i spent at least 4x the time before i got an image that was at least decent.   at the block party there was a large discussion about how repulsed they were of my flier because of that image.   ..so like at least we had a fun topic of conversation to bond over.

[the-pi-guy]

i've spent less time with trying to use AI to create images and no time trying to make videos. 

i think i've told this story here before but,..  my 1 experience with image generation was attempting to get an image to use for the block party i was organizing.  i thought AI might might do a great job at making that task less work than what i've done in prior years which is cobble together some clip art.

i was soooo very wrong.  for sure i'm inexperienced at this but the images AI generated were horrible.  terrifying really.  and soooo much repetition.    i spent at least 4x the time before i got an image that was at least decent.  at the block party there was a large discussion about how repulsed they were of my flier because of that image.  ..so like at least we had a fun topic of conversation to bond over.

I don't think any of these things are close to being a good replacement for anything. I think a lot of it is still really cool. And a lot of it is good enough for what I'm using it for. Which there are lots of things I'm definitely not using it for. 

[Legend]

Programing wise, it's only a google replacement imo.

Like I needed to set up an email Sieve to copy all incoming mail to one inbox. I don't even know what Sieve is. But instead of googling it, I can get the three lines I need and be done.

[the-pi-guy]

[Legend]

Gemini was much better at pokemon.

And interesting video. To me it feels like progress has slowed to a crawl, so it's interesting that this metric paints the opposite picture.


I feel like AI is so much worse when doing novel things.

[kitler53]

from my perspective,.. an 80% success rate is statistically identical to 100% fail rate.  

[the-pi-guy]

For some reason, Gemma 3:12b is super slow on my PC. 

I had better luck with Mistral 3 24b. 

Llama 3 is pretty good too. 

[the-pi-guy]

Gemma 3:1b is hilariously bad. 

I got some results that didn't even seem like real words.  

[the-pi-guy]

ChatGPT's hallucination problem is getting worse according to OpenAI's own tests and nobody understands why

Hmmm, I wonder why.

[kitler53]

it's generative AI.  making up battleship isn't a problem it's the point.  

[the-pi-guy]

I find it interesting the differences between models.

Like some models really like certain names. If you tell it to write a story about a road trip, they like to pick different names. The Gemma 3 models really like the name Liam.

QwQ really wants to talk about snacks for the road trip, which I haven't seen with the other models.

QwQ is the only model I've seen that will randomly say the Chinese word for something randomly...

[Legend]

I find it interesting the differences between models.

Like some models really like certain names. If you tell it to write a story about a road trip, they like to pick different names. The Gemma 3 models really like the name Liam.

QwQ really wants to talk about snacks for the road trip, which I haven't seen with the other models.

QwQ is the only model I've seen that will randomly say the Chinese word for something randomly...

I get random chinese words with gemini 2.0 flash.

For your road trip and name tests, I've really wanted to train my own micro model to fix that. Maybe I've even already posted it, but train a really dumb llm on a small corpus that you like. Then run it in parallel with the smart llm and merge the token predictions per next token. Should make it sound like the small corpus without having to fine tune the big llm.

it's generative AI.  making up battleship isn't a problem it's the point. 

Well it's a problem for everyone else.

I had my ai the other day tell me it's more energy effecient to leave my hot tub on 24/7!!!

Edt: and just now the ai told me that a loop running race is downhill with a net elevation change. I asked how that could be possible, since it starts and ends at the same point, and it said "uneven terrain."

[the-pi-guy]

qwq is kind of impressive in a lot of ways. 

But I also find it really annoying. It does some "thinking" before it does what you ask. 

Like imagine if you ask it to write a romantic four line poem about flowers. 

You would probably get in response, a few pages first about how the user wants a romantic poem about flowers, they probably mean roses since roses are a romantic flower. 

It puts out some of the longest text responses I've seen in any of these local models I've tried, yet I've literally had it not do what I asked a few times, just because it spent so long trying to figure out what I wanted.