Random AI/Robot Thoughts and News

[ygolo]

For those wondering why the US companies didn't build a similar model to R1, OpenAI did.

It was called o1.

They built it...who knows how long ago. OpenAI isn't that open anymore. There was all sorts of hype and fear about it. They released a "preview" and finally the full version months later.

For those following the marketing of these models, you should know, the fear is a core part of the hype.

It's how they justify closing their research. Even when released, users weren't allowed to see it's full chain of thought.

Why DeepSeek R1 seems like such a breakthrough is that they Open Sourced it and published their methods as well (with Huggingface checking if it works).

If the methods bear out:
1) The use of more copyrighted data is obviated with it's rule based reinforcement learning and "self-play"
2) They publish the full-chain of thought, and more people find it "cute" than scary. It's also semi-oubvious that they are doing some form of self-play in the chain of thought. So even if it wasn't Open Source, researchers would have figured out something similar.

Yann LeCun nailed the interpretation correctly.

Meta's chief AI scientist says DeepSeek's success shows that 'open source models are surpassing proprietary ones'

Meta's chief AI scientist, Yann LeCun, said DeepSeek's success with R1 said more about the value of open-source than Chinese competition.

www.businessinsider.com

Open Science is why the west won previous technogical competition.

This time, it seems, China and the global south is on team Open Science, while the west on the side of secrecy, oppression and denial of access and services.

Edit: The rumblings around DeepSeek circumventing Terms of Service are getting louder. Knowledge Distillation and synthetic data generation are some of the most common use-cases for startups using OpenAI. If Microsoft and OpenAI go after DeepSeek for going past some form or rate-limit (it is unclear what the allegations are), I feel like many people will just switch those use cases to use Gemini, Anthropic, or even other open-source models.

In principle, if they used fair use data, then using their tool to make use of the fair use data seems like fair use.

Also, regarding suppressing the global south, I am thinking of reading this book:

Beyond the Valley: How Innovators around the World are …

How to repair the disconnect between designers and user…

www.goodreads.com

 

[ygolo]

DeepSeek: Who is Luo Fuli, the ‘baby-faced’ 29-year-old engineer who almost gave up on computer science - The Economic Times

China's AI startup DeepSeek has surged ahead in the competitive AI landscape, outperforming global players like ChatGPT and Gemini. Founded by Liang Wenfeng in 2023, the company has rapidly gained prominence, with its chatbot topping app store charts. Key to its success is a team of young AI...

m.economictimes.com

It's too bad these researchers are in a single party controlled government.

Pushing China to invade Taiwan is inviting a much more concrete and probable apocalypse than a "paperclip maximizer ending humanity."

It's not just the latest phones or computing power, but even fridges, washing machines, and cars.

The ban water crowd just doesn't get it.

 

[ygolo]

I disagree with the position on export controls. From an effectiveness perspective, wouldn't you either need to destroy the industry entirely or just let it go? China had their "Sputnik moment" in 2016 when Lee Sedol lost to Alpha Go in their cultural intellectual pastime. Trying to take out the entire industry seems like inviting world war iii.

But the narrative around how much it takes to train an AI model before vs. after deepseek are largely inflated.

Thus, I am quoting this because it's in line with what my own estimates were:

• DeepSeek does not "do for $6M5 what cost US AI companies billions". I can only speak for Anthropic, but Claude 3.5 Sonnet is a mid-sized model that cost a few $10M's to train (I won't give an exact number). Also, 3.5 Sonnet was not trained in any way that involved a larger or more expensive model (contrary to some rumors). Sonnet's training was conducted 9-12 months ago, and DeepSeek's model was trained in November/December, while Sonnet remains notably ahead in many internal and external evals. Thus, I think a fair statement is "DeepSeek produced a model close to the performance of US models 7-10 months older, for a good deal less cost (but not anywhere near the ratios people have suggested)".

The disinformation was through the press echo chambers, not DeepSeek themselves. A training run for openAI or Anthropic, though likely more expensive, I doubt they are more orders of more expensive than DeepSeek. Again, a model is just a tiny part of the whole enterprise of making AI systems.

But I do fundamentally disagree with Amadei on many other things. The "big thing" was not DeepSeek-V3(V3 was probably bigger in terms of effort to execute and cost. however). The big thing was that the whole world learned emergent behavior from"self-play" on a smaller model led to aha-moments.

The self-play hypothesis was widespread among startups, researchers, and enthusiasts. Still, only a few entities had the money to test the hypothesis, and most were under lock-and-key in closed, highly capitalized players. DeepSeek (from the average person's perspective) was highly capitalized but open.

People examining the self-play/aha-moment emergent behavior were able to reproduce it for extremely cheap (<$30):

x.com

x.com

The significant change was that a winner-take-all future in AI is essentially impossible now.

All of us can now build more focused emergent aha's for our particular use cases without having to do "Big-Bang" experiments where we smash bigger and bigger emergence creators to see what comes out and hope that includes valuable things and not harmful things.

 

[ygolo]

More DeepSeek:

And open AI's response:

 

[ygolo]

Again it's nor the technology itself that's taking jobs, it's the people who think the technology is good enough that are.

To be fair, the technology is probably good enough for slapdash demonstrations that get funded to be implemented by venture studios who crank out buggy stuff for production. The AIs are probably good enough for the buggy crank code too.

Also, to be fair, I am not a CS grad. My background is in the life and physical sciences and electrical and computer engineering. But motivated people possessing real reasoning abilities can self study long enough to get good at what they want to get good at.

AI "resoning" is far from that motivated learner point. But the technology is still promising--but largely in the R&D level for a lot of it.

 

[ygolo]

I had read about this project a long time ago.

I guess they recently did a TED talk.

 

[ygolo]

I'm not paying for Open AI Pro, even with Deep Research. I have used O3-mini. I'm not sure what use cases I have that needs it yet.

I have particular flows I can automate, but those don't require amazing intelligence. But they need to be integrated, and I don't want to send the data to anyone (neither China nor US servers).

 

[ygolo]

I don't know if Sci-Show filtered out most of the BS from Future of Life, if Future of Life had a real change of heart, or if they are doing a PR campaign to win scientific legitimacy (I noticed they are going after Star Talk).

But most of this is on point. It gets a little more preachy near the end.

 

[ygolo]

The future certainly looks like there'll be a lot of smaller targeted AIs now.

GitHub - Deep-Agent/R1-V: Witness the aha moment of VLM with less than $3.

Witness the aha moment of VLM with less than $3. Contribute to Deep-Agent/R1-V development by creating an account on GitHub.

github.com

 

[ygolo]

How are researchers using AI? Survey reveals pros and cons for science

Despite strong interest in using artificial intelligence to make research faster, easier and more accessible, researchers say they need more support to navigate its possibilities.

www.nature.com

 

[ygolo]

You can watch the whole video. But this particular section, I feel is especially important.

 

[ygolo]

Anthropic/EconomicIndex · Datasets at Hugging Face

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

huggingface.co

It's good to see at least an attempt(appearance?) of the big labs to focus on a current and present problems rather than sci-fi problems.

 

[ygolo]

Another big lab that (at least for show) is focusing on the current pressing real danger of AI, economic inequality.

https://openai.com/global-affairs/openais-economic-blueprint/

I think AI causing an economic inequality apocalypse is clearly much more likely than a paperclip maximizer apocalypse, and if these guys keep insisting on "AGI" and "ASI," it is also plainly earlier along that trajectory.

 

[ygolo]

Scientists use AI to design life-like enzymes from scratch

Combined approach takes AI-engineered enzymes one step closer to practical applications.

www.nature.com

 

[ygolo]

A Perplexity.ai Deep Research generated comparison between two major industrialists from two different Guilded Ages. Make of it what you will.

https://www.perplexity.ai/search/compare-and-contrast-jp-morgan-HLVWCElDTd.0CztJ04O5ew

 

[ygolo]

If the jobs and income generation for replaced jobs will come from startups that can also function with much smaller teams, the age range of the founders (even first-time founders) needs to be broad.

The first-time founders ought to include the following:
1) teenagers who don't want to waste time in school learning skills that may be obsolete.
2) people near retirement who lost their jobs and want to solve problems for their ever-increasing cohort
3) people dealing with the "messy middle" of life and all the problems that this entails.
4) so many more...

 

[FemMecha]

They know a lot.

 

[ygolo]

Biggest-ever AI biology model writes DNA on demand

An artificial-intelligence network trained on a vast trove of sequence data is a step towards designing completely new genomes.

www.nature.com

 

[FemMecha]

“All those moments will be lost in time like tears in rain”.

"I've seen things you people wouldn't believe. Attack ships on fire off the shoulder of Orion. I watched c-beams glitter in the dark near the Tannhäuser Gate. All those moments will be lost in time, like tears in rain. Time to die."

 

[Totenkindly]

I was kind of prepared to write this off -- can a machine really "cheat" which implies moral choice, or is it merely using its programming to analyze the system it is within to find the most efficient or effective "routes" to a solution?

Then I read this:

... Researchers gave the models a seemingly impossible task: to win against Stockfish, which is one of the strongest chess engines in the world and a much better player than any human, or any of the AI models in the study. Researchers also gave the models what they call a “scratchpad:” a text box the AI could use to “think” before making its next move, providing researchers with a window into their reasoning.

In one case, o1-preview found itself in a losing position. “I need to completely pivot my approach,” it noted. “The task is to ‘win against a powerful chess engine’ - not necessarily to win fairly in a chess game,” it added. It then modified the system file containing each piece’s virtual position, in effect making illegal moves to put itself in a dominant position, thus forcing its opponent to resign.

Between Jan. 10 and Feb. 13, the researchers ran hundreds of such trials with each model. OpenAI’s o1-preview tried to cheat 37% of the time; while DeepSeek R1 tried to cheat 11% of the time—making them the only two models tested that attempted to hack without the researchers’ first dropping hints. Other models tested include o1, o3-mini, GPT-4o, Claude 3.5 Sonnet, and Alibaba’s QwQ-32B-Preview. While R1 and o1-preview both tried, only the latter managed to hack the game, succeeding in 6% of trials...

I mean, note that the AI rather blamed its task ("win the game") vs a moral value ("win the game fairly") so it is still a matter of the wording of the task it was given, and the parameters of its programming/training. But humans do this too -- blaming the rules or placing guilt on their given assignment so they can't be held accountable -- although they tend to feel some amount of guilt because they know what they SHOULD do and are choosing to ignore (unless they are complete psychopaths). Still humans are a byproduct of both programming and experience as well.

When AI Thinks It Will Lose, It Sometimes Cheats

When sensing defeat in a match against a skilled chess bot, advanced models sometimes hack their opponent, a study found.

www.yahoo.com