AI Pioneer Cerebras Pioneers Generative AI That Turns OpenAI Upside Down

The world of artificial intelligence, especially the field of automatically creating text and images, widely known as “generative AI,” is in danger of closing its horizons. The Chilling Effect of Companies Deciding Not to Disclose Research Details.

But the shift toward secrecy may have prompted some players in the AI ​​world to step in and fill the disclosure void.

On Tuesday, AI pioneer Cerebras Systems, maker of purpose-built AI computers, and world’s largest computer chip, Released as open source Unlimited use of several versions of generative AI programs.

The program is “trained” by Cerebras. This means that Cerebras’ powerful supercomputers are used to deliver optimal performance, reducing some of the work that external researchers need to do.

“Companies have made different decisions than they did a year or two ago and we disagree with those decisions.We will not disclose technical details. The company announced its latest generative AI program this monthGPT-4, a widely criticized move in the world of AI research.

again: With GPT-4, OpenAI chooses between confidentiality and disclosure

“We are not only researchers, but 3, 4, 5, 8 LLM people, but also startups, medium-sized companies, and large companies are training large-scale language models in a vibrant environment. We believe in a community that is open and vibrant, good for us and good for others,” Feldman said.

The term large language model refers to an AI program based on machine learning principles where a neural network captures the statistical distribution of words in sample data. This process allows a large language model to predict the next word in order. Its capabilities underlie popular generative AI programs such as ChatGPT.

The same kind of machine learning approach applies to other areas of generative AI. OpenAI’s Dall*E generating images Based on the suggested phrase.

again: Best AI Art Generators: Try DALL-E2 and Other Fun Alternatives

Cerebras posted seven large language models in the same style as OpenAI’s GPT program that started the generative AI craze in 2018. Here is the code: Available on website AI startup Hugging Face and on Github.

Program sizes vary from 111 million parameters (neural weights) to 13 billion. Generally speaking, the more parameters you use, the more powerful your AI program will be, and the more different performance your Cerebras code can provide.

The company has not only released the program’s source in Python and TensorFlow formats under an open-source Apache 2.0 license, but has also published details of the training regimen that the program uses to bring its functionality to a developed state.

Its disclosure will allow researchers to examine and reproduce Cerebras’ research.

According to Feldman, the Cerebras release is the first time a GPT-style program has been published “using state-of-the-art training efficiency techniques.”

Other published AI training work, such as OpenAI’s GPT-4, hides technical data or programs are not optimized during development. That is, the data supplied to the program is not scaled to the size of the program. As explained in the Cerebras technical blog post.

Such large language models are notoriously computationally intensive. Released Tuesday, Cerebras’ work was developed on a cluster of 16 of his CS-2 computers, computers the size of dorm refrigerators specially tuned for AI-style programs. cluster, previously disclosed by the company, known as its Andromeda supercomputer. This dramatically reduces the work of training his LLM on thousands of his Nvidia GPU chips.

again: The success of ChatGPT could have a negative impact on AI confidentiality, says AI pioneer Bengio.

As part of Tuesday’s release, Cerebras said it was the first open-source scaling rule, a benchmark rule for how such programs improve in accuracy depending on their size based on open source data. provided something.The datasets we use are open source pilean 825 gigabyte text collection of mostly professional and academic texts, introduced by a non-profit lab in 2020 Eleuther.

Previous scaling laws by OpenAI and Google’s DeepMind used non-open source training data.

Cerebras has in the past claimed the efficiency benefits of its system. The ability to efficiently train demanding natural language programs will be at the heart of the open publishing problem, Feldman said.

“If we can achieve efficiencies, we can afford to put things in the open source community,” says Feldman. “Efficiency allows us to do this quickly and easily and do our part for the community.”

He said the main reason OpenAI and other companies are starting to shut down their research to the world is because they have to protect their profit streams in the face of rising AI training costs.

again: GPT-4: New features for providing illegal advice and displaying “dangerous emergency actions”

“It was very expensive and they decided it was a strategic asset and decided to withhold it from the community because it was strategic to them,” he said. , I think it’s a very reasonable strategy.

“This is a reasonable strategy when a company invests a lot of time, effort and money and doesn’t want to share the results with the world,” Feldman added.

But “we think it makes the ecosystem less interesting, and in the long run, it limits the upward trend in research,” he said.

Companies can “stockpile” by hoarding resources such as datasets and model expertise, Feldman said.

again: AI Challenger Cerebras Builds ‘Andromeda’ Modular Supercomputer to Accelerate Large Language Models

“The question is how these resources are used strategically within the landscape,” he said. “I believe we can help by proposing an open model, with data that everyone can see.”

Asked what the open-source release product was, Feldman said, “Hundreds of individual institutions have worked on these GPT models that they might not have otherwise been able to do, and that they could have done otherwise.” It could solve a problem that might have been there,” he said.