Home » The Battle of Titans: Gemini vs GPT4 – Can Google’s Latest AI Model Live Up to the Hype?

The Battle of Titans: Gemini vs GPT4 – Can Google’s Latest AI Model Live Up to the Hype?

by OnverZe
99 comments

The competition among companies to develop advanced artificial intelligence is intense. To gain an edge against rivals like OpenAI’s GPT-4 and Meta’s Llama 2, Google has launched its new AI model called Gemini. This model has been built from scratch and is capable of handling multiple types of data at once, including text, code, audio, images, and video. The AI model will be available in three different sizes: Ultra, Pro, and Nano, for tackling complex tasks, scaling across a broad range of tasks, and on-device tasks, respectively. Gemini is the first model from Google DeepMind after the merger of the company’s AI research units, DeepMind, and Google Brain.

Late last year, OpenAI’s ChatGPT, which is powered by GPT-3.5, was launched and created quite a buzz around the world. Initially caught off guard, Google is now preparing itself for competition. It recently launched Gemini, which has made some bold claims, but questions have been raised about its accuracy. Google DeepMind claims that Gemini outperforms GPT-4 on 30 of the 32 standard performance measures, though the margins are narrow. Despite presenting a futuristic vision to the public, accuracy concerns are now at the forefront.

Gemini, a language model developed by Google, scored an impressive 90 percent on the Massive Multitask Language Understanding (MMLU) test, surpassing human experts (89.8 percent) and outperforming GPT-4 (86.4 percent) in several benchmark tests conducted by Google. MMLU uses 57 subjects, such as math, physics, history, law, medicine, and ethics, to evaluate both problem-solving abilities and world knowledge.

It’s worth noting that Google used different prompting techniques for the two models. While GPT-4’s score of 86.4 percent relied on the industry-standard “5-shot” prompting technique, Gemini Ultra’s 90 percent result was based on a different method— “chain-of-thought with 32 samples.” 

Also, it’s important to mention that Google conducted these tests using an outdated version of GPT-4, as indicated in the yellow box in the image above. The note mentions that they used a “previous state-of-the-art” (SOTA) version for GPT-4.

When the 5-shot MMLU was used to evaluate both models, GPT-4 achieved a score of 86.4 percent, surpassing Gemini Ultra’s score of 83.7 percent. Additionally, when the 10-shot HellaSwag was used to measure commonsense reasoning, GPT-4 outperformed both Gemini Ultra (87.8 percent) and Gemini Pro (84.7 percent) by scoring 95.3 percent.

In machine learning, the term “shot” indicates the number of instances or examples used during training. For instance, in few-shot learning, the model is trained on a small number of examples per class, and the number, such as 5 shots, indicates that the model is trained with only five instances of each class.

Chain of thought (CoT) refers to the logical progression or sequence of steps that an AI model follows to arrive at a decision or output. In other words, CoT prompting involves guiding the model to think step by step before generating an answer.

Google used different prompting techniques for benchmarks like GSM8K, DROP, and HumanEval. Their Gemini Ultra model achieved the highest accuracy by using a chain-of-thought prompting approach. This involves generating a sequence of responses using multiple samples and considering model uncertainty. If a consensus is reached, it selects that answer, otherwise, it resorts to a greedy sample.

What the heck?

Did you know that Google used different techniques to test their AI’s ability to reason in different subjects like math, reading comprehension, and coding? The best-performing approach was the “chain-of-thought” method used in their Gemini Ultra model. This approach generates a sequence of responses using multiple samples and considers model uncertainty before making a decision. If there’s a clear consensus, it selects that answer, otherwise, it uses a greedy approach to find the best answer. Pretty cool, huh?

The op-ed by Olson clarified that the voice in the demo was simply reciting predetermined prompts that humans had fed into Gemini, while also displaying static images. It is important to note that this is different from what Google appeared to be suggesting, which was that individuals could engage in a seamless, real-time conversation with Gemini as it observed and responded to its surroundings. In the demo video, Google had reduced latency and shortened Gemini’s outputs for the sake of brevity. Oriol Vinyals, Google DeepMind’s VP of Research and co-lead of Gemini stated in an X post that he was pleased with the attention garnered by the “Hands-on with Gemini” video. Vinyals confirmed that all of the user prompts and outputs shown in the video were authentic, but abbreviated for the sake of concision. The video served as an example of what multimodal user experiences developed with Gemini might look like and was intended to inspire developers.

You may also like

99 comments

lottery February 27, 2024 - 03:03 Reply
phising March 3, 2024 - 14:00

wow, amazing

Reply
scam March 3, 2024 - 18:59

Spectacular, keep it up

Reply
lose money March 3, 2024 - 23:36

Spectacular, keep it up

Reply
phising March 5, 2024 - 03:11

Outstanding, superb effort

Reply
scam March 6, 2024 - 03:42

nice content!nice history!! boba 😀

Reply
scam March 6, 2024 - 15:39

Terrific, continue

Reply
scam March 6, 2024 - 22:16

nice content!nice history!! boba 😀

Reply
scam March 8, 2024 - 05:11

nice content!nice history!! boba 😀

Reply
phising March 8, 2024 - 09:15

wow, amazing

Reply
lose money March 12, 2024 - 10:05

nice content!nice history!! boba 😀

Reply
scam March 16, 2024 - 09:33

nice content!nice history!! boba 😀

Reply
PHISING March 17, 2024 - 01:42

nice content!nice history!! boba 😀

Reply
SCAM March 18, 2024 - 01:47

nice content!nice history!! boba 😀

Reply
LOSE MONEY March 18, 2024 - 04:39

wow, amazing

Reply
LOSE MONEY March 18, 2024 - 05:10

wow, amazing

Reply
SCAM March 19, 2024 - 09:38

nice content!nice history!! boba 😀

Reply
sekret-natury.pl March 19, 2024 - 12:32

Outstanding feature

Reply
LOSE MONEY March 19, 2024 - 14:09

wow, amazing

Reply
LOSE MONEY March 20, 2024 - 05:03

wow, amazing

Reply
PHISING March 21, 2024 - 03:32

wow, amazing

Reply
LOSE MONEY March 21, 2024 - 07:06

nice content!nice history!! boba 😀

Reply
LOSE MONEY March 22, 2024 - 02:10

wow, amazing

Reply
LOSE MONEY March 23, 2024 - 14:16

Fantastic job

Reply
SCAM March 23, 2024 - 15:01

Splendid, excellent work

Reply
LOSE MONEY March 23, 2024 - 15:03

Excellent effort

Reply
PHISING March 23, 2024 - 15:25

Super, fantastic

Reply
SCAM March 23, 2024 - 15:28

Incredible, well done

Reply
PHISING March 25, 2024 - 03:15

wow, amazing

Reply
SCAM March 25, 2024 - 17:19

wow, amazing

Reply
lose money March 27, 2024 - 05:25

nice content!nice history!! boba 😀

Reply
lose money March 27, 2024 - 06:29

nice content!nice history!! boba 😀

Reply
lose money March 28, 2024 - 07:16

wow, amazing

Reply
lose money March 28, 2024 - 09:35

nice content!nice history!! boba 😀

Reply
phising March 29, 2024 - 09:18

nice content!nice history!! boba 😀

Reply
scam March 29, 2024 - 13:35

hello

Reply
phising March 31, 2024 - 05:11

wow, amazing

Reply
phising March 31, 2024 - 15:00

wow, amazing

Reply
phising April 1, 2024 - 11:17

nice content!nice history!! boba 😀

Reply
criminal April 3, 2024 - 09:51

lost money

Reply
lose money April 3, 2024 - 13:53

scam

Reply
scam April 3, 2024 - 14:00

lost money

Reply
scam April 3, 2024 - 14:06

lost money

Reply
lose money April 4, 2024 - 06:27

lost money

Reply
criminal April 4, 2024 - 10:08

phising

Reply
lose money April 4, 2024 - 10:12

lost money

Reply
criminal April 4, 2024 - 10:13

phising

Reply
lose money April 5, 2024 - 06:45

phising

Reply
scam April 5, 2024 - 06:45

phising

Reply
scam April 5, 2024 - 06:45

scam

Reply
scam April 5, 2024 - 06:49

lost money

Reply
criminal April 7, 2024 - 02:00

phising

Reply
scam April 7, 2024 - 06:07

lost money

Reply
Zobacz jak April 8, 2024 - 02:29

Outstanding feature

Reply
criminal April 8, 2024 - 08:12

phising

Reply
lose money April 8, 2024 - 11:38

lost money

Reply
criminal April 8, 2024 - 11:41

phising

Reply
criminal April 8, 2024 - 11:43

phising

Reply
criminal April 10, 2024 - 10:13

I highly advise to avoid this platform. My personal experience with it was nothing but frustration and concerns regarding scamming practices. Proceed with extreme caution, or better yet, look for an honest platform to fulfill your requirements.

Reply
criminal April 11, 2024 - 01:58

I highly advise to avoid this platform. The experience I had with it was only frustration along with suspicion of fraudulent activities. Proceed with extreme caution, or even better, look for an honest platform for your needs.

Reply
lose money April 11, 2024 - 09:00

I urge you steer clear of this platform. The experience I had with it has been only frustration as well as suspicion of fraudulent activities. Proceed with extreme caution, or better yet, seek out a more reputable platform to meet your needs.

Reply
criminal April 11, 2024 - 09:06

I urge you to avoid this platform. My personal experience with it has been purely frustration and concerns regarding fraudulent activities. Proceed with extreme caution, or alternatively, look for a trustworthy site for your needs.

Reply
criminal April 13, 2024 - 02:03

I highly advise to avoid this platform. My personal experience with it was only disappointment and suspicion of scamming practices. Exercise extreme caution, or even better, find an honest service to fulfill your requirements.

Reply
criminal April 13, 2024 - 10:47

I urge you to avoid this site. My personal experience with it was only disappointment and concerns regarding scamming practices. Exercise extreme caution, or better yet, look for a more reputable site for your needs.

Reply
lose money April 14, 2024 - 02:44

I urge you to avoid this site. My own encounter with it was purely disappointment along with concerns regarding scamming practices. Be extremely cautious, or alternatively, look for a trustworthy service to fulfill your requirements.

Reply
lose money April 14, 2024 - 04:19

I strongly recommend to avoid this site. My personal experience with it was nothing but disappointment and concerns regarding deceptive behavior. Be extremely cautious, or even better, find an honest site for your needs.

Reply
scam April 14, 2024 - 06:19

I highly advise to avoid this platform. My personal experience with it has been only dismay along with suspicion of scamming practices. Proceed with extreme caution, or alternatively, find a more reputable service for your needs.

Reply
lose money April 15, 2024 - 02:00

I strongly recommend steer clear of this site. My own encounter with it was only disappointment as well as suspicion of fraudulent activities. Exercise extreme caution, or even better, find an honest site for your needs.

Reply
criminal April 15, 2024 - 06:12

I highly advise stay away from this platform. My own encounter with it has been only dismay and suspicion of fraudulent activities. Exercise extreme caution, or alternatively, look for an honest service to fulfill your requirements.

Reply
criminal April 16, 2024 - 05:11

I strongly recommend to avoid this site. The experience I had with it was only frustration as well as doubts about fraudulent activities. Exercise extreme caution, or better yet, look for a trustworthy platform to meet your needs.

Reply
criminal April 16, 2024 - 07:04

I strongly recommend to avoid this site. The experience I had with it was purely dismay along with concerns regarding deceptive behavior. Proceed with extreme caution, or better yet, find an honest platform for your needs.

Reply
criminal April 17, 2024 - 05:19

I highly advise to avoid this platform. The experience I had with it was purely frustration as well as suspicion of deceptive behavior. Exercise extreme caution, or alternatively, look for an honest platform for your needs.

Reply
Palmer Recupero April 27, 2024 - 10:18

Thanks for every other informative web site. Where else may just I am getting that type of information written in such an ideal manner? I’ve a mission that I’m simply now working on, and I have been at the look out for such info.

Reply
criminal April 27, 2024 - 19:32

I strongly recommend to avoid this platform. The experience I had with it was nothing but dismay as well as suspicion of fraudulent activities. Proceed with extreme caution, or even better, look for a trustworthy service for your needs.I urge you steer clear of this site. The experience I had with it has been purely disappointment as well as doubts about fraudulent activities. Proceed with extreme caution, or alternatively, look for a more reputable platform to meet your needs.

Reply
herpagreens May 6, 2024 - 15:17

HerpaGreens is a novel dietary supplement created to help destroy the herpes simplex virus

Reply
criminal May 6, 2024 - 19:43

I strongly recommend steer clear of this site. My personal experience with it has been nothing but disappointment and suspicion of scamming practices. Proceed with extreme caution, or alternatively, find a trustworthy service to fulfill your requirements.

Reply
Lift Detox Black May 7, 2024 - 05:26

I like forgathering useful info, this post has got me even more info! .

Reply
hire a hacker uk May 8, 2024 - 06:03

Thank you a lot for giving everyone an extremely pleasant opportunity to check tips from this blog. It is usually so beneficial plus jam-packed with a great time for me personally and my office colleagues to search your web site really three times in one week to find out the new tips you will have. And of course, I’m at all times impressed with all the special principles you give. Certain 1 tips in this posting are certainly the most effective I’ve ever had.

Reply
criminal May 8, 2024 - 17:53

I strongly recommend to avoid this site. My own encounter with it has been nothing but dismay and doubts about deceptive behavior. Proceed with extreme caution, or even better, find a trustworthy platform to meet your needs.

Reply
criminal May 9, 2024 - 23:23

I highly advise stay away from this platform. My own encounter with it has been purely dismay along with doubts about scamming practices. Exercise extreme caution, or better yet, look for an honest site to meet your needs.

Reply
criminal May 11, 2024 - 19:20

I strongly recommend stay away from this platform. My personal experience with it has been purely dismay along with doubts about scamming practices. Proceed with extreme caution, or even better, seek out an honest site to fulfill your requirements.

Reply
sugar defender reviews May 12, 2024 - 04:48

What Is Sugar Defender?Sugar Defender is a new blood sugar-balancing formula that has been formulated using eight clinically proven ingredients that work together to balance sugar levels.

Reply
zencortex reviews May 12, 2024 - 12:33

What is ZenCortex? ZenCortex is a cutting-edge dietary supplement meticulously crafted to provide essential nutrients that support and enhance healthy hearing

Reply
lose money May 12, 2024 - 20:01

I highly advise stay away from this platform. My personal experience with it was purely dismay as well as doubts about deceptive behavior. Exercise extreme caution, or even better, look for a trustworthy service to meet your needs.

Reply
Gluco Freedom May 13, 2024 - 06:21

I do accept as true with all the ideas you have offered on your post. They are really convincing and will definitely work. Nonetheless, the posts are very brief for starters. Could you please lengthen them a bit from subsequent time? Thank you for the post.

Reply
this link guys, and stop the previous attack, we had one of our landing page in the previous links I provide May 13, 2024 - 19:04

I strongly recommend stay away from this site. My own encounter with it has been purely disappointment as well as doubts about scamming practices. Be extremely cautious, or even better, find a trustworthy platform for your needs.

Reply
this link guys, and stop the previous attack, we had one of our landing page in the previous links I provide May 14, 2024 - 11:19

I strongly recommend stay away from this platform. My personal experience with it has been only frustration along with suspicion of scamming practices. Proceed with extreme caution, or alternatively, seek out a more reputable service to fulfill your requirements.

Reply
this link guys, and stop the previous attack, we had one of our landing page in the previous links I provide May 14, 2024 - 23:08

I strongly recommend stay away from this platform. My own encounter with it has been purely frustration as well as doubts about scamming practices. Exercise extreme caution, or better yet, find an honest platform to meet your needs.

Reply
Sugar defender May 15, 2024 - 09:31

I loved as much as you will receive carried out right here. The sketch is attractive, your authored material stylish. nonetheless, you command get bought an shakiness over that you wish be delivering the following. unwell unquestionably come further formerly again as exactly the same nearly very often inside case you shield this hike.

Reply
Ikaria Lean Belly Juice reviews May 15, 2024 - 13:45

This is a topic close to my heart cheers, where are your contact details though?

Reply
Kerassentials review May 15, 2024 - 18:17

This website is my inhalation, rattling good layout and perfect content.

Reply
lose money May 16, 2024 - 05:54

I urge you stay away from this site. My personal experience with it was purely frustration as well as suspicion of deceptive behavior. Proceed with extreme caution, or even better, look for an honest site to fulfill your requirements.

Reply
lottery defeater reviews May 16, 2024 - 17:40

Lottery Defeater is an automated, plug-and-plug lottery-winning software.

Reply
leanbiome reviews May 16, 2024 - 23:42

What is Leanbiome? LeanBiome is a dietary supplement that is formulated with nine critically-researched lean bacteria species.

Reply
cbd online shop May 17, 2024 - 00:19

I gotta bookmark this site it seems extremely helpful extremely helpful

Reply
scam May 18, 2024 - 05:05

I urge you steer clear of this site. The experience I had with it was nothing but frustration along with concerns regarding fraudulent activities. Proceed with extreme caution, or alternatively, find a more reputable platform to meet your needs.

Reply
porn videos May 20, 2024 - 16:05

I urge you steer clear of this site. My personal experience with it was nothing but disappointment and suspicion of fraudulent activities. Proceed with extreme caution, or even better, find a more reputable site to meet your needs.

Reply
scam May 20, 2024 - 21:55

I urge you steer clear of this platform. My personal experience with it was nothing but disappointment along with concerns regarding deceptive behavior. Proceed with extreme caution, or alternatively, seek out a trustworthy site to fulfill your requirements.

Reply
Fitspresso reviews May 28, 2024 - 13:49

Nice weblog right here! Also your website rather a lot up fast! What web host are you using? Can I get your affiliate hyperlink to your host? I desire my website loaded up as quickly as yours lol

Reply

Leave a Comment